pudl.extract.xbrl#

Generic extractor for all FERC XBRL data.

Module Contents#

Classes#

FercXbrlDatastore

Simple datastore wrapper for accessing ferc1 xbrl resources.

Functions#

_get_sqlite_engine(→ sqlalchemy.engine.Engine)

Create SQLite engine for specified form and drop tables.

xbrl2sqlite([ferc_to_sqlite_settings, pudl_settings, ...])

Clone the FERC Form 1 XBRL Databsae to SQLite.

convert_form(form_settings, form, datastore, sqlite_engine)

Clone a single FERC XBRL form to SQLite.

Attributes#

logger

pudl.extract.xbrl.logger[source]#
class pudl.extract.xbrl.FercXbrlDatastore(datastore: pudl.workspace.datastore.Datastore)[source]#

Simple datastore wrapper for accessing ferc1 xbrl resources.

get_taxonomy(year: int, form: pudl.settings.XbrlFormNumber)[source]#

Returns the path to the taxonomy entry point within the an archive.

get_filings(year: int, form: pudl.settings.XbrlFormNumber)[source]#

Return list of filings from archive.

pudl.extract.xbrl._get_sqlite_engine(form_number: int, pudl_settings: dict, clobber: bool) → sqlalchemy.engine.Engine[source]#

Create SQLite engine for specified form and drop tables.

Parameters:

  • form_number – FERC form number.

  • pudl_settings – Dictionary of settings.

  • clobber – Flag indicating whether or not to drop tables.

pudl.extract.xbrl.xbrl2sqlite(ferc_to_sqlite_settings: pudl.settings.FercToSqliteSettings = FercToSqliteSettings(), pudl_settings: dict = None, clobber: bool = False, datastore: pudl.workspace.datastore.Datastore = None, batch_size: int | None = None, workers: int | None = None)[source]#

Clone the FERC Form 1 XBRL Databsae to SQLite.

Parameters:

  • ferc_to_sqlite_settings – Object containing Ferc to SQLite validated settings.

  • pudl_settings – Dictionary containing paths and database URLs used by PUDL.

  • clobber – Flag indicating whether or not to drop tables.

  • datastore – Instance of a datastore to access the resources.

  • batch_size – Number of XBRL filings to process in a single CPU process.

  • workers – Number of CPU processes to create for processing XBRL filings.

Returns:

None

pudl.extract.xbrl.convert_form(form_settings: pudl.settings.FercGenericXbrlToSqliteSettings, form: pudl.settings.XbrlFormNumber, datastore: FercXbrlDatastore, sqlite_engine: sqlalchemy.engine.Engine, pudl_settings: dict = None, batch_size: int | None = None, workers: int | None = None)[source]#

Clone a single FERC XBRL form to SQLite.

Parameters:

  • form_settings – Validated settings for converting the desired XBRL form to SQLite.

  • form – FERC form number.

  • datastore – Instance of a FERC XBRL datastore for retrieving data.

  • sqlite_engine – SQLAlchemy connection to mirrored database.

  • pudl_settings – Dictionary containing paths and database URLs used by PUDL.

  • batch_size – Number of XBRL filings to process in a single CPU process.

  • workers – Number of CPU processes to create for processing XBRL filings.

Returns:

None