pudl.extract.xbrl#

Generic extractor for all FERC XBRL data.

Module Contents#

Classes#

FercXbrlDatastore

Simple datastore wrapper for accessing ferc1 xbrl resources.

Functions#

_get_sqlite_engine(→ sqlalchemy.engine.Engine)

Create SQLite engine for specified form and drop tables.

xbrl2sqlite(→ None)

Clone the FERC Form 1 XBRL Databsae to SQLite.

convert_form(→ None)

Clone a single FERC XBRL form to SQLite.

Attributes#

logger

pudl.extract.xbrl.logger[source]#
class pudl.extract.xbrl.FercXbrlDatastore(datastore: pudl.workspace.datastore.Datastore)[source]#

Simple datastore wrapper for accessing ferc1 xbrl resources.

get_taxonomy(year: int, form: pudl.settings.XbrlFormNumber) → tuple[io.BytesIO, str][source]#

Returns the path to the taxonomy entry point within the an archive.

get_filings(year: int, form: pudl.settings.XbrlFormNumber) → list[ferc_xbrl_extractor.instance.InstanceBuilder][source]#

Return list of filings from archive.

pudl.extract.xbrl._get_sqlite_engine(form_number: int, output_path: pathlib.Path, clobber: bool) → sqlalchemy.engine.Engine[source]#

Create SQLite engine for specified form and drop tables.

Parameters:

  • form_number – FERC form number.

  • output_path – path to PUDL outputs.

  • clobber – Flag indicating whether or not to drop tables.

pudl.extract.xbrl.xbrl2sqlite(context) → None[source]#

Clone the FERC Form 1 XBRL Databsae to SQLite.

pudl.extract.xbrl.convert_form(form_settings: pudl.settings.FercGenericXbrlToSqliteSettings, form: pudl.settings.XbrlFormNumber, datastore: FercXbrlDatastore, sqlite_engine: sqlalchemy.engine.Engine, output_path: pathlib.Path, batch_size: int | None = None, workers: int | None = None) → None[source]#

Clone a single FERC XBRL form to SQLite.

Parameters:

  • form_settings – Validated settings for converting the desired XBRL form to SQLite.

  • form – FERC form number.

  • datastore – Instance of a FERC XBRL datastore for retrieving data.

  • sqlite_engine – SQLAlchemy connection to mirrored database.

  • pudl_settings – Dictionary containing paths and database URLs used by PUDL.

  • batch_size – Number of XBRL filings to process in a single CPU process.

  • workers – Number of CPU processes to create for processing XBRL filings.

Returns:

None