pudl.constants

A warehouse for constant values required to initilize the PUDL Database.

This constants module stores and organizes a bunch of constant values which are used throughout PUDL to populate static lists within the data packages or for data cleaning purposes.

Module Contents

Classes

Partition

Data partition.

Attributes

ENTITIES

Columns kept for either entity or annual EIA tables in the harvesting process.

WORKING_PARTITIONS

Per-dataset descriptions of what raw input data partitions can be processed.

PUDL_TABLES

Core PUDL DB tables by data source. Used to validate ETL inputs.

COLUMN_DTYPES

pudl.constants.ENTITIES :Dict[str, Tuple[List[str], List[str], List[str], Dict[str, str]]][source]

Columns kept for either entity or annual EIA tables in the harvesting process.

For each entity type (key), the ID columns, static columns, and annual columns, followed by any custom data type fixes.

The order of the entities matters. Plants must be harvested before utilities, since plant location must be removed before the utility locations are harvested.

class pudl.constants.Partition[source]

Bases: TypedDict

Data partition.

years :Tuple[int, Ellipsis][source]
year_month :str[source]
states :Tuple[str, Ellipsis][source]
pudl.constants.WORKING_PARTITIONS :Dict[str, Partition][source]

Per-dataset descriptions of what raw input data partitions can be processed.

Most of our datasets are distributed in chunks that correspond to a given year, state, or other logical partition. Not all available partitions of the raw have data have been integrated into PUDL. The sub-keys within each dataset partition dictionary refer to metadata in the data packages we have archived on Zenodo, which contain the original raw input data.

Note: ferc714 is not partitioned by year and is available only as a single file containing all data.

pudl.constants.PUDL_TABLES :Dict[str, Tuple[str]][source]

Core PUDL DB tables by data source. Used to validate ETL inputs.

pudl.constants.COLUMN_DTYPES :Dict[str, Dict[str, Any]][source]