pudl.analysis.ml_tools.models#
Provides tooling for developing/tracking ml models within PUDL.
The main interface from this module is the pudl_model() decorator, which
is meant to be applied to a dagster graph. This decorator will handle finding all
configuration for a model/passing configuration to dagster, creating an
ExperimentTracker for the model, and ultimately will return a graph_asset
from the model.
There are a few different ways to provide configuration for a PUDL model. First, configuration will come from default values for any dagster Config’s which are associated with op’s which make up the model graph. For more info on dagster configuration, see https://docs.dagster.io/concepts/configuration/config-schema. The next way to provide configuration is through the yaml file: pudl.package_data.settings.pudl_models.yml. Any configuration in this file should be follow dagster’s config-schema formatting, see the ferc_to_ferc entry as an example. Configuration provided this way will override any default values. The final way to provide configuration is through the dagster UI. To provide configuration this way, click Open Launchpad in the UI, and values can be edited here. This configuration will override both default values and yaml configuration, but will only be used for a single run.
Module Contents#
Functions#
|
Load model configuration from yaml file. |
|
Get default config values for model. |
|
Decorator for an ML model that will handle providing configuration to dagster. |
Attributes#
- pudl.analysis.ml_tools.models.get_yml_config(experiment_name: str) dict[source]#
Load model configuration from yaml file.
- pudl.analysis.ml_tools.models.get_default_config(model_graph: dagster.GraphDefinition) dict[source]#
Get default config values for model.
- pudl.analysis.ml_tools.models.pudl_model(asset_name: str, config_from_yaml: bool = False) dagster.AssetsDefinition[source]#
Decorator for an ML model that will handle providing configuration to dagster.