o }oi@sddlmZmZddlmZmZmZddlmZddl m Z ddl m Z ddl mZeGdddZeGd d d ZeGd d d ZeGd ddZeGdddZGdddZdS)) dataclassfield)AnyDictOptional)MISSING)config) DatasetConfig) exp_managerc@s2eZdZUeZeed<dZeed<dZ e ed<dS) SchedConfignamegmin_lr last_epochN) __name__ __module__ __qualname__rr str__annotations__r floatrintrrL/home/ubuntu/.local/lib/python3.10/site-packages/nemo/core/config/modelPT.pyr s  r c@s*eZdZUeZeed<dZee ed<dS) OptimConfigr Nsched) rrrrr rrrrr rrrrr s rc@sReZdZUdZdZeeed<dZeeed<dZ eeed<dZ ee ed<dS) ModelConfigz( Model component inside ModelPT Ntrain_ds validation_dstest_dsoptim) rrr__doc__rrr rrrrrrrrrr&s rc@sJeZdZUedddZeeefed<edddZ eeefed<dS) HydraConfigcCsddiS)Ndir.rrrrr5zHydraConfig.default_factoryruncCs dddiiS)Nroothandlersrrrrrr$6s job_loggingN) rrrrr(rrrrr+rrrrr!3s "r!c@sneZdZUeZeed<eZeed<e dddZ e j ed<e dddZ eeed<e d ddZeed <d S) NemoConfigr modelcCstjddddddS)NddpFgpu)strategyenable_checkpointingloggerlog_every_n_steps accelerator)r TrainerConfigrrrrr$>s zNemoConfig.r&trainercCstSN)r ExpManagerConfigrrrrr$Br%r cCstSr8)r!rrrrr$CshydraN)rrrrr rrr-rrr7rr6r rrr:r!rrrrr,9s  r,c@seZdZdefddZddeefddZddeefdd Zddeefd d Z dde j d ee j fd dZ ddZdefddZdS)ModelConfigBuilder model_cfgcCs"||_d|_d|_d|_d|_dS)a Base class for any Model Config Builder. A Model Config Builder is a utility class that accepts a ModelConfig dataclass, and via a set of utility methods (that are implemented by the subclassed ModelConfigBuilder), builds a finalized ModelConfig that can be supplied to a NemoModel dataclass as the `model` component. Subclasses *must* implement the private method `_finalize_cfg`. Inside this method, they must update `self.model_cfg` with all interdependent config options that need to be set (either updated by user explicitly or with their default value). The updated model config must then be preserved in `self.model_cfg`. Example: # Create the config builder config_builder = ModelConfigBuilder() # Update the components of the config that are modifiable config_builder.set_X(X) config_builder.set_Y(Y) # Create a "finalized" config dataclass that will contain all the updates # that were specified by the builder model_config = config_builder.build() # Use model config as is (or further update values), then create a new Model model = nemo..models.Model(cfg=model_config, trainer=Trainer()) Supported build methods: - set_train_ds: All model configs can accept a subclass of `DatasetConfig` as their training config. Subclasses can override this method to enable auto-complete by replacing `Optional[DatasetConfig]` with `Optional[]`. - set_validation_ds: All model configs can accept a subclass of `DatasetConfig` as their validation config. Subclasses can override this method to enable auto-complete by replacing `Optional[DatasetConfig]` with `Optional[]`. - set_test_ds: All model configs can accept a subclass of `DatasetConfig` as their test config. Subclasses can override this method to enable auto-complete by replacing `Optional[DatasetConfig]` with `Optional[]`. - set_optim: A build method that supports changes to the Optimizer (and optionally, the Scheduler) used for training the model. The function accepts two inputs - `cfg`: A subclass of `OptimizerParams` - any OptimizerParams subclass can be used, in order to select an appropriate Optimizer. Examples: AdamParams. `sched_cfg`: A subclass of `SchedulerParams` - any SchedulerParams subclass can be used, in order to select an appropriate Scheduler. Examples: CosineAnnealingParams. Note that this argument is optional. - build(): The method which should return a "finalized" ModelConfig dataclass. Subclasses *should* always override this method, and update the signature of this method with the return type of the Dataclass, so that it enables autocomplete for the user. Example: def build(self) -> EncDecCTCConfig: return super().build() Any additional build methods must be added by subclasses of ModelConfigBuilder. Args: model_cfg: N)r< train_ds_cfgvalidation_ds_cfg test_ds_cfg optim_cfg)selfr<rrr__init__Gs C zModelConfigBuilder.__init__NcfgcC ||j_dSr8)r<rrArCrrr set_train_ds zModelConfigBuilder.set_train_dscCrDr8)r<rrErrrset_validation_dsrGz$ModelConfigBuilder.set_validation_dscCrDr8)r<rrErrr set_test_dsrGzModelConfigBuilder.set_test_ds sched_cfgc CstGdddt|j}|jjdd}|d |ddt|}|durHtGdddt|j}|jjdd}|d d|it|}||_||j _ dS) Nc@ eZdZdS)z8ModelConfigBuilder.set_optim..WrappedOptimConfigNrrrrrrrWrappedOptimConfigrMParams)r rc@rK)z8ModelConfigBuilder.set_optim..WrappedSchedConfigNrLrrrrWrappedSchedConfigrNrQr r) rr __class__rreplacelowervarsr rr<r) rArCrJrM optim_name wrapped_cfgrQ sched_namewrapped_sched_cfgrrr set_optims zModelConfigBuilder.set_optimcCstr8)NotImplementedErrorrArrr _finalize_cfgsz ModelConfigBuilder._finalize_cfgreturncCs||jSr8)r]r<r\rrrbuildszModelConfigBuilder.buildr8)rrrrrBrr rFrHrIrOptimizerParamsSchedulerParamsrZr]r_rrrrr;FsIr;N) dataclassesrrtypingrrr omegaconfr nemo.corernemo.core.classes.datasetr nemo.utilsr r rrr!r,r;rrrrs"