o }oiW!@sddlmZddlmZddlmZmZmZeGdddZeGdddeZ eGdd d eZ eGd d d e Z eGd d d e Z eGddde Z eGddde ZeGddde ZeGddde ZeGddde ZeGddde ZeGddde ZeGddde ZeGddde Z eGd d!d!eZeGd"d#d#eZeGd$d%d%ZeGd&d'd'eZd(ed)efd*d+Zd(ed,eeeefd-efd.d/Zee e e e eee eeeeeeed0Zd1S)2) dataclass)partial)AnyDictOptionalc@eZdZUdZdZeed<dS)SchedulerParams Base configuration for all schedulers. It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). last_epochN)__name__ __module__ __qualname____doc__r int__annotations__rrO/home/ubuntu/.local/lib/python3.10/site-packages/nemo/core/config/schedulers.pyr rc@s2eZdZUdZdZeeed<dZeeed<dS)!SquareRootConstantSchedulerParamsr Nconstant_stepsconstant_ratio) r r rrrrfloatrrrrrrrs rc@s>eZdZUdZdZeed<dZee ed<dZ ee ed<dS)WarmupSchedulerParamsr r max_stepsN warmup_steps warmup_ratio) r r rrrrrrrrrrrrrr)s  rc@>eZdZUdZdZeeed<dZeeed<dZ eed<dS)WarmupHoldSchedulerParamsr N hold_steps hold_ratiomin_lr) r r rrrrrrr r"rrrrr5 rc@r)"WarmupAnnealingHoldSchedulerParamsr Nrrr!r") r r rrrrrrrr"rrrrr$Ar#r$c@r)SquareAnnealingParamsz Square Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). gh㈵>r"Nr r rrr"rrrrrrr%Mrr%c@r)SquareRootAnnealingParamsz Square Root Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). r!r"Nr&rrrrr'Wrr'c@r)CosineAnnealingParams Cosine Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). r!r"Nr&rrrrr(arr(c@r)NoamAnnealingParamsr)r!r"Nr&rrrrr*krr*c@r)NoamHoldAnnealingParams Polynomial Hold Decay Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). g? decay_rateN)r r rrr-rrrrrrr+urr+c@s"eZdZUdZdZeeed<dS)WarmupAnnealingParamsz Warmup Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). Nr)r r rrrrrrrrrrr.s r.c@seZdZdZdS) InverseSquareRootAnnealingParamsz Inverse Square Root Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). N)r r rrrrrrr/sr/c@*eZdZUdZdZeed<dZeed<dS)PolynomialDecayAnnealingParamsz Polynomial Decay Annealing parameter config It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). ?powerFcycleN r r rrr3rrr4boolrrrrr1  r1c@r0)"PolynomialHoldDecayAnnealingParamsr,r2r3Fr4Nr5rrrrr8r7r8c@s*eZdZUdZdZeed<dZeed<dS) StepLRParamsz Config for StepLR. It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). 皙? step_sizegammaN)r r rrr;rrr<rrrrr9r7r9c@r)ExponentialLRParamsz Config for ExponentialLR. It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). ?r<N)r r rrr<rrrrrrr=rr=c@s~eZdZUdZdZeed<dZeed<dZ e ed<dZ e ed <d Z eed <d Zeed <dZe ed<dZeed<dZeed<dS)ReduceLROnPlateauParamsz Config for ReduceLROnPlateau. It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). minmoder:factor patienceFverboseg-C6? thresholdrelthreshold_modercooldownr"g:0yE>epsN)r r rrrAstrrrBrrDrrEr6rFrHrIr"rJrrrrr?s         r?c@seZdZUdZdZeed<dZeed<dZe ed<dZ e e ed <d Z e ed <d Zeed <dZe ed<dZeed<dZeed<dZeed<dS)CyclicLRParamsz Config for CyclicLR. NOTE: # `scale_fn` is not supported It is not derived from Config as it is not a NeMo object (and in particular it doesn't need a name). gMbP?base_lrr:max_lri step_size_upNstep_size_down triangularrAr2r<r4 scale_modeTcycle_momentumg? base_momentumr> max_momentum)r r rrrMrrrNrOrrPrrArKr<rRrSr6rTrUrrrrrLs         rLnamescheduler_paramscCs"|tvr td||t|<dS)aF Checks if the schduler config name exists in the registry, and if it doesnt, adds it. This allows custom schedulers to be added and called by name during instantiation. Args: name: Name of the optimizer. Will be used as key to retrieve the optimizer. scheduler_params: SchedulerParams class zFCannot override pre-existing optimizers. Conflicting optimizer name = N)AVAILABLE_SCHEDULER_PARAMS ValueError)rVrWrrrregister_scheduler_paramss  rZkwargsreturncKs<|tvrtd|dtt|}t|fi|}|S)aC Convenience method to obtain a SchedulerParams class and partially instantiate it with optimizer kwargs. Args: name: Name of the SchedulerParams in the registry. kwargs: Optional kwargs of the optimizer used during instantiation. Returns: a partially instantiated SchedulerParams z%Cannot resolve scheduler parameters 'z('. Available scheduler parameters are : )rXrYkeysr)rVr[rWrrrget_scheduler_configs r^)rWarmupPolicyParamsWarmupHoldPolicyParamsr$r%r'r/rr(r*r+r.r1r8r?N) dataclassesr functoolsrtypingrrrrrrrr$r%r'r(r*r+r.r/r1r8r9r=r?rLrKrZr^rXrrrrst                 "