o }o™i‚ã@s,ddlmZmZGdd„deƒZeed<dS)é)ÚAVAILABLE_SCHEDULERSÚCosineAnnealingcs*eZdZdZddddœ‡fdd„ Z‡ZS)ÚCosineAnnealingExpzÀ Setting max_steps_for_lr_sched for this scheduler in the config is experimental and " not recommended. The scheduler can use max_steps automatically from " trainer.max_steps. réÿÿÿÿN)Úmin_lrÚ last_epochÚmax_steps_for_lr_schedc sBtƒjd||||dœ|¤Ž|r||_|j|j|j|_dSdS)N)Ú optimizerÚ max_stepsrr©)ÚsuperÚ__init__r Úconstant_stepsÚ warmup_stepsÚ decay_steps)Úselfr r rrrÚkwargs©Ú __class__r úe/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/nlp/parts/megatron_lr_schedulers.pyr s þzCosineAnnealingExp.__init__)Ú__name__Ú __module__Ú __qualname__Ú__doc__r Ú __classcell__r r rrrsrN)Únemo.core.optim.lr_schedulerrrrr r r rÚs