o wi1@s<ddlZddlmZejj ddedejefddZdS) N)PytorchOptimizerModuleh㈵>lrreturnc Cs*ddlm}tjttj||dddddS)Nr)Adamg?)g?gffffff?g:0yE>)r weight_decaybetaseps) optimizer_fn) torch.optimrrunConfigrPartial)rrri/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/speechlm/recipes/optim/adam.pypytorch_adam_with_flat_lrs r)r) nemo_runr nemo.lightning.pytorch.optimrclifactoryfloatr rrrrrs