o ©Ì³iÕã@s2ddlZddlmZdejdejddfdd„ZdS)éN)ÚnnÚmodelÚscalerÚreturncCsBd}| ¡D]}|s|j}| |¡}|jdur|j|9_qdS)aˆ Utility to scale the gradients of a model. This is useful for gradient accumulation where we want to normalize the gradients by the total number of tokens seen. Inputs: model (nn.Module): model whose gradients should be scaled scaler (torch.Tensor): scaling factor to apply to the gradients Outputs: None (grad fields are modified in place) N)Ú parametersÚdeviceÚtoÚgrad)rrrÚp©r úS/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/training/_grad_scaler.pyÚ scale_grads s   €úr )ÚtorchrÚModuleÚTensorr r r r r Ús