o }oi 7@sZddlmZddlmZmZmZmZddlZddlm Z ddl m m Z ddl mZddlmZddlmZddlmZddlmZdd lmZmZed \ZZed d ed \ZZerbdd lmZGdddeedZGdddeZ GdddeZ!GdddeZ"GdddeZ#GdddeZ$Gdddej%j&Z'ej(j)j*ej(j+j,fddZ-dS))ABCMeta) TYPE_CHECKINGDictOptionalTupleN)parallel_state)MegatronModule)Tensor)_Loss)logging) safe_importsafe_import_fromzmodelopt.torch.distillDistillationLossBalancer)alt)TransformerConfigc sleZdZdZddddeejffdd Zded ed e eeffd d Z ddede de d efddZ Z S)BaseLossz5Abstract base class for Megatron distillation losses.N model_configrprojection_layerct||_||_dS) Constructor. Args: model_config: MCore transformer config. projection_layer: Module which projects student activations to teacher's hidden dim. N)super__init___config _projectionselfrr __class__^/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/modelopt/distill/loss.pyr's  zBaseLoss.__init__ predictionstargetsreturncCs@t|tr|d|d}}|jdur||}|}||fS)zKPerforms projection of student tensor to match teacher's size if necessary.rN) isinstancetuplerdetach)rr r!rrr pre_forward3s   zBaseLoss.pre_forwardFloss tp_reduceis_sequence_parallelcCs|dd}|||fS)z@Reshapes tensor from [s, b] to [b, s] for upcoming loss masking.r) transpose contiguous)rr'r(r)rrr post_forward?s zBaseLoss.post_forwardN)FF)__name__ __module__ __qualname____doc__rnnModulerr rr&boolr- __classcell__rrrrr$s  $ r) metaclassc@s&eZdZdZdededefddZdS)MSELosszJCalculates MSE loss between two tensors without reducing the sequence dim.r r!r"cCs6|||\}}tj||dd}|jdd}||S)zForward function. Args: predictions: Student model tensors (size [s, b, h]) targets: Teacher model tensors (size [s, b, h]) Returns: MSE loss of tensors (size [b, s]) none reductiondim)r&Fmse_losssumr-rr r!r'rrrforwardHs   zMSELoss.forwardN)r/r0r1r2r rCrrrrr8Esr8csHeZdZdZd dddeejffdd Zded ed efd d Z Z S)HiddenStateCosineLossz Calculates Cosine loss between two tensors without reducing the sequence dim. The tensors are assumed to be intermediate activations, so extra restrictions are in place. Nrrrcs:tj||d|jjdkr|jjstddSdSdS)r)rr*z``HiddenStateCosineLoss`` only works with tensors with full hidden dim. Ensure the tensor inputs meet this requirement or use `--sequence_parallel` if tensor parallel is enabled.N)rrrtensor_model_parallel_sizesequence_parallelr warningrrrrras zHiddenStateCosineLoss.__init__r r!r"cCsj|||\}}tj|d|d|d|d|ddd}|j|jdd}|j||jj dS)z Forward function. Args: predictions: Student model tensors (size [s, b, h]) targets: Teacher model tensors (size [s, b, h]) Returns: Cosine loss of tensors (size [b, s]) r<r*r9r:N)r)) r&r?cosine_embedding_lossviewsizenew_onesshaper-rrFrBrrrrCqs zHiddenStateCosineLoss.forwardr.) r/r0r1r2rr3r4rr rCr6rrrrrDZsrDcsFeZdZdZddddedeffdd Zd ed ed efd dZZ S) LogitsKLLossz[Calculates KL-Divergence loss between two logits tensors without reducing the sequence dim.?Frr temperaturereversecst|||_||_dS)aConstructor. Args: model_config: MCore transformer config. temperature: Divide tensors by this value prior to calculating loss. reverse: Whether to reverse the loss as KLD(teacher, student) instead of KLD(student, teacher) N)rr _temperature_reverse)rrrPrQrrrrs  zLogitsKLLoss.__init__r r!r"cCs|||\}}||j}||j}|jjdkrtj|dd\}}tjj|tjj j t d||j dd}tjt|dd}t|t d}tj|dd\}}tjj|tjj j t d||j dd}tjt|dd} t| t d} |j\} } } |t| | | d| | | } |t|| | d| | | }|jrtjtj|| ddddd}nAtjtj| |ddddd}n2|jrtjtjtj|ddtj|dddd dd}ntjtjtj|ddtj|dddd dd}|j|dd S) zForward function. Args: predictions: Student model tensors (size [s, b, h]) targets: Teacher model tensors (size [s, b, h]) Returns: KLD loss of tensors (size [b, s]) r*r<r=opgroup)rVr9T)r; log_targetr:)r()r&floatrRrrEtorchmax distributed all_reduceReduceOpMAXrget_tensor_model_parallel_group unsqueezerAexpall_reduce_autogradr%rMlogrJexpandrSr?kl_div log_softmaxsoftmaxr-)rr r!output_teacheroutput_studentteacher_logits_max_ denom_teacherstudent_logits_max denom_studentslenbszsharded_vocab_sizestudent_log_probteacher_log_probr'rrrrCsp         zLogitsKLLoss.forwardrOF) r/r0r1r2rXr5rr rCr6rrrrrNs rNcsFeZdZdZd dedeffdd Zdeee fd e fd d Z Z S) "LogitsAndIntermediatesLossBalancerz LossBalancer implementation for Logit and Intermediate losses. Dynamically weighs distillation and original losses to balance during training. rOF kd_loss_scaleskip_original_losscr)ayConstructor. Args: kd_loss_scale: Multiply distillation losses by this before weighing. (Not used when `skip_original_loss` is True.) skip_original_loss: Used to signal whether the original loss should be used, regardless of whether it was passed into ``mtd.DistillationModel.compute_kd_loss()`` or not. N)rr_kd_loss_scale_skip_original_loss)rrvrwrrrrs  z+LogitsAndIntermediatesLossBalancer.__init__ loss_dictr"c Cs|tjj}|D] }|tjr|}q ||}t|t t |d}|dkr7| | }||}n| |}|}|j rF||} n||} | | | 9} || |j} | ||d} | S)zForward function. Args: loss_dict: All individual scalar losses, passed in during ``mtd.DistillationModel.compute_kd_loss()`` Returns: Aggregate total scalar loss. r*r)kd_loss logits_lossintermediate_loss)popmtdloss_balancersSTUDENT_LOSS_KEY startswithrNr/rAvaluesrZlenitem new_tensorryrx) rrz original_loss_key logits_keyr|r} dynamic_scaleintermediate_loss_scaled total_lossr{out_dictrrrrCs,      z*LogitsAndIntermediatesLossBalancer.forwardrt) r/r0r1r2rXr5rrstrr rCr6rrrrrus" rucs8eZdZdZd fdd Zdefdd Zd d ZZS) ProjectionLayerz>Module to project student layer activations to teacher's size.student_configrteacher_configcsrtj|d|j|jkrt|_dSt|j|j|_||jt |jj d|j j t |jj d|j j dS)z Constructor. Args: student_config: Student's MCore transformer config. teacher_config: Teacher's MCore transformer config. )configrFN)rr hidden_sizer3Identity_fitLinearapply _init_weightssetattrweightrrFbias)rrrrrrr-s  zProjectionLayer.__init__student_tensorcCs ||S)zp Forward function. Args: student_tensor: Tensor to be fit to teacher size. )r)rrrrrrC?s zProjectionLayer.forwardcCs>t|tjr|j|jj|jdur|jjdSdSdS)zInitialize the weights.N) r#r3rr init_methodrdatarzero_)rmodulerrrrHs  zProjectionLayer._init_weights)rrrr) r/r0r1r2rr rCrr6rrrrr*s  rc@s(eZdZdZeddZeddZdS) _AllReducez@Implementation from old PyTorch `torch.distributed.nn.parallel`.cCs,|||_|_|}tjj|||d|S)NrT)rVrUclonerYr[r\)ctxrUrVtensorrrrrCSsz_AllReduce.forwardcCsddt|j|j|fSr.)rrrUrV)r grad_outputrrrbackward[sz_AllReduce.backwardN)r/r0r1r2 staticmethodrCrrrrrrPs  rcCst|||S)zCustom all-reduce function. Needed instead of other all-reduce functions available when the computation following the all-reduce call differs per rank. In KL loss, this corresponds to the different numerators. )rr)rrUrVrrrrbasrb).abcrtypingrrrrrYtorch.nnr3torch.nn.functional functionalr? megatron.corermegatron.core.transformerrr torch.nn.modules.lossr nemo.utilsr nemo.utils.import_utilsr r rrkobjectr,megatron.core.transformer.transformer_configrrr8rDrNrurautogradFunctionrr[r]SUMrVWORLDrbrrrrs.         !0f:&