o }oi`&@sddlmZmZmZmZmZddlZddlmZddl m Z ddlm Z m Z ddl mZddlmZddlmZdd lmZdd lmZdd lmZd d lmZmZmZerbddlmZddlm Z ed\Z!Z"GdddeZ#Gdddej$Z%dS)) TYPE_CHECKINGCallableDictOptionalTupleN)parallel_state)TransformerConfig)Tensornn)llm))average_losses_across_data_parallel_group)MaskedTokenLossReduction)logging) safe_import) unwrap_model)#adjust_distillation_model_for_mcoreload_distillation_configteacher_provider) TokenizerSpec)OptimizerModulezmodelopt.torch.distillc s`eZdZdZfddZdeeefdedeeeeefffddZ d ed efd d Z Z S) _DistillationLossReductionzACustom masking and reduction callable used only in training mode.cs0tj|i|||_t|_t|_dSN)super__init___distillation_loss_fnrget_context_parallel_world_size_cp_size$get_tensor_model_parallel_world_size_tp_size)selfdistillation_loss_fnargskwargs __class___/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/modelopt/distill/model.pyr)s z#_DistillationLossReduction.__init__batch forward_outreturncst|tr |\}d<|d}j|fddd}t|d|d|dg}|dd |d d |d d d }|d|fS) N loss_maskcs|dS)Nr+)_masked_token_loss)xr(r r&r';sz4_DistillationLossReduction.forward..) student_lossloss_reduction_fnkd_loss logits_lossintermediate_lossrr)avgkd_logits_train_losskd_intermediate_train_loss) isinstancetupler,rr )r r(r)lm_losslosseslosses_averagedreportr&r.r'forward/s       z"_DistillationLossReduction.forward loss_outputmaskc Cst|tr |\}}}nd\}}|}|r&t}tj||jdd|}| d}| d}t||} |j dkr`t | d| dg} tj j| td| d| d} n| |} |sh|rrtj j| td| S)zIThe function takes as input per-token loss and masks non-required values.)FFr)dim)groupr)r:r;sumfloatrget_tensor_model_parallel_ranktorch tensor_splitrviewreshapercat distributed all_reduceget_context_parallel_groupget_tensor_model_parallel_group) r rArB tp_reduceis_sequence_parallelnum_valid_tokensidxr=r+loss_sumlossr&r&r'r,Hs$    z-_DistillationLossReduction._masked_token_loss) __name__ __module__ __qualname____doc__rrstrr rr@r, __classcell__r&r&r$r'r&s  .rcseZdZdZ    ddedededeededd ed d eeej gej fffd d Z ddZ de fddZ edefddZddZeddZd deffdd ZfddZZS)!DistillationGPTModelz;Custom GPT subclass for distillation-related modifications.Nconfigteacher_configteacher_ckpt_pathdistillation_config_pathoptimr tokenizerrmodel_transformcsptstdt||||||_||_||_d|_t|t j r't|t j s,t d|j jdur6tddS)aeConstructor. This subclass of GPTModel takes the configs of a student and teacher model and overrides the model construction step to create a ModelOpt `DistillationModel` as the underlying MCore model. This model abstracts both student and teacher as a single module whose forward pass runs both, and whose loss function automatically calculates a distillation loss on the output logits. NOTE: This class saves checkpoints which will be re-loaded as the student's original class. This allows one to continue using the model after distillation without this special class. Args: config: Config of student model. teacher_config: Config of teacher model. teacher_ckpt_path: Path to teacher checkpoint (to restore weights). distillation_config_path: Path to distillation config YAML file. If not provided, by default will perform logits-only distillation. optim: Optimizer. tokenizer: Tokenizer. model_transform: Transform to apply to model during setup. z5nvidia-modelopt is needed to use DistillationGPTModelFz{Student and Teacher configs should both inherit from llm.GPTConfig. Configs may not work properly with DistillationGPTModelNzFModelOpt Distillation incompatible with interleaved pipeline schedule.) HAVE_MODELOPT RuntimeErrorrr_teacher_config_teacher_ckpt_path_distillation_config_path _train_calledr:r GPTConfigrwarningr_$virtual_pipeline_model_parallel_size ValueError)r r_r`rarbrcrdrer$r&r'rjs zDistillationGPTModel.__init__cCst|drdS|j|j}dD] }t|j|t|j|qt|j|j|j}t |j|j g|j|j df|j |j d}tj|d|fgd}t||d||_dS)Nmodule)tensor_model_parallel_sizepipeline_model_parallel_sizecontext_parallel_sizesequence_parallelpipeline_dtype)rdtrainer) teacher_model criterion loss_balancerr2)mode) distill_cfg)hasattrr_configure_modelrdsetattrrhgetattrrrjrrirvrxrymtdconvertrrp)r modelattrr{ kd_configdistillation_modelr&r&r'r}s     z$DistillationGPTModel.configure_modelr*cOstd)Nz_Please restore a checkpoint of this model to its original class to call `get_inference_wrapper`)NotImplementedError)r r"r#r&r&r'get_inference_wrappersz*DistillationGPTModel.get_inference_wrappercCs|js t|jjd|_|jS)N)r!)_training_loss_reductionr core_modulecompute_kd_lossr r&r&r'training_loss_reductions z,DistillationGPTModel.training_loss_reductioncOs,dd|D}|jj|g|Ri|S)NcSsi|] \}}|dd|qS)zmodule.)replace).0kvr&r&r' sz8DistillationGPTModel.load_state_dict..)itemsrload_state_dict)r state_dictr"r#r&r&r'rsz$DistillationGPTModel.load_state_dictcCs t|jSr)rrprr&r&r'rs z DistillationGPTModel.core_moduleTrzcsd|_t|S)NT)rkrtrain)r rzr$r&r'rs zDistillationGPTModel.traincs0|dkr|js||dSd|_t||S)NtrainingF)rkrr __setattr__)r namevaluer$r&r'rs  z DistillationGPTModel.__setattr__)NNNN)T)rXrYrZr[rr\rrr Modulerr}r rpropertyrrrrboolrrr]r&r&r$r'r^gs</" r^)&typingrrrrrrI megatron.corermegatron.core.transformerrr r nemo.collectionsr 2nemo.collections.nlp.modules.common.megatron.utilsr nemo.lightning.megatron_parallelr nemo.utilsrnemo.utils.import_utilsrnemo.utils.model_utilsrutilsrrr1nemo.collections.common.tokenizers.tokenizer_specrnemo.lightning.pytorch.optimrrrfrGPTModelr^r&r&r&r's$           A