o wi%@sddlmZmZmZmZmZddlZddlmZddl m Z ddlm Z m Z ddl mZddlmZddlmZdd lmZdd lmZd d lmZmZmZer\dd lmZddlmZddlmm Z!GdddeZ"Gdddej#Z$dS)) TYPE_CHECKINGCallableDictOptionalTupleN)parallel_state)TransformerConfig)Tensornn)llm))average_losses_across_data_parallel_group)MaskedTokenLossReduction)logging) unwrap_model)#adjust_distillation_model_for_mcoreload_distillation_configteacher_provider) TokenizerSpec)OptimizerModulec s`eZdZdZfddZdeeefdedeeeeefffddZ d ed efd d Z Z S) _DistillationLossReductionzACustom masking and reduction callable used only in training mode.cs0tj|i|||_t|_t|_dSN)super__init___distillation_loss_fnrget_context_parallel_world_size_cp_size$get_tensor_model_parallel_world_size_tp_size)selfdistillation_loss_fnargskwargs __class__h/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/collections/llm/modelopt/distill/model.pyr(s z#_DistillationLossReduction.__init__batch forward_outreturncst|tr |\}d<|d}j|fddd}t|d|d|dg}|dd |d d |d d d }|d|fS) N loss_maskcs|dS)Nr*)_masked_token_loss)xr'rr%r&:sz4_DistillationLossReduction.forward..) student_lossloss_reduction_fnkd_loss logits_lossintermediate_lossrr)avgkd_logits_train_losskd_intermediate_train_loss) isinstancetupler+rr )rr'r(lm_losslosseslosses_averagedreportr%r-r&forward.s       z"_DistillationLossReduction.forward loss_outputmaskc Cst|tr |\}}}nd\}}|}|r&t}tj||jdd|}| d}| d}t||} |j dkr`t | d| dg} tj j| td| d| d} n| |} |sh|rrtj j| td| S)zIThe function takes as input per-token loss and masks non-required values.)FFr)dim)groupr)r9r:sumfloatrget_tensor_model_parallel_ranktorch tensor_splitrviewreshapercat distributed all_reduceget_context_parallel_groupget_tensor_model_parallel_group) rr@rA tp_reduceis_sequence_parallelnum_valid_tokensidxr<r*loss_sumlossr%r%r&r+Gs$    z-_DistillationLossReduction._masked_token_loss) __name__ __module__ __qualname____doc__rrstrr rr?r+ __classcell__r%r%r#r&r%s  .rcseZdZdZ    ddedededeededd ed d eeej gej fffd d Z ddZ de fddZ edefddZddZeddZd deffdd ZfddZZS)!DistillationGPTModelz;Custom GPT subclass for distillation-related modifications.Nconfigteacher_configteacher_ckpt_pathdistillation_config_pathoptimr tokenizerrmodel_transformcsdt||||||_||_||_d|_t|tjr!t|tjs&t d|j j dur0t ddS)aeConstructor. This subclass of GPTModel takes the configs of a student and teacher model and overrides the model construction step to create a ModelOpt `DistillationModel` as the underlying MCore model. This model abstracts both student and teacher as a single module whose forward pass runs both, and whose loss function automatically calculates a distillation loss on the output logits. NOTE: This class saves checkpoints which will be re-loaded as the student's original class. This allows one to continue using the model after distillation without this special class. Args: config: Config of student model. teacher_config: Config of teacher model. teacher_ckpt_path: Path to teacher checkpoint (to restore weights). distillation_config_path: Path to distillation config YAML file. If not provided, by default will perform logits-only distillation. optim: Optimizer. tokenizer: Tokenizer. model_transform: Transform to apply to model during setup. Fz{Student and Teacher configs should both inherit from llm.GPTConfig. Configs may not work properly with DistillationGPTModelNzFModelOpt Distillation incompatible with interleaved pipeline schedule.)rr_teacher_config_teacher_ckpt_path_distillation_config_path _train_calledr9r GPTConfigrwarningr^$virtual_pipeline_model_parallel_size ValueError)rr^r_r`rarbrcrdr#r%r&ris zDistillationGPTModel.__init__cCst|drdS|j|j}dD] }t|j|t|j|qt|j|j|j}t |j|j g|j|j df|j |j d}tj|d|fgd}t||d||_dS)Nmodule)tensor_model_parallel_sizepipeline_model_parallel_sizecontext_parallel_sizesequence_parallelpipeline_dtype)rctrainer) teacher_model criterion loss_balancerr1)mode) distill_cfg)hasattrr^configure_modelrcsetattrregetattrrrgrrfrsrurvmtdconvertrrm)rmodelattrrx kd_configdistillation_modelr%r%r&rzs     z$DistillationGPTModel.configure_modelr)cOstd)Nz_Please restore a checkpoint of this model to its original class to call `get_inference_wrapper`)NotImplementedError)rr!r"r%r%r&get_inference_wrappersz*DistillationGPTModel.get_inference_wrappercCs|js t|jjd|_|jS)N)r )_training_loss_reductionr core_modulecompute_kd_lossrr%r%r&training_loss_reductions z,DistillationGPTModel.training_loss_reductioncOs,dd|D}|jj|g|Ri|S)NcSsi|] \}}|dd|qS)zmodule.)replace).0kvr%r%r& sz8DistillationGPTModel.load_state_dict..)itemsrload_state_dict)r state_dictr!r"r%r%r&rsz$DistillationGPTModel.load_state_dictcCs t|jSr)rrmrr%r%r&rs z DistillationGPTModel.core_moduleTrwcsd|_t|S)NT)rhrtrain)rrwr#r%r&rs zDistillationGPTModel.traincs0|dkr|js||dSd|_t||S)NtrainingF)rhrr __setattr__)rnamevaluer#r%r&rs  z DistillationGPTModel.__setattr__)NNNN)T)rWrXrYrZrr[rrr Modulerrzr rpropertyrrrrboolrrr\r%r%r#r&r]fs<-" r])%typingrrrrrrH megatron.corermegatron.core.transformerrr r nemo.collectionsr 2nemo.collections.nlp.modules.common.megatron.utilsr nemo.lightning.megatron_parallelr nemo.utilsrnemo.utils.model_utilsrutilsrrr1nemo.collections.common.tokenizers.tokenizer_specrnemo.lightning.pytorch.optimrmodelopt.torch.distilldistillr}rGPTModelr]r%r%r%r&s"         A