o }oiA@sddlmZddlmZddlmZmZmZmZddl Z ddl m m Z ddl m Z ddlmZmZmZddlmZddlmZmZmZdd lmZdd lmZeredd lmZdd lmZdd l m!Z!eGdddeZ"eGddde"Z#eGddde"Z$GdddeZ%e&e%dGdddej'de%fZ(e)e%dGdddej'e%dfZ*ej+ddddej,fd d!Z-ej+d"d#ddej,fd$d%Z.ej+ddddej,fd&d'Z/ej+d#d"ddej,fd(d)Z0gd*Z1dS)+) dataclass)Path) TYPE_CHECKING AnnotatedCallableOptionalN)nn) GPTConfigGPTModeltorch_dtype_from_mcore_config)Config)OptimizerModuleioteardown) TransformFns) dtype_from_hfAutoModelForCausalLM AutoTokenizer) TokenizerSpecc@seZdZUdZdZeed<dZeed<dZeed<dZ eed <d Z eed <d Z e ed <dZ e ed<dZe ed<dZeed<dZeed<dZeed<dZe ed<dZeed<ejZeed<dZeed<dZeed<dZeed<dZeed <d!S)" ChatGLMConfigzP Configuration class for the ChatGLM Config, inheriting from GPTConfig.  num_layersi hidden_sizei5ffn_hidden_size num_attention_headsnum_query_groupsg{Gz?init_method_stdghidden_dropoutattention_dropoutRMSNorm normalizationFadd_bias_linearT add_qkv_biasg?rotary_percentrotary_interleavedactivation_funcgated_linear_unitropeposition_embedding_type#share_embeddings_and_output_weightsimake_vocab_size_divisible_byN)__name__ __module__ __qualname____doc__rint__annotations__rrrrr floatr!r"r$strr%boolr&r'r(Fsilur)rr*r,r-r.r:r:Z/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/gpt/model/chatglm.pyr$s(                 rc@eZdZUdZdZeed<dS)ChatGLM2Config6Bz] Configuration class for the ChatGLM2Config6B Config, inheriting from ChatGLMConfig. i seq_lengthNr/r0r1r2r>r3r4r:r:r:r;r=> r=c@r<)ChatGLM3Config6Bz] Configuration class for the ChatGLM3Config6B Config, inheriting from ChatGLMConfig. i r>Nr?r:r:r:r;rAGr@rAc sfeZdZdZ    d deeeeefdeededdee e j ge j fffdd Z Z S) ChatGLMModelz ChatGLM model implementation based on the GPT model architecture. This class provides a high-level interface for ChatGLM models, implementing the specific architecture and settings needed for ChatGLM models. Nconfigoptim tokenizerrmodel_transformcstj|pt|||ddS)N)rDrErF)super__init__r)selfrCrDrErF __class__r:r;rHXszChatGLMModel.__init__)NNNN)r/r0r1r2rrrr r rrModulerH __classcell__r:r:rJr;rBPs rBhfc@sXeZdZdZdefddZdedefddZdd Ze dd d Z e de fd dZ dS)HFChatGLMImporterz Importer for converting Hugging Face ChatGLM models to NeMo format. This class handles the conversion of Hugging Face's ChatGLMForCausalLM models to NeMo's ChatGLM format, including weight mapping and configuration translation. returncCst|j|jdS)z Initialize a NeMo ChatGLMModel instance. Returns: ChatGLMModel: Initialized NeMo Llama model with the appropriate configuration and tokenizer. )rE)rBrCrErIr:r:r;initkszHFChatGLMImporter.init output_pathcCsjddlm}|jt|ddd}|}||}||||||td|t ||~~|S)z Apply the conversion from HF to NeMo format. Args: output_path: Path where the converted model will be saved Returns: Path: Path to the saved NeMo model rrTautotrust_remote_code torch_dtypez0Converted ChatGLM model to Nemo, model saved to ) transformersrfrom_pretrainedr6rR nemo_setup convert_state nemo_saveprintr)rIrSrsourcetargettrainerr:r:r;applyus     zHFChatGLMImporter.applyc Cs,ddddddddd }tj|||ttgd S) aS Convert state dict from HF format to NeMo format. Maps the weights from the HF model to the NeMo model according to the appropriate mapping scheme. Args: source: Source HF model target: Target NeMo model Returns: The result of applying the transforms embedding.word_embeddings.weight2decoder.layers.*.self_attention.linear_proj.weight&decoder.layers.*.mlp.linear_fc1.weight&decoder.layers.*.mlp.linear_fc2.weightrr.fp16bf16 params_dtype)rXr{rYr6rrrrrr>multi_query_group_numpadded_vocab_sizertorchfloat16bfloat16)rI HFAutoConfigr^outputr:r:r;rCs    zHFChatGLMImporter.configN)rPr) r/r0r1r2rBrRrrar[propertyrErrCr:r:r:r;rObs   rOrc@sJeZdZdZejdfdddZddedefdd Zd d Z e d d Z dS)HFChatGLMExporterz Exporter for converting NeMo ChatGLMModel to Hugging Face format. This class handles the conversion of NeMo's ChatGLMModel to Hugging Face's ChatGLMForCausalLM format, including weight mapping and configuration translation. NrPrcCsddlm}m}ddlm}|durd}| |j|dd}|j|d|d}t|d|WdS1s:wYdS) Nr)r{r)no_init_weightszTHUDM/chatglm3-6bTrxrUr) rXr{rtransformers.modeling_utilsrrY from_configtyperegister_for_auto_class)rIdtype model_namer{rrrChf_modelr:r:r;rRs $zHFChatGLMExporter.initrScCsT|t|\}}|jt|j|d}|||}|}|||j||S)N)r) nemo_loadr6rRr rCr[cpusave_pretrainedrE)rIrStarget_model_namer^_r_r:r:r;ras   zHFChatGLMExporter.applycCsLddddddd}tttjdd tjd tjd d tjd g}tj||||d S)ac Convert state dict from NeMo format to HF format. Maps the weights from the NeMo model to the HF model according to the appropriate mapping scheme. Args: source: Source NeMo model target: Target HF model Returns: The target model with weights transferred from source rkrlrmrnrorp)rcrdrerfrgrhrbrj) source_key target_keyfnrirqrr)_export_qkv_weight_export_qkv_biasrstate_transformr prune_paddingru)rIr^r_rsrtr:r:r;r[s4 zHFChatGLMExporter.convert_statecCstt|jjjS)z Get the tokenizer from the NeMo model. Returns: TokenizerSpec: Tokenizer from the NeMo model )r load_contextr6modelrErQr:r:r;rE)szHFChatGLMExporter.tokenizer)rPr)N) r/r0r1r2rrrRrrar[rrEr:r:r:r;rs -rzBtransformer.encoder.layers.*.self_attention.query_key_value.weightz1decoder.layers.*.self_attention.linear_qkv.weight)rrctxc CsV|jj}|j}|j}||}|j}|j}|}|||df} |||df} |j||||||gdd\} } } | j| } | j| } | j| } t d||df |}t |D]B}t || |||d|ddddff}t || ||dddddff}t || ||dddddff}qY|||d||g}|S)Nrdimr)r_rCrrr kv_channelssizesplitviewremptytype_asrangecatreshape)rhf_qkv_weightsmegatron_confighead_numrheads_per_groupr head_sizeold_tensor_shapenew_q_tensor_shapenew_kv_tensor_shapeqkv qkv_weightsir:r:r;rv4s,    0(*rvz@transformer.encoder.layers.*.self_attention.query_key_value.biasz/decoder.layers.*.self_attention.linear_qkv.biascCs"|jj}|j}|j}||}|j}||f}||f}|j||||||gdd\} } } | j|} | j|} | j|} td|f |} t |D]9} t | | | || d|ddff} t | | | | dddff} t | | | | dddff} qI| ||d|g} | S)Nrrrr) r_rCrrrrrrrrrrr)r hf_qkv_biasrrrrrrrrrrqkv_biasrr:r:r;rwUs.    *"$rwcs|jj}|j}|j}|||j}|j}|d|}||||g}tfddt |D}t |d} t d|d} ||d|} || d|} || d|} tj| | | fddS)Nrc,g|]}td|d|qSrrarange.0rrr:r; z&_export_qkv_weight..rrr) r^rCrrrrrrrrr)rrrrrrr qkv_total_dimq_slicek_slicev_sliceq_weightk_weightv_weightr:rr;rvs&  rc s|jj}|j}|j}|||j}|d|}|||g}tfddt|D}t |d}t d|d} ||d} ||d} || d} t| | | fS)Nrcrrrrrr:r;rrz$_export_qkv_bias..rr) r^rCrrrrrrrr) rrrrrrrrrrq_biask_biasv_biasr:rr;rs0  r)rr=rArB)2 dataclassesrpathlibrtypingrrrrrtorch.nn.functionalr functionalr8#nemo.collections.llm.gpt.model.baser r r nemo.collections.llm.utilsr nemo.lightningr rrnemo.lightning.io.staternemo.lightning.pytorch.utilsrrXrryr1nemo.collections.common.tokenizers.tokenizer_specrrr=rArBmodel_importerModelConnectorrOmodel_exporterrr TransformCTXrvrwrr__all__r:r:r:r;s^           q_