o }oi8@sddlmZddlmZddlmZmZmZmZddl Z ddl m m Z ddl m Z ddlmZmZmZddlmZddlmZmZmZdd lmZdd lmZeredd lmZdd lmZdd l m!Z!eGdddeZ"eGddde"Z#GdddeZ$e%e$dGdddej&de$fZ'e(e$dGdddej&e$dfZ)ej*ddddej+fddZ,ej*ddddej+fd d!Z-gd"Z.dS)#) dataclass)Path) TYPE_CHECKING AnnotatedCallableOptionalN)nn) GPTConfigGPTModeltorch_dtype_from_mcore_config)Config)OptimizerModuleioteardown) TransformFns) dtype_from_hfAutoModelForCausalLM AutoTokenizer) TokenizerSpecc@seZdZUdZdZeed<ejZ e ed<dZ e ed<dZ e ed<d Zeed <d Zeed <d Zeed<dZeed<dZeed<dZe ed<dS)Baichuan2ConfigzR Configuration class for the Baichuan2 Config, inheriting from GPTConfig. RMSNorm normalizationactivation_funcTgated_linear_unitFadd_bias_linear seq_lengthg{Gz?init_method_stdgư>layernorm_epsilonghidden_dropoutattention_dropout#share_embeddings_and_output_weightsN)__name__ __module__ __qualname____doc__rstr__annotations__Fsilurrrboolrrintrr floatr!r"r#r/r/[/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/llm/gpt/model/baichuan.pyr$s         rc@sZeZdZUdZdZeed<dZeed<dZeed<dZ eed<dZ eed <d Z e ed <d S) Baichuan2Config7Bz[ Configuration class for the Baichuan2 7B Config, inheriting from Baichuan2Config. num_layersr hidden_sizenum_attention_headsnum_query_groupsi+ffn_hidden_sizeropeposition_embedding_typeN) r$r%r&r'r3r-r)r4r5r6r7r9r(r/r/r/r0r16s      r1c sfeZdZdZ    d deeeeefdeededdee e j ge j fffdd Z Z S) Baichuan2Modelz Baichuan2 model implementation based on the GPT model architecture. This class provides a high-level interface for Baichuan2 models, implementing the specific architecture and settings needed for Baichuan2 models. Nconfigoptim tokenizerrmodel_transformcstj|pt|||ddS)N)r<r=r>)super__init__r)selfr;r<r=r> __class__r/r0r@Ls zBaichuan2Model.__init__)NNNN)r$r%r&r'rrrr r rrModuler@ __classcell__r/r/rBr0r:Ds r:hfc@sXeZdZdZdefddZdedefddZdd Ze dd d Z e de fd dZ dS)HFBaichuan2Importerz Importer for converting Hugging Face Baichuan2 models to NeMo format. This class handles the conversion of Hugging Face's BaichuanForCausalLM models to NeMo's Baichuan2 format, including weight mapping and configuration translation. returncCst|j|jdS)z Initialize a NeMo Baichuan2Model instance. Returns: Baichuan2Model: Initialized NeMo Llama model with the appropriate configuration and tokenizer. )r=)r:r;r=rAr/r/r0initaszHFBaichuan2Importer.init output_pathcCsjddlm}|jt|ddd}|}||}||||||td|t ||~~|S)z Apply the conversion from HF to NeMo format. Args: output_path: Path where the converted model will be saved Returns: Path: Path to the saved NeMo model rrTautotrust_remote_code torch_dtypez1Converted Baichuan model to Nemo, model saved to ) transformersrfrom_pretrainedr(rJ nemo_setup convert_state nemo_saveprintr)rArKrsourcetargettrainerr/r/r0applyks     zHFBaichuan2Importer.applycC<dddddddd}ttjd d tjd g}tj||||d S) aS Convert state dict from HF format to NeMo format. Maps the weights from the HF model to the NeMo model according to the appropriate mapping scheme. Args: source: Source HF model target: Target NeMo model Returns: The result of applying the transforms embedding.word_embeddings.weight2decoder.layers.*.self_attention.linear_proj.weight&decoder.layers.*.mlp.linear_fc2.weight.make_vocab_size_divisible_byFr2r8alibi) r3r4r7r5rr rrr#r9fp16bf16 params_dtype)rPr{rQr(rnum_hidden_layersr4intermediate_sizer5initializer_range rms_norm_epsr~rtorchfloat16bfloat16)rA HFAutoConfigrVroutputr/r/r0r;s&   zHFBaichuan2Importer.configN)rHr) r$r%r&r'r:rJrrYrSpropertyr=rr;r/r/r/r0rGXs "  rGrc@sJeZdZdZejdfdddZddedefdd Zd d Z e d d Z dS)HFBaichuan2Exporterz Exporter for converting NeMo Baichuan2Model to Hugging Face format. This class handles the conversion of NeMo's Baichuan2Model to Hugging Face's BaichuanForCausalLM format, including weight mapping and configuration translation. NrHrcCsddlm}m}ddlm}|durd}| |j|dd}|j|d|d}t|d |WdS1s:wYdS) z Initialize a HF BaichuanForCausalLM instance. Args: dtype: Data type for model parameters Returns: AutoModelForCausalLM: Initialized HF Baichuan model r)r{r)no_init_weightsNzbaichuan-inc/Baichuan2-7B-BaseTrxrMr) rPr{rtransformers.modeling_utilsrrQ from_configtyperegister_for_auto_class)rAdtype model_namer{rrr;hf_modelr/r/r0rJs $zHFBaichuan2Exporter.initrKcCsT|t|\}}|jt|j|d}|||}|}|||j||S)N)r) nemo_loadr(rJr r;rScpusave_pretrainedr=)rArKtarget_model_namerV_rWr/r/r0rYs   zHFBaichuan2Exporter.applycCrZ) ac Convert state dict from NeMo format to HF format. Maps the weights from the NeMo model to the HF model according to the appropriate mapping scheme. Args: source: Source NeMo model target: Target HF model Returns: The target model with weights transferred from source rbrcrdrerfrgrh)r[r\r]r^r_r`rarjrirkro) _export_qkvrrsr split_fc1rurvr/r/r0rS rwz!HFBaichuan2Exporter.convert_statecCstt|jjjS)z Get the tokenizer from the NeMo model. Returns: TokenizerSpec: Tokenizer from the NeMo model )r load_contextr(modelr=rIr/r/r0r=-szHFBaichuan2Exporter.tokenizer)rHr)N) r$r%r&r'rrrJrrYrSrr=r/r/r/r0rs "rz&model.layers.*.self_attn.W_pack.weightz1decoder.layers.*.self_attention.linear_qkv.weight)rlrmctxc Csn|jj}|j}|j}||}|j}|j}|dd|f}|d}||f|dd} ||f|dd} |dj | } |dj | } |dj | } t d|f|dd |}t |D]B}t || |||d|ddddff}t || ||dddddff}t || ||dddddff}qe|||d||g}|S)Nrr})rWr;r5r6r4 kv_channels unflattensizesqueezeviewremptytype_asrangecatreshape)r qkv_weightsmegatron_confighead_numr6heads_per_groupr4 head_sizeold_tensor_shapenew_q_tensor_shapenew_kv_tensor_shapeqkvir/r/r0rr8s(   0(*rrc s|jj}|j}|j}|||j}|j}|d|}||||g}tfddt |D}t |d} t d|d} t||d||| d||| d|gS)Nr}cs,g|]}td|d|qS)r})rarange).0rrr/r0 gsz_export_qkv..r) rVr;r5r6r4rrrrrr) rrrrr6r4r qkv_total_dimq_slicek_slicev_slicer/rr0rVs*  r)rr1r:)/ dataclassesrpathlibrtypingrrrrrtorch.nn.functionalr functionalr*#nemo.collections.llm.gpt.model.baser r r nemo.collections.llm.utilsr nemo.lightningr rrnemo.lightning.io.staternemo.lightning.pytorch.utilsrrPrryr1nemo.collections.common.tokenizers.tokenizer_specrrr1r:model_importerModelConnectorrGmodel_exporterrrs TransformCTXrrr__all__r/r/r/r0sH           ]