o ̳i+@sddlZddlmZmZmZddlZddddddd d d d d dd Zddddd dd dd d d ddd ZdedeeefdefddZ deeej fdeeej ffddZ deeej fdeeej ffddZ    d9deeej fde de de d e deeej ff d!d"Z    d9deeej fde de de d e f d#d$Zd%d&d'd(Zd)d*d+d,d-d.d/d0d1Zgd2Z d:d3eeefd4eefd5d6Z    d9deeej fde de de d ee f d7d8ZdS);N)AnyDictOptionaltok_embeddings.weightz norm.scale output.weightzlayers.{}.attn.k_proj.weightzlayers.{}.attn.q_proj.weightzlayers.{}.attn.v_proj.weightz!layers.{}.attn.output_proj.weightzlayers.{}.sa_norm.scalezlayers.{}.mlp_norm.scalezlayers.{}.mlp.w1.weightzlayers.{}.mlp.w2.weightzlayers.{}.mlp.w3.weight) rz norm.weightrzlayers.{}.attention.wk.weightzlayers.{}.attention.wq.weightzlayers.{}.attention.wv.weightzlayers.{}.attention.wo.weightzlayers.{}.attention_norm.weightzlayers.{}.ffn_norm.weightz layers.{}.feed_forward.w1.weightz layers.{}.feed_forward.w2.weightz layers.{}.feed_forward.w3.weight) zmodel.embed_tokens.weightz'model.layers.{}.self_attn.q_proj.weightz'model.layers.{}.self_attn.k_proj.weightz'model.layers.{}.self_attn.v_proj.weightz'model.layers.{}.self_attn.o_proj.weightz-model.layers.{}.self_attn.rotary_emb.inv_freqz$model.layers.{}.mlp.gate_proj.weightz"model.layers.{}.mlp.up_proj.weightz$model.layers.{}.mlp.down_proj.weightz&model.layers.{}.input_layernorm.weightz/model.layers.{}.post_attention_layernorm.weightzmodel.norm.weightzlm_head.weightkey mapping_dictreturnc Csz/tdd|dDr)tdd|}td|d}||}||}W|S||}W|StyD}z td|d |d}~ww) Ncss|]}|VqdSN)isdigit).0krT/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/models/convert_weights.py 2sz!get_mapped_key...z(\.\d+)z.{}z\d+rz8Error converting the state dict. Found unexpected key: "zG". Please make sure you're loading a checkpoint with the right format. ) anysplitresubsearchgroupformatKeyError Exception)rr abstract_key layer_numnew_keyerrrget_mapped_key/s"   r state_dictcCs4i}|D]\}}|dvrt|t}|||<q|S)a Convert a state dict from Meta's format to torchtune's format. State dicts from multiple checkpoint files should be consolidated into a single state dict before calling this function. Eg of Meta-format state dict can be found in the ``meta-llama/Llama-2-7b`` repo in HF (https://huggingface.co/meta-llama/Llama-2-7b). Args: state_dict (Dict[str, torch.Tensor]): State dict in Meta's format. Returns: Dict[str, torch.Tensor]: State dict in torchtune's format. )z rope.freqs)itemsr _FROM_META)r converted_state_dictrvaluerrrr meta_to_tuneCs r%cCs>i}ddtD}|D] \}}t||}|||<q|S)av Convert a state dict from torchtune's format to Meta's format. This function doesn't handle any sharding or splitting of state dicts. It follows the state_dict IN -> state_dict OUT pattern. Args: state_dict (Dict[str, torch.Tensor]): State dict in torchtune's format. Returns: Dict[str, torch.Tensor]: State dict in Meta's format. cSi|]\}}||qSrrr r vrrr hz tune_to_meta..)r"r!r)r r#inverted_mapping_dictrr$rrrr tune_to_meta[s    r,  num_heads num_kv_headsdimhead_dimc sxi}dur |fdd}|D]$\}}d|vr9t|t} d|vr,|||}n d|vr5|||}||| <q|S)a( Convert a state dict from HF's format to torchtune's format. State dicts from multiple checkpoint files should be consolidated into a single state dict before calling this function. Eg of HF-format state dict can be found in the ``meta-llama/Llama-2-7b-hf`` repo in HF (https://huggingface.co/meta-llama/Llama-2-7b-hf). Args: state_dict (Dict[str, torch.Tensor]): State dict in HF's format. num_heads (int): Number of heads in the model. num_kv_heads (int): Number of heads in the key/value projection layers. dim (int): Dimension of the model. head_dim (int): Dimension of the head. If not provided, it will be calculated as dim // num_heads. Returns: Dict[str, torch.Tensor]: State dict in torchtune's format. Ncs(||dddd|SNview transposereshapetn_headsr1r2rr_permute zhf_to_tune.._permutezrotary_emb.inv_freqq_projk_proj)r!r_FROM_HF) r r/r0r1r2r#r>rr$rrr=r hf_to_tuneqs   rCc si}ddtD}dur|fdd}|D] \}} t||} d|vr1|| |} n d|vr:|| |} | || <q|S)an Convert a state dict from torchtune's format to HF's format. This function doesn't handle any sharding or splitting of state dicts. It follows the state_dict IN -> state_dict OUT pattern. Args: state_dict (Dict[str, torch.Tensor]): State dict in torchtune's format. num_heads (int): Number of heads in the model. num_kv_heads (int): Number of heads in the key/value projection layers. dim (int): Dimension of the model. head_dim (int): Dimension of model attention heads. Default None. Returns: Dict[str, torch.Tensor]: State dict in HF's format. cSr&rrr'rrrr)r*ztune_to_hf..Ncs(||dddd|Sr3r6r:r=rrr>r?ztune_to_hf.._permuter@rA)rBr!r) r r/r0r1r2r#r+r>rr$rrr=r tune_to_hfs    rDlora_Alora_Blora_magnitude_vector)lora_alora_b magnituder@rAv_projo_proj gate_proj down_projup_projlm_head)r@rArK output_projw1w2w3output)target_modulesr lora_alphaadapter_configbase_model_name_or_pathcsztfddtDstdtddD] }|tvr(td|qtttjdd<|r;|d<S)Ncsg|]}|vqSr)keys)r xrYrr sz/tune_to_peft_adapter_config..zPEFT adapter config requires z, found rVzUnknown target module rZ)all_PEFT_CONFIG_EXPECTED_KEYS ValueErrorr[_TO_PEFT_TARGET_MODULESlistmapget)rYrZr rr]rtune_to_peft_adapter_configs rfcsi}i}tD]D\}}tD];\} } | durq|dkr0| dd|} | dd|} n| dd|d} | dd|d} || | iqqdurU||fdd} |D]*\}}t||}d|vrvd|vrv| ||}n d|vrd|vr| ||}||d |<q_|S) NrJz.weightrcs2|jd}||dd|dd||S)Nr4r5)shaper7r8r9)r;r<rankr2rr_permute_lora_matrixs  z:tune_to_peft_adapter_weights.._permute_lora_matrixr@rFrAzbase_model.model.) _TO_PEFT_KEYSr!rBreplaceupdater)r r/r0r1r2r# full_mappingpeft_keypeft_valhf_keyhf_val adapter_key adapter_valrkrr$rrrjrtune_to_peft_adapter_weightss0    rv)r-r-r.Nr )rtypingrrrtorchr"rBstrrTensorr%r,intrCrDrlrbr`rfrvrrrrs&&   3  0