o ©Ì³iÖã@s ddlmZddlZddlmZidd“dd“dd “d d “d d “dd“dd“dd“dd“dd“dd“dd“dd“dd“dd “d!d"“Zd!Z # # $  %d1d&eeejfd'e d(e d)e d*e d+e d,eeejffd-d.„Z # # $  %d1d&eeejfd'e d(e d)e d*e d+e f d/d0„Z dS)2é)ÚDictN)Úget_mapped_keyzmodel.embed_tokens.weightztok_embeddings.weightz'model.layers.{}.self_attn.q_proj.weightzlayers.{}.attn.q_proj.weightz%model.layers.{}.self_attn.q_proj.biaszlayers.{}.attn.q_proj.biasz'model.layers.{}.self_attn.k_proj.weightzlayers.{}.attn.k_proj.weightz%model.layers.{}.self_attn.k_proj.biaszlayers.{}.attn.k_proj.biasz'model.layers.{}.self_attn.v_proj.weightzlayers.{}.attn.v_proj.weightz%model.layers.{}.self_attn.v_proj.biaszlayers.{}.attn.v_proj.biasz'model.layers.{}.self_attn.o_proj.weightz!layers.{}.attn.output_proj.weightz-model.layers.{}.self_attn.rotary_emb.inv_freqz$model.layers.{}.mlp.gate_proj.weightzlayers.{}.mlp.w1.weightz"model.layers.{}.mlp.up_proj.weightzlayers.{}.mlp.w3.weightz$model.layers.{}.mlp.down_proj.weightzlayers.{}.mlp.w2.weightz&model.layers.{}.input_layernorm.weightzlayers.{}.sa_norm.scalez/model.layers.{}.post_attention_layernorm.weightzlayers.{}.mlp_norm.scalezmodel.norm.weightz norm.scalezlm_head.weightz output.weighté éFÚ state_dictÚ num_headsÚ num_kv_headsÚdimÚhead_dimÚtie_word_embeddingsÚreturnc CsTi}|dur ||}| ¡D]\}}|rt|vrqd|vrqt|tƒ} ||| <q|S)a³ Convert a state dict from HF's format to TorchTune's format, which contains the weights of a Qwen2 model. State dicts from multiple checkpoint files should be consolidated into a single state dict before calling this function. The logic is identical to :func:`~torchtune.models.convert_weights.hf_to_tune`, but may not load output projection weights. Args: state_dict (Dict[str, torch.Tensor]): State dict in HF's format. num_heads (int): Number of heads in the model. num_kv_heads (int): Number of heads in the key/value projection layers. dim (int): Dimension of the model. head_dim (int): Dimension of the head. If not provided, it will be calculated as dim // num_heads. tie_word_embeddings (bool): Whether the model's input and output word embeddings should be tied. Returns: Dict[str, torch.Tensor]: State dict in torchtune's format. Nzrotary_emb.inv_freq)ÚitemsÚQWEN2_TIED_KEYrÚ_FROM_HF) rrrr r r Úconverted_state_dictÚkeyÚvalueÚnew_key©rú[/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/models/qwen2/_convert_weights.pyÚqwen2_hf_to_tune%sÿ  rc CsNi}dd„t ¡Dƒ}|dur||}| ¡D] \}} t||ƒ} | || <q|S)a Convert a state dict from torchtune's format to HF's format. This function doesn't handle any sharding or splitting of state dicts. It follows the state_dict IN -> state_dict OUT pattern. Args: state_dict (Dict[str, torch.Tensor]): State dict in torchtune's format. num_heads (int): Number of heads in the model. num_kv_heads (int): Number of heads in the key/value projection layers. dim (int): Dimension of the model. head_dim (int): Dimension of the head. If not provided, it will be calculated as dim // num_heads. tie_word_embeddings (bool): Whether the model's input and output word embeddings should be tied. Returns: Dict[str, torch.Tensor]: State dict in HF's format. cSsi|]\}}||“qSrr)Ú.0ÚkÚvrrrÚ lsz$qwen2_tune_to_hf..N)rr r) rrrr r r rÚinverted_mapping_dictrrrrrrÚqwen2_tune_to_hfRs  r)rrrNF) ÚtypingrÚtorchÚ torchtune.models.convert_weightsrrrÚstrÚTensorÚintÚboolrrrrrrÚs–  ÿþýüûúùø ÷ ö õ ô óòñðú ÿþýüûú  ù/ú ÿþýüûú