o TÃi: ã@s:ddlTddlmZ Gdd„deƒZGdd„deƒZdS)é)Ú*)ÚLayerContainerc @s`eZdZUdZeed<eed<eed<eed<eed<e ed<e ed<d d d d d ddddœZ dS)ÚQwenTransformerContainerz= Transformer layer container for the Qwen model. Úqkv_wÚqkv_bÚ attn_out_wÚmlp_1_wÚmlp_2_wÚattn_norm_gammaÚmlp_norm_gammaz qkv_w.paramsz qkv_b.paramszattn_out_w.paramszmlp_1_w.up_paramszmlp_1_w.gate_paramszmlp_2_w.paramszattn_norm_gamma.paramszmlp_norm_gamma.params)zattn.c_attn.weightzattn.c_attn.biaszattn.c_proj.weightz mlp.w1.weightz mlp.w2.weightzmlp.c_proj.weightz ln_1.weightz ln_2.weightN) Ú__name__Ú __module__Ú __qualname__Ú__doc__ÚFusedQKVParameterÚ__annotations__ÚAttentionOutputParameterÚGatedMLPParameterÚ MLP2ParameterÚ NormParameterÚ PARAM_MAPPING©rrúo/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/inference/v2/model_implementations/qwen/container.pyr)s$  ørc@s6eZdZUdZeed<eed<eed<ddddœZd S) ÚQwenNonTransformerContainerzA Non-Transformer layer container for the Qwen model. Úword_embÚ word_unembedÚ final_normzword_emb.paramszfinal_norm.paramszword_unembed.params)ztransformer.wte.weightztransformer.ln_f.weightzlm_head.weightN) r r rrÚEmbeddingParameterrÚUnembedParameterrrrrrrrAs  ýrN)Úcommon_parametersÚlayer_container_baserrrrrrrÚs