o TÃi" ã@sŒddlmZddlmZmZmZmZddlZddlm Z ddl m Z ddl m Z ddlmZd d lmZGd d „d e ƒZGd d„de ƒZdS)é)Úabstractmethod)ÚAnyÚDictÚOptionalÚTypeN)ÚDeepSpeedConfigModelé)Ú DSModuleBase)ÚDSModuleRegistryBase©Ú DSMoEConfigé)ÚInferenceParametercsîeZdZdZedeefdd„ƒZdede e e fddf‡fdd „ Z e d ejdefd d „ƒZe d ejdefd d„ƒZe d ejdefdd„ƒZ  ddejdejdejdejdeejdeejdejfdd„Zee dejfdd„ƒƒZ‡ZS)Ú DSMoEBasea Base mixing for MoE modules. The interface represented by this module is: expert_assignments = gate(hidden_states) intermediate = ragged_linear(hidden_states, expert_assignments) output = ragged_linear(intermediate, expert_assignments) ÚreturncCótS©Nr ©rrúf/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/inference/v2/modules/interfaces/moe_base.pyÚ config_classózDSMoEBase.config_classÚconfigÚimplementation_configNcstƒ ||¡dSr)ÚsuperÚ__init__)Úselfrr©Ú __class__rrrszDSMoEBase.__init__ÚparamcCódS)z¨ Perform any necessary transformations of the gate parameter. Args: param (torch.Tensor): gate_w (shape: [num_experts, model_dim]) Nr©rrrrrÚtransform_gate_param"szDSMoEBase.transform_gate_paramcCr)zþ Perform any necessary transformations of the parameter. The specific component being transformed should be inferred from the shape of the parameter. Args: param (torch.Tensor): One of either mlp_1_w, mlp_1_b Nrr rrrÚtransform_moe_mlp_1_param,s z#DSMoEBase.transform_moe_mlp_1_paramcCr)a} Perform any necessary transformations of the parameter. The specified component being transformed should be inferred from the shape of the parameter. This interface is separate from transform_moe_1_param because the two components may have identical shapes. Args: param (torch.Tensor): One of either mlp_2_w or mlp_2_b Nrr rrrÚtransform_moe_mlp_2_param7s z#DSMoEBase.transform_moe_mlp_2_paramÚ hidden_statesÚgate_wÚmlp_1_wÚmlp_2_wÚmlp_1_bÚmlp_2_bcCstƒ‚r)ÚNotImplementedError)rr$r%r&r'r(r)rrrÚforwardDszDSMoEBase.forwardcCr)zB Returns the pre-allocated, padded output Tensor. Nr)rrrrÚoutputMszDSMoEBase.output)NN)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú staticmethodrrrr rÚstrrrrÚtorchÚTensorrr!r"r#rr+Úpropertyr,Ú __classcell__rrrrrs>"  úÿþýüûú ú rc@s0eZdZUiZeed<edeefdd„ƒZ dS)Ú DSMoERegistryÚregistryrcCrr)rrrrrÚassociated_classYrzDSMoERegistry.associated_classN) r-r.r/r8rÚ__annotations__r1rr r9rrrrr7Vs r7)ÚabcrÚtypingrrrrr3Údeepspeed.runtime.config_utilsrÚ ds_moduler Úmodule_registryr Úconfigsr Úinference_parameterrrr7rrrrÚs      D