o TÃiã@s€ddlmZmZmZmZddlZddlmZddlm Z ddl m Z ddl m Z dd lmZGd d „d e ƒZGd d „d e ƒZdS)é)ÚAnyÚDictÚOptionalÚTypeN)ÚDeepSpeedConfigModelé)ÚRaggedBatchWrapperé)Ú DSModuleBase)ÚDSModuleRegistryBase©ÚDSUnembedConfigc s†eZdZdZedeefdd„ƒZdede e e fddf‡fdd „ Z  dd e jd e jd ed ee jdee jde jf dd„Z‡ZS)Ú DSUnembedBasez¿ Base mixin for unmebedding modules. The interface represented by this module is: if config.do_normalization hidden = layer_norm(hidden) logits = hidden @ projection ÚreturncCótS©Nr ©rrúj/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/inference/v2/modules/interfaces/unembed_base.pyÚ config_classózDSUnembedBase.config_classÚconfigÚimplementation_configNcstƒ ||¡dSr)ÚsuperÚ__init__)Úselfrr©Ú __class__rrrszDSUnembedBase.__init__Ú hidden_statesÚvocab_embeddingÚragged_metadataÚgammaÚbetacCstƒ‚)a’ Forward interface. Gamma and beta are optional parameters passed depending on `self.config.do_normalization`. Args: hidden_states (torch.Tensor): Hidden states of shape [tokens, model_dim] vocab_embedding (torch.Tensor): Embedding matrix of shape [vocab_size, model_dim] ragged_metadata (RaggedBatchWrapper): Metadata for the ragged batch. gamma (Optional[torch.Tensor]): Gamma parameter for layer norm. beta (Optional[torch.Tensor]): Beta parameter for layer norm. Returns: torch.Tensor: Unembedded hidden states of shape [n_seqs, model_dim] )ÚNotImplementedError)rrrrr r!rrrÚforward!szDSUnembedBase.forward)NN)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú staticmethodrrrr rÚstrrrÚtorchÚTensorrrr#Ú __classcell__rrrrrs("ûÿþýüûûrc@s0eZdZUiZeed<edeefdd„ƒZ dS)ÚDSUnembedRegistryÚregistryrcCrr)rrrrrÚassociated_class;rz"DSUnembedRegistry.associated_classN) r$r%r&r.rÚ__annotations__r(rr r/rrrrr-8s r-)Útypingrrrrr*Údeepspeed.runtime.config_utilsrÚraggedrÚ ds_moduler Úmodule_registryr Úconfigsr rr-rrrrÚs     '