o TÃiÿ ã@s˜ddlmZddlmZmZmZmZddlZddlm Z ddl m Z ddl m Z dd lmZdd lmZdd lmZGd d „d e ƒZGdd„deƒZdS)é)Úabstractmethod)ÚAnyÚDictÚOptionalÚTypeN)ÚDeepSpeedConfigModelé)ÚRaggedBatchWrapperé)Ú DSModuleBase)ÚDSModuleRegistryBase©ÚDSEmbeddingsConfig)ÚInferenceParameterc s¶eZdZdZedeefdd„ƒZdede e e fddf‡fdd „ Z d e jdefd d „Zeede jfd d„ƒƒZ   ddede jdee jdee jdee jdef dd„Z‡ZS)ÚDSEmbeddingBasea  Base mixin for embedding modules. The interface represented by this module is: hidden_out = embedding(input_ids) + position_embedding(position_ids) + token_type_embedding(token_type_ids) with optional normalization. ÚreturncCótS©Nr ©rrúl/home/ubuntu/.local/lib/python3.10/site-packages/deepspeed/inference/v2/modules/interfaces/embedding_base.pyÚ config_classózDSEmbeddingBase.config_classÚconfigÚimplementation_configNcstƒ ||¡dSr)ÚsuperÚ__init__)Úselfrr©Ú __class__rrr!szDSEmbeddingBase.__init__Ú embed_paramcCótƒ‚)a3 Perform any necessary transformations on an embedding parameter. This module assumes that all embedding parameters would require the same set of transformations. Parameters: embed_param (torch.Tensor): Embedding parameter. Shape is of [vocab_size, hidden_size] ©ÚNotImplementedError)rrrrrÚtransform_param$szDSEmbeddingBase.transform_paramcCr )a Pre-allocated output Tensor. This currently needs to be exposed for gather operations on the output. TODO(cmikeh2): This is not ideal. We need a better abstraction for this, such as giving access to the inference comm object to the DSModule. r!)rrrrÚoutput.s zDSEmbeddingBase.outputÚ ragged_batchÚword_embeddingsÚposition_embeddingsÚtoken_type_idsÚtoken_type_embeddingscCr )aV Parameters: ragged_batch (torch.Tensor): Ragged batch of token ids + associated metadata. word_embeddings (torch.Tensor): Word embeddings. position_embeddings (torch.Tensor): Position embeddings. If passed, IDs will be inferred from the ragged batch itself. token_type_ids (torch.Tensor): Token type ids. token_type_embeddings (torch.Tensor): Token type embeddings. Returns: torch.Tensor: Hidden states. This should be the sum of the relevant encodings for the model. r!)rr%r&r'r(r)rrrÚforward:szDSEmbeddingBase.forward)NNN)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú staticmethodrrrrrÚstrrrÚtorchÚTensorrr#Úpropertyrr$r rr*Ú __classcell__rrrrrs2 "  ûÿþýüûûrc@s0eZdZUiZeed<edeefdd„ƒZ dS)ÚDSEmbeddingRegistryÚregistryrcCrr)rrrrrÚassociated_classSrz$DSEmbeddingRegistry.associated_classN) r+r,r-r6rÚ__annotations__r/rr r7rrrrr5Ps r5)ÚabcrÚtypingrrrrr1Údeepspeed.runtime.config_utilsrÚraggedr Ú ds_moduler Úmodule_registryr ÚconfigsrÚinference_parameterrrr5rrrrÚs       =