o ̳i @sFddlmZddlZddlmZmZddlmZGdddejZ dS))TupleN)nnTensor) functionalcsheZdZdZdededdffdd Zdedeeeffd d Zde fd d Z d edefddZ Z S)VectorQuantizedEmbeddingsa Vector quantized embedding layer that takes in the output of an encoder and performs a nearest-neighbor lookup in the embedding space. Vector quantization was introduced in Oord et al. 2017 (https://arxiv.org/pdf/1711.00937.pdf) to generate high-fidelity images, videos, and audio data. This module currently does not support pre-training of the embeddings via EMA. Code was adapted from torchmultimodal's `Codebook module `_. Args: num_embeddings (int): Number of vectors in the embedding space. embedding_dim (int): Dimensionality of the embedding vectors. num_embeddings embedding_dimreturnNcs.ttt|||_||_||_dSN) super__init__r Parametertorchempty embeddingrr)selfrr __class__S/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/modules/vq_embeddings.pyr s  z"VectorQuantizedEmbeddings.__init__zc Cs|j\}}}||jkrtd|d|j|d|}tj||jddd}tj|dd}||}||| }||||} |||} | | fS) a Args: z (Tensor): Tensor containing a batch of encoder outputs of shape ``(b, s, d)``, where b is batch size, s is sequence length or time, and d is ``embedding_dim``. Returns: Tuple[Tensor, Tensor]: The quantized input and the embedding vector ids that were used. Raises: ValueError: if input embedding dimension does not match embedding dimension of module z)Expected last dimension of input tensor (z) to be embedding size of g@)p)dim) shaper ValueErrorviewrcdistrargmindecodedetach) rrbszseq_len z_embed_dimz_flat distancestoken_ids_flatquantized_flat quantized token_idsrrrforward)s    z!VectorQuantizedEmbeddings.forwardcCsd|j|jS)Nz#num_embeddings={}, embedding_dim={})formatrr)rrrr extra_reprOsz$VectorQuantizedEmbeddings.extra_reprr+cCst||jSr )Fr)rr+rrrr!Tsz VectorQuantizedEmbeddings.decode) __name__ __module__ __qualname____doc__intr rrr,strr.r! __classcell__rrrrrs &r) typingrrrrtorch.nnrr/Modulerrrrrs