o ei=@s^dZddlZddlmZddlmmZddlmZGdddej Z Gdddej Z dS)z_ Gumbel Softmax implementation with multiple groups possible. Authors * Rudolf A. Braun 2022 N) vector_normcs0eZdZdZfddZddZddZZS)GumbelVectorQuantizeraVector quantization using gumbel softmax. Copied from fairseq implementation. Arguments --------- input_dim: int Input dimension (channels). num_vars: int Number of quantized vectors per group. temp_tuple: float Temperature for training. this should be a tuple of 3 elements: (start, stop, decay factor). groups: int Number of groups for vector quantization. vq_dim: int Dimensionality of the resulting quantized vector. Example ------- >>> quantiser = GumbelVectorQuantizer(128, 100, (2.0, 0.25, 0.999995,), 2, 50 ) >>> inputs = torch.rand(10, 12, 128) >>> output = quantiser(inputs) >>> output["x"].shape torch.Size([10, 12, 50]) cst||_||_||_||_||dks"Jd|d|d||}tt d||||_ tj |j t |j|||_tj j|jjdddtj |jjt|dksaJ||\|_|_|_|j|_tjttt|j|jdd |_dS) Nrzdim z must be divisible by groups z for concatenation)meanstdF) requires_grad)super__init__groups input_dimnum_varsvq_dimnn Parametertorch FloatTensorvarsinituniform_Linear weight_projnormal_weightzeros_biaslenmax_tempmin_temp temp_decay curr_templogtensorfloatmax_ent)selfr r temp_tupler rvar_dim __class__Y/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/nnet/quantisers.pyr &s.  zGumbelVectorQuantizer.__init__cCst|j|j||j|_dS)z-Update the temperature given the current stepN)maxrrrr )r%stepsr*r*r+ update_tempFs z!GumbelVectorQuantizer.update_tempc Cs|j|j|jd}|j\}}}|d|}||}||||jd}|d\}}|j|j d|ddd|||jd}t j | dd} t t j| t | ddd |d<t j||||jd ddj dd} t t j| t | ddd |d <|j|d <|jrtj| |jd d |}n|}|||d}|j} |d| }||||j|jd}|d }|||d}||d<|S)6Forward the latent vector to obtain a quantised output)r temprg?rdimgHz>code_perplexity prob_perplexr0T)tauhardx)r r r shapereshaperviewr, new_zerosscatter_rrr#expsumr!softmaxtrainingFgumbel_softmaxtype_asr unsqueeze) r%r9resultbsztszfsz_khard_x hard_probs avg_probsrr*r*r+forwardLsX        zGumbelVectorQuantizer.forward)__name__ __module__ __qualname____doc__r r.rP __classcell__r*r*r(r+rs   rcs(eZdZdZfddZddZZS)RandomProjectionQuantizeraVector quantization using a projection and a randomly initialised codebook this is useful for models like BEST-RQ for instance. The output is the indices of the closest code in the codebook for each time step of the input. ref: https://arxiv.org/pdf/2202.01855 Arguments --------- input_dim: int Input dimension (channels). cb_dim: int Size of each code in the codebook. cb_vocab: int Number of codes in the codebook Example ------- >>> quantiser = RandomProjectionQuantizer(16, 16, 32) >>> inputs = torch.rand(10, 12, 16) >>> output = quantiser(inputs) >>> output.shape torch.Size([10, 12]) c s\t||_||_||_t||f}|dtj ||dt t ||dS)NPCB)r r r cb_dimcb_vocabremptyregister_bufferrrxavier_uniform_rC normalizerandn)r%r rYrZP_initr(r*r+r s z"RandomProjectionQuantizer.__init__cCs:tj||jdd}t|jd|dddjddS)r/r2rr1)rCr^rWrrXrFargmin)r%r9r*r*r+rPs z!RandomProjectionQuantizer.forward)rQrRrSrTr rPrUr*r*r(r+rVs rV) rTrtorch.nnrtorch.nn.functional functionalrC torch.linalgrModulerrVr*r*r*r+s  q