o }oi@sVddlZddlZddlZddlmZmZddlmZmZm Z dgZ GdddeZ dS)N) NeuralModule typecheck) LengthsType NeuralTypeSpectrogramTypeSSLPretrainWithMaskedPatchcsVeZdZdZeddZeddZ  dded effd d Z e d d Z Z S)ra Zeroes out fixed size time patches of the spectrogram. All samples in batch are guaranteed to have the same amount of masked time steps. Note that this may be problematic when we do pretraining on a unbalanced dataset. For example, say a batch contains two spectrograms of length 87 and 276. With mask_fraction=0.7 and patch_size=10, we'll obrain mask_patches=7. Each of the two data will then have 7 patches of 10-frame mask. Args: patch_size (int): up to how many time steps does one patch consist of. Defaults to 10. mask_fraction (float): how much fraction in each sample to be masked (number of patches is rounded up). Range from 0.0 to 1.0. Defaults to 0.7. cCstdtttdtdS)z)Returns definitions of module input typesBCDTr ) input_speclength)rrtuplerselfrg/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/audio/modules/ssl_pretrain_masking.py input_types+s z&SSLPretrainWithMaskedPatch.input_typescCsdtdtiS)z*Returns definitions of module output typesaugmented_specr)rrrrrr output_types3sz'SSLPretrainWithMaskedPatch.output_types ffffff? patch_size mask_fractioncsNt|dkrtd||dks|dkrtd|||_||_dS)Nrz,patch_size must be positive, got patch_size=g?zAmask_fraction must be in the range [0.0, 1.0], got mask_fraction=)super__init__ ValueErrorrr)rrr __class__rrr8s  z#SSLPretrainWithMaskedPatch.__init__c Cs|}|jr[t|D]M\}}t||j}t||j}||jt||jdk}||j|kr7||j}t||} | D]} d||dddd| |j| d|jf<q?q |S|j|j} t j | d|j d} | | |jk} t | d} || }|S)a- Apply Patched masking on the input_spec. During the training stage, the mask is generated randomly, with approximately `self.mask_fraction` of the time frames being masked out. In the validation stage, the masking pattern is fixed to ensure consistent evaluation of checkpoints and to prevent overfitting. Note that the same masking pattern is applied to all data, regardless of their lengths. On average, approximately `self.mask_fraction` of the time frames will be masked out. rrN)devicez T -> 1 1 1 T)training enumeratetolistrangerintrrandomsampletorcharangesizer#einops rearrangefloat) rr rridxcur_lenpatches len_fraction mask_patchesmasked_patchesmp chunk_lengthmaskrrrforwardGs&  . z"SSLPretrainWithMaskedPatch.forward)rr) __name__ __module__ __qualname____doc__propertyrrr(r0rrr: __classcell__rrrrrs  ) r)r.r+nemo.core.classesrrnemo.core.neural_typesrrr__all__rrrrrs