o 7wiD4@sTdZddlZddlZGdddejjZGdddejjZGdddejjZdS) a\Frequency-Domain Sequential Data Augmentation Classes This module comprises classes tailored for augmenting sequential data in the frequency domain, such as spectrograms and mel spectrograms. Its primary purpose is to enhance the resilience of neural models during the training process. Authors: - Peter Plantinga (2020) - Mirco Ravanelli (2023) Ncs6eZdZdZ      d fdd Zd d ZZS) SpectrogramDropaThis class drops slices of the input spectrogram. Using `SpectrogramDrop` as an augmentation strategy helps a models learn to rely on all parts of the signal, since it can't expect a given part to be present. Reference: https://arxiv.org/abs/1904.08779 Arguments --------- drop_length_low : int The low end of lengths for which to drop the spectrogram, in samples. drop_length_high : int The high end of lengths for which to drop the signal, in samples. drop_count_low : int The low end of number of times that the signal can be dropped. drop_count_high : int The high end of number of times that the signal can be dropped. replace: str - 'zeros': Masked values are replaced with zeros. - 'mean': Masked values are replaced with the mean value of the spectrogram. - 'rand': Masked values are replaced with random numbers ranging between the maximum and minimum values of the spectrogram. - 'cutcat': Masked values are replaced with chunks from other signals in the batch. - 'swap': Masked values are replaced with other chunks from the same sentence. - 'random_selection': A random selection among the approaches above. dim : int Corresponding dimension to mask. If dim=1, we apply time masking. If dim=2, we apply frequency masking. Example ------- >>> # time-masking >>> drop = SpectrogramDrop(dim=1) >>> spectrogram = torch.rand(4, 150, 40) >>> print(spectrogram.shape) torch.Size([4, 150, 40]) >>> out = drop(spectrogram) >>> print(out.shape) torch.Size([4, 150, 40]) >>> # frequency-masking >>> drop = SpectrogramDrop(dim=2) >>> spectrogram = torch.rand(4, 150, 40) >>> print(spectrogram.shape) torch.Size([4, 150, 40]) >>> out = drop(spectrogram) >>> print(out.shape) torch.Size([4, 150, 40]) zeroscs~t||_||_||_||_||_||_||krtd||kr'tdgd|_ |j|j vr=tdd |j dS)Nz*Low limit must not be more than high limit)rmeanrandcutcatswaprandom_selectionz(Invalid 'replace' option. Select one of z, ) super__init__drop_length_lowdrop_length_highdrop_count_lowdrop_count_highreplacedim ValueError replace_optsjoin)selfrrrrrr __class__\/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/speechbrain/augment/freq_domain.pyrIs"   zSpectrogramDrop.__init__cCsf|dkr|d|jd|jd}|j\}}}|jdkr!|}n|}tj|j|jdd|jd}|dkr7|Stj|j|j ||f|jd d}tjdt d|| ||f|jd  d}tj ||jd ddd} || k| ||k} | j dd } |jdkr| dn| d} |jd krt|jd d|_|jd kr|| d}n|jdkr|} || | }nu|jdkr| } |} t|}|| | | }| } d| || |}nH|jdkrtj|ddd}| } d| || |}n+|jdkr-tjd|jdd|jd}tj||dd}| } d| || |}|j|jS)a Apply the DropChunk augmentation to the input spectrogram. This method randomly drops chunks of the input spectrogram to augment the data. Arguments --------- spectrogram : torch.Tensor Input spectrogram of shape `[batch, time, fea]`. Returns ------- torch.Tensor Augmented spectrogram of shape `[batch, time, fea]`. rrrlowhighsizedevicer)r%)rr Nrrr r shiftsdimsr )rviewshapetorchrandintrrr%rr unsqueezemaxarangeanyrrandomchoicer masked_fill_rdetachmin rand_likefloatrollitem)r spectrogram batch_size time_durationfea_sizeDn_masksmask_lenmask_posr0maskrmax_spectrogrammin_spectrogramrand_spectrogramrolled_spectrogramshiftrrrforwardms                 zSpectrogramDrop.forward)rrrrrr__name__ __module__ __qualname____doc__rrI __classcell__rrrrrs9$rcs*eZdZdZd fdd ZddZZS) Warpinga Apply time or frequency warping to a spectrogram. If `dim=1`, time warping is applied; if `dim=2`, frequency warping is applied. This implementation selects a center and a window length to perform warping. It ensures that the temporal dimension remains unchanged by upsampling or downsampling the affected regions accordingly. Reference: https://arxiv.org/abs/1904.08779 Arguments --------- warp_window : int, optional The width of the warping window. Default is 5. warp_mode : str, optional The interpolation mode for time warping. Default is "bicubic." dim : int, optional Dimension along which to apply warping (1 for time, 2 for frequency). Default is 1. Example ------- >>> # Time-warping >>> warp = Warping() >>> spectrogram = torch.rand(4, 150, 40) >>> print(spectrogram.shape) torch.Size([4, 150, 40]) >>> out = warp(spectrogram) >>> print(out.shape) torch.Size([4, 150, 40]) >>> # Frequency-warping >>> warp = Warping(dim=2) >>> spectrogram = torch.rand(4, 150, 40) >>> print(spectrogram.shape) torch.Size([4, 150, 40]) >>> out = warp(spectrogram) >>> print(out.shape) torch.Size([4, 150, 40]) rbicubicrcs t||_||_||_dS)N)r r warp_window warp_moder)rrRrSrrrrrs  zWarping.__init__c CsX|jdkr |dd}|j}|j}|dkr|d}|jd}|||kr,|j|St|||dd}t||||ddd}tjj j |ddddd|f||jdf|j dd}tjj j |dddd|df|||jdf|j dd}||ddddd|f<||dddd|df<|j|}|jdkr|dd}|S) a9 Apply warping to the input spectrogram. Arguments --------- spectrogram : torch.Tensor Input spectrogram with shape `[batch, time, fea]`. Returns ------- torch.Tensor Augmented spectrogram with shape `[batch, time, fea]`. rrrr rNT)mode align_corners) r transposer+rRr.r*r,r-nn functional interpolaterS) rr; original_sizewindow len_originalcwleftrightrrrrIs:            zWarping.forward)rrQrrJrrrrrPs)rPcs*eZdZdZdfdd ZddZZS) RandomShiftaShifts the input tensor by a random amount, allowing for either a time or frequency (or channel) shift depending on the specified axis. It is crucial to calibrate the minimum and maximum shifts according to the requirements of your specific task. We recommend using small shifts to preserve information integrity. Using large shifts may result in the loss of significant data and could potentially lead to misalignments with corresponding labels. Arguments --------- min_shift : int The minimum channel shift. max_shift : int The maximum channel shift. dim: int The dimension to shift. Example ------- >>> # time shift >>> signal = torch.zeros(4, 100, 80) >>> signal[0,50,:] = 1 >>> rand_shift = RandomShift(dim=1, min_shift=-10, max_shift=10) >>> lengths = torch.tensor([0.2, 0.8, 0.9,1.0]) >>> output_signal, lengths = rand_shift(signal,lengths) >>> # frequency shift >>> signal = torch.zeros(4, 100, 80) >>> signal[0,:,40] = 1 >>> rand_shift = RandomShift(dim=2, min_shift=-10, max_shift=10) >>> lengths = torch.tensor([0.2, 0.8, 0.9,1.0]) >>> output_signal, lengths = rand_shift(signal,lengths) rrcs4t||_||_||_|j|jkrtddS)Nzmax_shift must be >= min_shift)r r min_shift max_shiftrr)rrbrcrrrrrjs  zRandomShift.__init__cCshtj|j|jdd|jd}tj|||jd}|jdkr0|||j|j}tj |ddd}||fS)aL Arguments --------- waveforms : tensor Shape should be `[batch, time]` or `[batch, time, channels]`. lengths : tensor Shape should be a single dimension, `[batch]`. Returns ------- Tensor of shape `[batch, time]` or `[batch, time, channels]` rr r!r'r&g?)r6r/) r,r-rbrcr%r9r:rr+clamp)r waveformslengthsN_shiftsrrrrIts zRandomShift.forward)rrrrJrrrrraGs" ra)rNr2r,rWModulerrPrarrrrs Fq