o 7wiP@s^dZddlZddlZddlmmZddlmZddl m Z e e Z Gdddejj ZdS)zWClasses for implementing data augmentation pipelines. Authors * Mirco Ravanelli 2022 N)lengths_arg_exists) get_loggercspeZdZdZddddddddddddedffdd Zd d Zd d Zd dZddZ ddZ ddZ Z S) AugmenterasApplies pipelines of data augmentation. Arguments --------- parallel_augment: bool If False, the augmentations are applied sequentially with the order specified in the pipeline argument. When True, all the N augmentations are concatenated in the output on the batch axis. parallel_augment_fixed_bs: bool If False, each augmenter (performed in parallel) generates a number of augmented examples equal to the batch size. Thus, overall, with this option N*batch size artificial data are generated, where N is the number of augmenters. When True, the number of total augmented examples is kept fixed at the batch size, thus, for each augmenter, fixed at batch size // N examples. This option is useful to keep controlled the number of synthetic examples with respect to the original data distribution, as it keep always 50% of original data, and 50% of augmented data. concat_original: bool if True, the original input is concatenated with the augmented outputs (on the batch axis). min_augmentations: int The number of augmentations applied to the input signal is randomly sampled between min_augmentations and max_augmentations. For instance, if the augmentation dict contains N=6 augmentations and we set select min_augmentations=1 and max_augmentations=4 we apply up to M=4 augmentations. The selected augmentations are applied in the order specified in the augmentations dict. If shuffle_augmentations = True, a random set of M augmentations is selected. max_augmentations: int Maximum number of augmentations to apply. See min_augmentations for more details. shuffle_augmentations: bool If True, it shuffles the entries of the augmentations dictionary. The effect is to randomply select the order of the augmentations to apply. repeat_augment: int Applies the augmentation algorithm N times. This can be used to perform more data augmentation. augment_start_index: int The index of the first element in the input batch from which data augmentation should begin. This argument allows you to specify the starting point for applying data augmentation. augment_end_index: int The index of the last element in the input batch at which data augmentation should stop. You can use this argument to define the endpoint for applying data augmentation within the batch. concat_start_index: int If `concat_original` is set to True, you can specify a subpart of the original batch to concatenate in the output. Use this argument to select the index of the first element from the original input batch to start copying from. concat_end_index: int If `concat_original` is set to True, you can specify a subpart of the original batch to concatenate in the output. Use this argument to select the index of the last element from the original input batch to end the copying process. augment_prob: float The probability (0.0 to 1.0) of applying data augmentation. When set to 0.0, the original signal is returned without any augmentation. When set to 1.0, augmentation is always applied. Values in between determine the likelihood of augmentation. augmentations: list List of augmentater objects to combine to perform data augmentation. enable_augmentations: list A list of booleans used to selectively enable or disable specific augmentation techniques within the 'augmentations' list. Each boolean corresponds to an augmentation object in the 'augmentations' list and should be of the same length and order. This feature is useful for performing ablations on augmentation techniques to tailor them for a specific task. Example ------- >>> from speechbrain.augment.time_domain import DropFreq, DropChunk >>> freq_dropper = DropFreq() >>> chunk_dropper = DropChunk(drop_start=100, drop_end=16000) >>> augment = Augmenter(parallel_augment=False, concat_original=False, augmentations=[freq_dropper, chunk_dropper]) >>> signal = torch.rand([4, 16000]) >>> output_signal, lengths = augment(signal, lengths=torch.tensor([0.2,0.5,0.7,1.0])) FNrg?cst||_||_||_| |_||_||_||_||_ | |_ | |_ | |_ ||_ | |_|d|_d|_t|j ts@td|j dkrItd|j durX|j |j krXtd|j durg|j |j krgtd|dursdgt| }nt|ts|tdt|t| krtdd d t| |D} d d t| D|_t|jdkrtd |jdkrtd|jdkrd|_td|j|jkrtd|j|_i|_|jD] \}}t|j|j|<qdS)NTz"repeat_augment must be an integer.rz&repeat_augment must be greater than 0.zBaugment_end_index must be smaller or equal to augment_start_index.z@concat_end_index must be smaller or equal to concat_start_index.z$enable_augmentations must be a list.z@enable_augmentations must have the same length as augmentations.cSsg|]\}}|r|qSr).0augenabledrrZ/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/speechbrain/augment/augmenter.py s z&Augmenter.__init__..cSs"i|] \}}|jjt||qSr) __class____name__str)ri augmentationrrr z&Augmenter.__init__..zBNo augmentation is applied because the augmentation list is empty.zCNo augmentations applied because max_augmentations is non-positive.z;min_augmentations is negative. Modified to be non-negative.z`min_augmentations is greater than max_augmentations. min_augmentations set to max_augmentations.)super__init__parallel_augmentparallel_augment_fixed_bsconcat_original augmentationsmin_augmentationsmax_augmentationsshuffle_augmentationsaugment_start_indexaugment_end_indexconcat_start_indexconcat_end_indexrepeat_augment augment_probcheck_min_max_augmentationsnum_augmentations do_augment isinstanceint ValueErrorlenlistzip enumerateloggerwarningrequire_lengthsitemsrforward)selfrrrrrrr rrrrr!renable_augmentationsaug_keyaug_funr rr rhs           zAugmenter.__init__cCsB|}|}g}g}|}t|D]{\} } |j| } t|jd} |jrG|jrGtd|jdt|d tj } | | }| | d}| ||} |j | rY| || df|| d}n| || df}t |t rut|dkrq|\}}ntd|js|}|| }q||||q|jr|||\}}||fS|}|}||fS)a-Applies data augmentation on the selected augmentations. Arguments --------- x : torch.Tensor (batch, time, channel) input to augment. lengths : torch.Tensor The length of each sequence in the batch. selected_augmentations: dict Dictionary containing the selected augmentation to apply. Returns ------- output : torch.Tensor Augmented outputs. output_lengths : torch.Tensor The corresponding length of each output. rr.)lengthszEThe function must return max two arguments (Tensor, Length[optional]))r+rtorcharangeshaperrlinspacer(tor&r.r%tupler'appendconcatenate_outputs)r1xr6selected_augmentations next_input next_lengthsoutputoutput_lengths out_lengthsk augment_name augment_funidx idx_startstop idx_startidx_stopoutrrr augmentsR           zAugmenter.augmentc Csd|_t|jkrd|_||fS|}|}|jdur#t|j|jdn|jd|_|j|jdkr=d|_t d||fSt j |j |j dd|jd|_t|j}|jdksc|jdksct|dkrjd|_||fS|jrrt||d|j}||j|j}||j|j}g}g}|j |_|jr|j|jdkrd|_n-d|_|jdurt|j|jdn|jd|_|||j|j|||j|jt|jD]} ||||\} } || || q|||\} } | | fS) aApplies data augmentation. Arguments --------- x : torch.Tensor (batch, time, channel) input to augment. lengths : torch.Tensor The length of each sequence in the batch. Returns ------- output : torch.Tensor Augmented outputs. output_lengths : torch.Tensor The corresponding length of each output. TFNrzNo augmentation is applied because the augmentation start index is greater than or equal to the number of examples in the input batch.rr)lowhighsizedevice) r$randomr!rminr:augment_end_index_batchrr,r-r8randintrrrT N_augmentr)rkeysr r(rshuffler skip_concatrrconcat_end_index_batchr>rangerOr?) r1r@r6 x_original len_originalaugmentations_lstrA output_lstoutput_len_lstrrDrErrr r0s            zAugmenter.forwardcs`tdd|Dfddt||D}fdd|D}tj|dd}tj|dd}||fS)a Concatenate a list of augmented signals, accounting for varying temporal lengths. Padding is applied to ensure all signals can be concatenated. Arguments --------- augment_lst : List of torch.Tensor List of augmented signals to be concatenated. augment_len_lst : List of torch.Tensor List of lengths corresponding to the augmented signals. Returns ------- concatenated_signals : torch.Tensor A tensor containing the concatenated signals. concatenated_lengths : torch.Tensor A tensor containing the concatenated signal lengths. Notes ----- This function takes a list of augmented signals, which may have different temporal lengths due to variations such as speed changes. It pads the signals to match the maximum temporal dimension found among the input signals and rescales the lengths accordingly before concatenating them. css|]}|jdVqdS)rNr:)rrOrrr sz0Augmenter.concatenate_outputs..cs"g|] \}}||jdqSrPrd)rlengthrDmax_lenrr r rz1Augmenter.concatenate_outputs..c s&g|]}t|d|jdfqS)rr)Fpadr:)rrDrgrr r srdim)maxr*r8cat)r1 augment_lstaugment_len_lstrDrErrgr r?s  zAugmenter.concatenate_outputscGs,|js|Sg}|D] }|||q |S)a9 Replicates the labels along the batch axis a number of times that corresponds to the number of augmentations. Indeed parallel and concatenation augmentations alter the time dimension. Arguments --------- *args : tuple Input label tensors to be replicated. Can be a uniq or a list of torch.Tensors. Returns ------- augmented_labels: torch.Tensor Labels corresponding to the augmented input. Returns as many torch.Tensor as given in input. )r$r>replicate_labels)r1argslist_of_augmented_labelslabelsrrr replicate_multiple_labelss z#Augmenter.replicate_multiple_labelscCsz|js|Sg}|jr|js||j|jg}||j|j}|jr,tj |g|j dd}||g|j }tj |dd}|S)a  Replicates the labels along the batch axis a number of times that corresponds to the number of augmentations. Indeed parallel and concatenation augmentations alter the time dimension. Arguments --------- labels : torch.Tensor Input label tensors to be replicated. Returns ------- augmented_labels: torch.Tensor Labels corresponding to the augmented input. Returns as many torch.Tensor as given in input. rrk) r$rr\rr]rrWrr8rnrYr )r1rtaugmented_labelsselected_labelsrrr rqs"   zAugmenter.replicate_labelscCsf|jdurd|_|jdurt|j|_|jt|jkr!t|j|_|jt|jkr1t|j|_dSdS)z=Checks the min_augmentations and max_augmentations arguments.Nr)rrr(r)r1rrr r"s    z%Augmenter.check_min_max_augmentations) r __module__ __qualname____doc__r)rrOr0r?rurqr" __classcell__rrr5r rs.WjI|1,r)rzrUr8torch.nn.functionalnn functionalrispeechbrain.utils.callchainsrspeechbrain.utils.loggerrr r,Modulerrrrr s