o }oi@stddlZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z m Z mZmZddlZddlZddlZddlmZmZddlmZddlmZddlmZdd lmZmZmZm Z dd l!m"Z"gd Z#d e e$d e ej%fddZ&eGdddZ'GdddZ(GdddeZ)edddZ*Gddde)Z+edddZ,Gddde)Z-edd dZ.Gd!d"d"e)Z/dS)#N) OrderedDict namedtuple) dataclass)CallableDictListOptionalTupleTypeUnion) AudioSegmentChannelSelectorType) collectionsflatten)Dataset) AudioSignalEncodedRepresentation LengthsType NeuralType)logging)AudioToTargetDataset!AudioToTargetWithReferenceDataset!AudioToTargetWithEmbeddingDatasetbatchreturnc s|d}t}|D]kfdd|D}t|}g}t||D]E\}}||kr_|jdkr7d||f}n|jdkrGd||ddf}n tddjdtjj |||<| |q!|t |tj |tjd f7}q |S) a"Collate a batch of items returned by __getitem__. Examples for each signal are zero padded to the same length (batch_length), which is determined by the longest example. Lengths of the original signals are returned in the output. Args: batch: List of dictionaries. Each element of the list has the following format ``` { 'signal_0': 1D or 2D tensor, 'signal_1': 1D or 2D tensor, ... 'signal_N': 1D or 2D tensor, } ``` 1D tensors have shape (num_samples,) and 2D tensors have shape (num_channels, num_samples) Returns: A tuple containing signal tensor and signal length tensor (in samples) for each signal. The output has the following format: ``` (signal_0, signal_0_length, signal_1, signal_1_length, ..., signal_N, signal_N_length) ``` Note that the output format is obtained by interleaving signals and their length. rcsg|] }|jdqS))shape).0bsignal^/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/audio/data/audio_to_audio.py Jsz%_audio_collate_fn..zSignal z has unsuported dimensions z1. Currently, only 1D and 2D arrays are supported.)dtype)keystuplemaxzipndim RuntimeErrorrtorchnn functionalpadappendstacktensorint32) rsignalsbatched signal_length batch_lengthb_signals_lenrr1r"r r#_audio_collate_fn(s&  r<c@sFeZdZUeeed<dZeee e fed<dZ eee ed<dS) SignalSetupr6Ndurationchannel_selectors) __name__ __module__ __qualname__rstr__annotations__r>rr floatlistr?r r"r"r"r#r=is r=c@seZdZdZ  dCdededeedefdd Ze d efd d Z e j d efdd Z e d efddZ e j d efddZ e d e fddZej d ee fddZe d e fddZej d ee fddZe d e fddZej d e fddZdejjd eeejffddZdejjd eeejffddZd eeejfd eeejffd!d"Zdejjd eeejffd#d$Zdejjd eeejffd%d&Ze   ' (dDd)eded*eed+ed,eded ej fd-d.Z!e   ' (dDd/e"eded*eed0ee"ed,eded e"ej fd1d2Z#e  dEd)e$ee"efded3ed4eed+eed ej f d5d6Z%e&  dEd)eded3ed4eed+eed ej f d7d8Z'e&d9e$ej e"ej fd ej fd:d;Z(e&d/e"ed e"efdd?Z*e&d@ed ej fdAdBZ+dS)FASRAudioProcessoraClass that processes an example from Audio collection and returns a dictionary with prepared signals. For example, the output dictionary may be the following ``` { 'input_signal': input_signal_tensor, 'target_signal': target_signal_tensor, 'reference_signal': reference_signal_tensor, 'embedding_vector': embedding_vector } ``` Keys in the output dictionary are ordered with synchronous signals given first, followed by asynchronous signals and embedding. Args: sample_rate: sample rate used for all audio signals random_offset: If `True`, offset will be randomized when loading a subsegment from a file. normalization_signal: Normalize all audio with a factor that ensures the signal `example[normalization_signal]` in `process` is in range [-1, 1]. All other audio signals are scaled by the same factor. Default is `None`, corresponding to no normalization. N:0yE> sample_rate random_offsetnormalization_signalepscCs.||_||_||_||_d|_d|_d|_dSN)rIrJrKrL sync_setup async_setupembedding_setup)selfrIrJrKrLr"r"r#__init__s zASRAudioProcessor.__init__rcC|jSrM) _sample_raterQr"r"r#rIzASRAudioProcessor.sample_ratevaluecCs |dkr td|||_dS)Nrz'Sample rate must be positive, received ) ValueErrorrTrQrWr"r"r#rIs cCrSrM_random_offsetrUr"r"r#rJrVzASRAudioProcessor.random_offsetcCs ||_dSrMrZrYr"r"r#rJ cCrS)zReturn the current setup for synchronous signals. Returns: A dataclass containing the list of signals, their duration and channel selectors. ) _sync_setuprUr"r"r#rNzASRAudioProcessor.sync_setupcC6|dus t|tr||_dStdt|d|d)aSetup signals to be loaded synchronously. Args: value: An instance of SignalSetup with the following fields - signals: list of signals (keys of example.audio_signals) which will be loaded synchronously with the same start time and duration. - duration: Duration for each signal to be loaded. If duration is set to None, the whole file will be loaded. - channel_selectors: A list of channel selector for each signal. If channel selector is None, all channels in the audio file will be loaded. NUnexpected type for value .) isinstancer=r]rXtyperYr"r"r#rNs cCrS)zReturn the current setup for asynchronous signals. Returns: A dataclass containing the list of signals, their duration and channel selectors. ) _async_setuprUr"r"r#rOr^zASRAudioProcessor.async_setupcCr_)aSetup signals to be loaded asynchronously. Args: Args: value: An instance of SignalSetup with the following fields - signals: list of signals (keys of example.audio_signals) which will be loaded asynchronously with signals possibly having different start and duration - duration: Duration for each signal to be loaded. If duration is set to None, the whole file will be loaded. - channel_selectors: A list of channel selector for each signal. If channel selector is None, all channels in the audio file will be loaded. Nr`rarb)rcr=rerXrdrYr"r"r#rOs cCrS)z3Setup signals corresponding to an embedding vector.)_embedding_setuprUr"r"r#rPsz!ASRAudioProcessor.embedding_setupcCr_)a$Setup signals corresponding to an embedding vector. Args: value: An instance of SignalSetup with the following fields - signals: list of signals (keys of example.audio_signals) which will be loaded as embedding vectors. Nr`rarb)rcr=rfrXrdrYr"r"r#rPs examplecCs|j|d}|j|d}|S)aProcess an example from a collection of audio examples. Args: example: an example from Audio collection. Returns: An ordered dictionary of signals and their tensors. For example, the output dictionary may be the following ``` { 'input_signal': input_signal_tensor, 'target_signal': target_signal_tensor, 'reference_signal': reference_signal_tensor, 'embedding_vector': embedding_vector } ``` Keys in the output dictionary are ordered with synchronous signals given first, followed by asynchronous signals and embedding. rg)audio) load_audio process_audio)rQrgrir"r"r#processs  zASRAudioProcessor.processcCspt}|jdur||}|||jdur!||}|||jdur0||}|||s6td|S)aGiven an example, load audio from `example.audio_files` and prepare the output dictionary. Args: example: An example from an audio collection Returns: An ordered dictionary of signals and their tensors. For example, the output dictionary may be the following ``` { 'input_signal': input_signal_tensor, 'target_signal': target_signal_tensor, 'reference_signal': reference_signal_tensor, 'embedding_vector': embedding_vector } ``` Keys in the output dictionary are ordered with synchronous signals given first, followed by asynchronous signals and embedding. NzUOutput dictionary is empty. Please use `_setup` methods to setup signals to be loaded) rrNload_sync_signalsupdaterOload_async_signalsrPload_embeddingr-)rQrgoutput sync_signals async_signals embeddingr"r"r#rjs         zASRAudioProcessor.load_audioricCsZ|jr+||j}|jdur|jjng}|D]}||vr*||||j||<q|S)zProcess audio signals available in the input dictionary. Args: audio: A dictionary containing loaded signals `signal: tensor` Returns: An ordered dictionary of signals and their tensors. N)rKabsr*rPr6rL)rQri norm_scale skip_signalsr!r"r"r#rk<s zASRAudioProcessor.process_audiocsjt}fdd|jjD}|j||jj|j|jjj|jd}t |jj|D] \}}t |||<q'|S)zLoad signals with the same start and duration. Args: example: an example from audio collection Returns: An ordered dictionary of signals and their tensors. csg|]}j|qSr") audio_files)rsrhr"r#r$_sz7ASRAudioProcessor.load_sync_signals..)rxr?rIr> fixed_offsetrJ) rrNr6get_samples_synchronizedr?rIr>offsetrJr+r.r4)rQrgrqsync_audio_files sync_samplesr!samplesr"rhr#rmUs  z#ASRAudioProcessor.load_sync_signalsc Cs`t}t|jjD]$\}}|j|j||j|jj||jj||j |j d}t |||<q |S)zLoad each async signal independently, no constraints on starting from the same time. Args: example: an example from audio collection Returns: An ordered dictionary of signals and their tensors. ) audio_filerIr>channel_selectorrzrJ) r enumeraterOr6 get_samplesrxrIr>r?r|rJr.r4)rQrgrqidxr!rr"r"r#roos   z$ASRAudioProcessor.load_async_signalsrFrr>rrzcCs"|j|g|||g||d}|dS)aGet samples from an audio file. For a single-channel signal, the output is shape (num_samples,). For a multi-channel signal, the output is shape (num_samples, num_channels). Args: audio_file: path to an audio file sample_rate: desired sample rate for output samples duration: Optional desired duration of output samples. If `None`, the complete file will be loaded. If set, a segment of `duration` seconds will be loaded. channel_selector: Optional channel selector, for selecting a subset of channels. fixed_offset: Optional fixed offset when loading samples. random_offset: If `True`, offset will be randomized when loading a short segment from a file. The value is randomized between fixed_offset and max_offset (set depending on the duration and fixed_offset). Returns: Numpy array with samples from audio file. The array has shape (num_samples,) for a single-channel signal or (num_channels, num_samples) for a multi-channel signal. )rxrIr>r?rzrJr)r{)clsrrIr>rrzrJrqr"r"r#rs zASRAudioProcessor.get_samplesrxr?c Cs|dur dgt|}|dur|}d}nO||} t| } | |} | dkr0td|d| d||| krMtd| d|d|d | d |}| }n|rZ| |} t|| }n|}t ||}g} t |D]\}}|j ||||||d }| |qi| S) aGet samples from multiple files with the same start and end point. Args: audio_files: list of paths to audio files sample_rate: desired sample rate for output samples duration: Optional desired duration of output samples. If `None`, the complete files will be loaded. If set, a segment of `duration` seconds will be loaded from all files. Segment is synchronized across files, so that start and end points are the same. channel_selectors: Optional channel selector for each signal, for selecting a subset of channels. fixed_offset: Optional fixed offset when loading samples. random_offset: If `True`, offset will be randomized when loading a short segment from a file. The value is randomized between fixed_offset and max_offset (set depending on the duration and fixed_offset). Returns: List with the same size as `audio_files` but containing numpy arrays with samples from each audio file. Each array has shape (num_samples,) or (num_channels, num_samples), for single- or multi-channel signal, respectively. For example, if `audio_files = [path/to/file_1.wav, path/to/file_2.wav]`, the output will be a list `output = [samples_file_1, samples_file_2]`. Nrz Fixed offset zs is larger than shortest file zs.zShortest file (z%s) is less than the desired duration zs + fixed offset z)s. Returned signals will be shortened to z seconds.rrIr| num_samplesr) len get_durationminrXrdebugrandomuniformmathfloorrget_samples_from_filer2)rrxrIr>r?rzrJr|raudio_durationsmin_audio_durationavailable_duration max_offsetrqrrsegment_samplesr"r"r#r{s@#   z*ASRAudioProcessor.get_samples_synchronizedr|rc Cst|tr|j|||||d}|St|tr3g}|D]}|j|||||d}||q||}|S|dur;g}|Stdt|)alGet samples from a single or multiple files. If loading samples from multiple files, they will be concatenated along the channel dimension. Args: audio_file: path or a list of paths. sample_rate: sample rate of the loaded samples offset: fixed offset in seconds num_samples: Optional, number of samples to load. If `None`, all available samples will be loaded. channel_selector: Select a subset of available channels. Returns: An array with shape (samples,) or (channels, samples) rNzUnexpected audio_file type )rcrCget_segment_from_filerFr2list_to_multichannelr-rd) rrrIr|rrra_filea_file_samplesr"r"r#rs4    z'ASRAudioProcessor.get_samples_from_filecCsh|durtj||||d}n tj|||||d}|jjdkr!|jS|jjdkr+|jjStd|jj)aGet a segment of samples from a single audio file. Args: audio_file: path to an audio file sample_rate: sample rate of the loaded samples offset: fixed offset in seconds num_samples: Optional, number of samples to load. If `None`, all available samples will be loaded. channel_selector: Select a subset of available channels. Returns: An array with shape (samples,) or (channels, samples) N)r target_srr|r)rr n_segmentsr|rr%r&zUnexpected samples shape: )r from_filesegment_from_filerr,Tr-r)rrIr|rrsegmentr"r"r#r:s&  z'ASRAudioProcessor.get_segment_from_filer!cCst|ts|St|dkr|St|dkr|dS|djdkr)tj|dd}|S|djdkr9tj|dd}|Std|djd)aConvert a list of signals into a multi-channel signal by concatenating the elements of the list along the channel dimension. If input is not a list, it is returned unmodified. Args: signal: list of arrays Returns: Numpy array obtained by concatenating the elements of the list along the channel dimension (axis=0). rr%)axisr&zUnexpected target with z dimensions.)rcrFrr,npr3 concatenater-)r! mc_signalr"r"r#ris   z&ASRAudioProcessor.list_to_multichannelcCsddt|D}|S)zGet duration for each audio file in `audio_files`. Args: audio_files: list of paths to audio files Returns: List of durations in seconds. cSsg|]}tj|dqS))path)librosar)rfr"r"r#r$sz2ASRAudioProcessor.get_duration..r)rxr>r"r"r#rs zASRAudioProcessor.get_durationcCsBt}t|jjD]\}}|j|}||}t|||<q |S)a Given an example, load embedding from `example.audio_files[embedding]` and return it in a dictionary. Args: example: An example from audio collection Returns: An dictionary of embedding keys and their tensors. )rrrPr6rxload_embedding_vectorr.r4)rQrgrqrr!embedding_filertr"r"r#rps    z ASRAudioProcessor.load_embeddingfilepathcCsR|dr"t|d}t|}Wd|S1swY|Std|)zLoad an embedding vector from a file. Args: filepath: path to a file storing a vector. Currently, it is assumed the file is a npy file. Returns: Array loaded from filepath. z.npyrbNz'Unknown embedding file format in file: )endswithopenrloadr-)rrrtr"r"r#rs   z'ASRAudioProcessor.load_embedding_vector)NrH)NNrF)NN),r@rArB__doc__rEboolrrCrRpropertyrIsetterrJr=rNrOrPrAudio OUTPUT_TYPErr.Tensorrlrjrkrmro classmethodintr rndarrayrrr{r r staticmethodrrrrprr"r"r"r#rGps    &+   )  T 4 .&# rGcseZdZdZeejdeee e ffddZ de j dedeeffdd Zdefd d Zd edee ejffd dZdefddZdeejfddZZS)BaseAudioDatasetaBase class of audio datasets, providing common functionality for other audio datasets. Args: collection: Collection of audio examples prepared from manifest files. audio_processor: Used to process every example from the collection. A callable with `process` method. For reference, please check ASRAudioProcessor. rcCsdS)z+Returns definitions of module output ports.Nr"rUr"r"r# output_typesszBaseAudioDataset.output_types collectionaudio_processor output_typecs t||_||_||_dS)zInstantiates an audio dataset.N)superrRrrr)rQrrr __class__r"r#rRs  zBaseAudioDataset.__init__cCsR|d}||jdkrdS||jdkr||jdStd|d||j)aReturns the number of channels for a particular signal in items prepared by this dictionary. More specifically, this will get the tensor from the first item in the dataset, check if it's a one- or two-dimensional tensor, and return the number of channels based on the size of the first axis (shape[0]). NOTE: This assumes that all examples have the same number of channels. Args: signal_key: string, used to select a signal from the dictionary output by __getitem__ Returns: Number of channels for the selected signal. rr%r&z*Unexpected number of dimension for signal z with shape ) __getitem__r,rr-)rQ signal_keyitemr"r"r# num_channelss zBaseAudioDataset.num_channelsindexcCs|j|}|jj|d}|S)aReturn a single example from the dataset. Args: index: integer index of an example in the collection Returns: Dictionary providing mapping from signal to its tensor. For example: ``` { 'input_signal': input_signal_tensor, 'target_signal': target_signal_tensor, } ``` rh)rrrl)rQrrgrqr"r"r#rs zBaseAudioDataset.__getitem__cCs t|jS)z-Return the number of examples in the dataset.)rrrUr"r"r#__len__ r\zBaseAudioDataset.__len__cCs|jt|S)zCollate items in a batch.)rr<)rQrr"r"r# _collate_fnszBaseAudioDataset._collate_fn)r@rArBrrabcabstractmethodrrrCrrrrrr rrRrrr.rrrr r __classcell__r"r"rr#rs  rAudioToTargetExamplez5input_signal input_length target_signal target_length)typename field_namescseZdZdZ        ddededededeed ed eed eed eed eedeedeeffdd Z e dee ee ffddZ ZS)ra A dataset for audio-to-audio tasks where the goal is to use an input signal to recover the corresponding target signal. Each line of the manifest file is expected to have the following format: .. code-block:: json {"input_key": "path/to/input.wav", "target_key": "path/to/target.wav", "duration": "duration_in_seconds"} Additionally, multiple audio files may be provided for each key in the manifest, for example, .. code-block:: json {"input_key": "path/to/input.wav", "target_key": ["path/to/path_to_target_ch0.wav", "path/to/path_to_target_ch1.wav"], "duration": "duration_in_seconds"} Keys for input and target signals can be configured in the constructor (`input_key` and `target_key`). Args: manifest_filepath: Path to manifest file in a format described above. sample_rate: Sample rate for loaded audio signals. input_key: Key pointing to input audio files in the manifest target_key: Key pointing to target audio files in manifest audio_duration: Optional duration of each item returned by __getitem__. If `None`, complete audio will be loaded. If set, a random subsegment will be loaded synchronously from target and audio, i.e., with the same start and end point. random_offset: If `True`, offset will be randomized when loading a subsegment from a file. max_duration: If audio exceeds this length, do not include in dataset. min_duration: If audio is less than this length, do not include in dataset. max_utts: Limit number of utterances. input_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. target_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. normalization_signal: Normalize audio signals with a scale that ensures the normalization signal is in range [-1, 1]. All audio signals are scaled by the same factor. Supported values are `None` (no normalization), 'input_signal', 'target_signal'. NFmanifest_filepathrI input_key target_keyaudio_durationrJ max_duration min_durationmax_uttsinput_channel_selectortarget_channel_selectorrKc sZ||d} tj|| ||| d}t||| d}tddg|| | gd|_tj||tddS)N) input_signal target_signalmanifest_filesaudio_to_manifest_keyrr max_numberrIrJrKrrr6r>r?rrr)rAudioCollectionrGr=rNrrRr)rQrrIrrrrJrrrrrrKrrrrr"r#rRBs*zAudioToTargetDataset.__init__rcCs\tdt}tdt}t|ddkr|n|tdt|ddkr%|n|tdtdS)aReturns definitions of module output ports. Returns: OrderedDict: Dictionary containing the following items: input_signal: Batched single- or multi-channel input audio signal input_length: Batched original length of each input signal target_signal: Batched single- or multi-channel target audio signal target_length: Batched original length of each target signal BrrCrrr%rr)r input_lengthr target_lengthrrrrrrQ sc_audio_type mc_audio_typer"r"r#rks    z!AudioToTargetDataset.output_typesNFNNNNNNr@rArBrrCrrrErrRrrrrrr"r"rr#rsJ.     )$r!AudioToTargetWithReferenceExamplezWinput_signal input_length target_signal target_length reference_signal reference_lengthc"seZdZdZ           ddedededed ed eed ed eed eedeedeedeedeededeedeef fdd Z e dee ee ffddZ ZS)raq A dataset for audio-to-audio tasks where the goal is to use an input signal to recover the corresponding target signal and an additional reference signal is available. This can be used, for example, when a reference signal is available from - enrollment utterance for the target signal - echo reference from playback - reference from another sensor that correlates with the target signal Each line of the manifest file is expected to have the following format .. code-block:: json {"input_key": "path/to/input.wav", "target_key": "path/to/path_to_target.wav", "reference_key": "path/to/path_to_reference.wav", "duration": "duration_in_seconds"} Keys for input, target and reference signals can be configured in the constructor. Args: manifest_filepath: Path to manifest file in a format described above. sample_rate: Sample rate for loaded audio signals. input_key: Key pointing to input audio files in the manifest target_key: Key pointing to target audio files in manifest reference_key: Key pointing to reference audio files in manifest audio_duration: Optional duration of each item returned by __getitem__. If `None`, complete audio will be loaded. If set, a random subsegment will be loaded synchronously from target and audio, i.e., with the same start and end point. random_offset: If `True`, offset will be randomized when loading a subsegment from a file. max_duration: If audio exceeds this length, do not include in dataset. min_duration: If audio is less than this length, do not include in dataset. max_utts: Limit number of utterances. input_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. target_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. reference_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. reference_is_synchronized: If True, it is assumed that the reference signal is synchronized with the input signal, so the same subsegment will be loaded as for input and target. If False, reference signal will be loaded independently from input and target. reference_duration: Optional, can be used to set a fixed duration of the reference utterance. If `None`, complete audio file will be loaded. normalization_signal: Normalize audio signals with a scale that ensures the normalization signal is in range [-1, 1]. All audio signals are scaled by the same factor. Supported values are `None` (no normalization), 'input_signal', 'target_signal', 'reference_signal'. NFTrrIrr reference_keyrrJrrrrrreference_channel_selectorreference_is_synchronizedreference_durationrKcs|||d}tj||| || d}t|||d}|r'tgd|| | | gd|_ntddg|| | gd|_tdg|g| gd|_tj||tddS) N)rrreference_signalrrrrrrr) rrrGr=rNrOrrRr)rQrrIrrrrrJrrrrrrrrrKrrrrr"r#rRsF  z*AudioToTargetWithReferenceDataset.__init__rc Csztdt}tdt}t|ddkr|n|tdt|ddkr%|n|tdt|ddkr4|n|tdtdS) aReturns definitions of module output ports. Returns: OrderedDict: Dictionary containing the following items: input_signal: Batched single- or multi-channel input audio signal input_length: Batched original length of each input signal target_signal: Batched single- or multi-channel target audio signal target_length: Batched original length of each target signal reference_signal: Batched single- or multi-channel reference audio signal reference_length: Batched original length of each reference signal rrrr%rrr)rrrrrreference_lengthrrr"r"r#rs     z.AudioToTargetWithReferenceDataset.output_types) NFNNNNNNTNNrr"r"rr#rs`9     =$r!AudioToTargetWithEmbeddingExamplezWinput_signal input_length target_signal target_length embedding_vector embedding_lengthcseZdZdZ        ddededededed eed ed eed eed eedeedeedeeffdd Z e dee ee ffddZ ZS)raA dataset for audio-to-audio tasks where the goal is to use an input signal to recover the corresponding target signal and an additional embedding signal. It is assumed that the embedding is in a form of a vector. Each line of the manifest file is expected to have the following format .. code-block:: json {"input_key": "path/to/input.wav", "target_key": "path/to/path_to_target.wav", "embedding_key": "path/to/path_to_reference.npy", "duration": "duration_in_seconds"} Keys for input, target and embedding signals can be configured in the constructor. Args: manifest_filepath: Path to manifest file in a format described above. sample_rate: Sample rate for loaded audio signals. input_key: Key pointing to input audio files in the manifest target_key: Key pointing to target audio files in manifest embedding_key: Key pointing to embedding files in manifest audio_duration: Optional duration of each item returned by __getitem__. If `None`, complete audio will be loaded. If set, a random subsegment will be loaded synchronously from target and audio, i.e., with the same start and end point. random_offset: If `True`, offset will be randomized when loading a subsegment from a file. max_duration: If audio exceeds this length, do not include in dataset. min_duration: If audio is less than this length, do not include in dataset. max_utts: Limit number of utterances. input_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. target_channel_selector: Optional, select subset of channels from each input audio file. If `None`, all channels will be loaded. normalization_signal: Normalize audio signals with a scale that ensures the normalization signal is in range [-1, 1]. All audio signals are scaled by the same factor. Supported values are `None` (no normalization), 'input_signal', 'target_signal'. NFrrIrr embedding_keyrrJrrrrrrKcsj|||d}tj||| || d}t||| d}tddg|| | gd|_tdgd|_tj||td dS) N)rrembedding_vectorrrrrrr)r6r) rrrGr=rNrPrrRr)rQrrIrrrrrJrrrrrrKrrrrr"r#rRGs2 z*AudioToTargetWithEmbeddingDataset.__init__rc Csptdt}tdt}t|ddkr|n|tdt|ddkr%|n|tdttdttdtdS) aReturns definitions of module output ports. Returns: OrderedDict: Dictionary containing the following items: input_signal: Batched single- or multi-channel input audio signal input_length: Batched original length of each input signal target_signal: Batched single- or multi-channel target audio signal target_length: Batched original length of each target signal embedding_vector: Batched embedded vector format embedding_length: Batched original length of each embedding vector rrrr%rr)rD)rrrrrembedding_length)rrrrrrrr"r"r#rus      z.AudioToTargetWithEmbeddingDataset.output_typesrrr"r"rr#r!sN,     .$r)0rrrrrr dataclassesrtypingrrrrr r r rnumpyrr.0nemo.collections.asr.parts.preprocessing.segmentr r +nemo.collections.common.parts.preprocessing#nemo.collections.common.parts.utilsrnemo.core.classesrnemo.core.neural_typesrrrr nemo.utilsr__all__dictrr<r=rGrrrrrrrr"r"r"r#sP $    ASUl