o ziE@sddlmZddlmZddlmZddlmZddlmZddlZddl m Z ddl m Z d d l mZd d l mZGd d d ZddZefdefddZGdddZGdddeZGddde ZGddde ZdS))Path)Callable)Dict)List)UnionN)SequentialSampler)DistributedSampler) AudioSignal)utilc@seZdZdZddddejddfdeedeede d ed eed e d e fd dZ      dde dedede dede de de fddZ dS) AudioLoaderaPLoads audio endlessly from a list of audio sources containing paths to audio files. Audio sources can be folders full of audio files (which are found via file extension) or by providing a CSV file which contains paths to audio files. Parameters ---------- sources : List[str], optional Sources containing folders, or CSVs with paths to audio files, by default None weights : List[float], optional Weights to sample audio files from each source, by default None relative_path : str, optional Path audio should be loaded relative to, by default "" transform : Callable, optional Transform to instantiate alongside audio sample, by default None ext : List[str] List of extensions to find audio within each source by. Can also be a file name (e.g. "vocals.wav"). by default ``['.wav', '.flac', '.mp3', '.mp4']``. shuffle: bool Whether to shuffle the files within the dataloader. Defaults to True. shuffle_state: int State to use to seed the shuffle of the files. NTrsourcesweights transform relative_pathextshuffle shuffle_statec CsXtj|||d|_ddt|jD|_|r!t|}||j||_||_||_ dS)N)rrcSs*g|]\}}tt|D]}||fq qS)rangelen).0src_idxsrcitem_idxrrL/home/ubuntu/.local/lib/python3.10/site-packages/audiotools/data/datasets.py :s z(AudioLoader.__init__..) r read_sources audio_lists enumerate audio_indices random_staterrrr) selfrrrrrrrstaterrr__init__,s    zAudioLoader.__init__ sample_ratedurationloudness_cutoff num_channelsoffset source_idxr global_idxc CsZ|dur|durz |j||} Wn.ddi} Yn%| dur3|j| t|j\}}|j||} n tj||j|jd\} }}| d} t|||} | dkrd|dur]tj| |||d} nt| ||d} |dkrl| } | |} | j |kr| t ||} | D] \} }|| j| <q| ||t|j|t| d}|jdur|jj|| d|d <|S) Npathnone)p)r)r$r*)r,r)r')signalr-rsourcer/)r2transform_args)rr!rr choose_from_list_of_listsrr zerossalient_excerptto_monoresampler) zero_pad_tointitemsmetadatastrrr instantiate)r#r$r(r)r*r+r,r-rr. audio_infor/r2kvitemrrr__call__GsX          zAudioLoader.__call__)r&r'NNNN)__name__ __module__ __qualname____doc__r AUDIO_EXTENSIONSrr>floatrboolr;r%rDrrrrr s^    r cCst|jt|jkSN)rparent)xyrrrdefault_matcherrPmatchercCsz|tdd|D}t|D]*\}}|D]#}|t|kr&|ddiq|||d|ds9||ddiqq|S)NcSg|]}t|qSrrrlrrrrzalign_lists..r/r0)npargmaxr rappendinsert)listsrR longest_listirNrVrrr align_listss r_c@seZdZdZddddddddedf d eeeeee effd e d e d e d e de de de de de de de fddZddZddZeddeeefde fddZdS) AudioDataseta#Loads audio from multiple loaders (with associated transforms) for a specified number of samples. Excerpts are drawn randomly of the specified duration, above a specified loudness threshold and are resampled on the fly to the desired sample rate (if it is different from the audio source sample rate). This takes either a single AudioLoader object, a dictionary of AudioLoader objects, or a dictionary of AudioLoader objects. Each AudioLoader is called by the dataset, and the result is placed in the output dictionary. A transform can also be specified for the entire dataset, rather than for each specific loader. This transform can be applied to the output of all the loaders if desired. AudioLoader objects can be specified as aligned, which means the loaders correspond to multitrack audio (e.g. a vocals, bass, drums, and other loader for multitrack music mixtures). Parameters ---------- loaders : Union[AudioLoader, List[AudioLoader], Dict[str, AudioLoader]] AudioLoaders to sample audio from. sample_rate : int Desired sample rate. n_examples : int, optional Number of examples (length of dataset), by default 1000 duration : float, optional Duration of audio samples, by default 0.5 loudness_cutoff : float, optional Loudness cutoff threshold for audio samples, by default -40 num_channels : int, optional Number of channels in output audio, by default 1 transform : Callable, optional Transform to instantiate alongside each dataset item, by default None aligned : bool, optional Whether the loaders should be sampled in an aligned manner (e.g. same offset, duration, and matched file name), by default False shuffle_loaders : bool, optional Whether to shuffle the loaders before sampling from them, by default False matcher : Callable How to match files from adjacent audio lists (e.g. for a multitrack audio loader), by default uses the parent directory of each file. without_replacement : bool Whether to choose files with or without replacement, by default True. Examples -------- >>> from audiotools.data.datasets import AudioLoader >>> from audiotools.data.datasets import AudioDataset >>> from audiotools import transforms as tfm >>> import numpy as np >>> >>> loaders = [ >>> AudioLoader( >>> sources=[f"tests/audio/spk"], >>> transform=tfm.Equalizer(), >>> ext=["wav"], >>> ) >>> for i in range(5) >>> ] >>> >>> dataset = AudioDataset( >>> loaders = loaders, >>> sample_rate = 44100, >>> duration = 1.0, >>> transform = tfm.RescaleAudio(), >>> ) >>> >>> item = dataset[np.random.randint(len(dataset))] >>> >>> for i in range(len(loaders)): >>> item[i]["signal"] = loaders[i].transform( >>> item[i]["signal"], **item[i]["transform_args"] >>> ) >>> item[i]["signal"].widget(i) >>> >>> mix = sum([item[i]["signal"] for i in range(len(loaders))]) >>> mix = dataset.transform(mix, **item["transform_args"]) >>> mix.widget("mix") Below is an example of how one could load MUSDB multitrack data: >>> import audiotools as at >>> from pathlib import Path >>> from audiotools import transforms as tfm >>> import numpy as np >>> import torch >>> >>> def build_dataset( >>> sample_rate: int = 44100, >>> duration: float = 5.0, >>> musdb_path: str = "~/.data/musdb/", >>> ): >>> musdb_path = Path(musdb_path).expanduser() >>> loaders = { >>> src: at.datasets.AudioLoader( >>> sources=[musdb_path], >>> transform=tfm.Compose( >>> tfm.VolumeNorm(("uniform", -20, -10)), >>> tfm.Silence(prob=0.1), >>> ), >>> ext=[f"{src}.wav"], >>> ) >>> for src in ["vocals", "bass", "drums", "other"] >>> } >>> >>> dataset = at.datasets.AudioDataset( >>> loaders=loaders, >>> sample_rate=sample_rate, >>> duration=duration, >>> num_channels=1, >>> aligned=True, >>> transform=tfm.RescaleAudio(), >>> shuffle_loaders=True, >>> ) >>> return dataset, list(loaders.keys()) >>> >>> train_data, sources = build_dataset() >>> dataloader = torch.utils.data.DataLoader( >>> train_data, >>> batch_size=16, >>> num_workers=0, >>> collate_fn=train_data.collate, >>> ) >>> batch = next(iter(dataloader)) >>> >>> for k in sources: >>> src = batch[k] >>> src["transformed"] = train_data.loaders[k].transform( >>> src["signal"].clone(), **src["transform_args"] >>> ) >>> >>> mixture = sum(batch[k]["transformed"] for k in sources) >>> mixture = train_data.transform(mixture, **batch["transform_args"]) >>> >>> # Say a model takes the mix and gives back (n_batch, n_src, n_time). >>> # Construct the targets: >>> targets = at.AudioSignal.batch([batch[k]["transformed"] for k in sources], dim=1) Similarly, here's example code for loading Slakh data: >>> import audiotools as at >>> from pathlib import Path >>> from audiotools import transforms as tfm >>> import numpy as np >>> import torch >>> import glob >>> >>> def build_dataset( >>> sample_rate: int = 16000, >>> duration: float = 10.0, >>> slakh_path: str = "~/.data/slakh/", >>> ): >>> slakh_path = Path(slakh_path).expanduser() >>> >>> # Find the max number of sources in Slakh >>> src_names = [x.name for x in list(slakh_path.glob("**/*.wav")) if "S" in str(x.name)] >>> n_sources = len(list(set(src_names))) >>> >>> loaders = { >>> f"S{i:02d}": at.datasets.AudioLoader( >>> sources=[slakh_path], >>> transform=tfm.Compose( >>> tfm.VolumeNorm(("uniform", -20, -10)), >>> tfm.Silence(prob=0.1), >>> ), >>> ext=[f"S{i:02d}.wav"], >>> ) >>> for i in range(n_sources) >>> } >>> dataset = at.datasets.AudioDataset( >>> loaders=loaders, >>> sample_rate=sample_rate, >>> duration=duration, >>> num_channels=1, >>> aligned=True, >>> transform=tfm.RescaleAudio(), >>> shuffle_loaders=False, >>> ) >>> >>> return dataset, list(loaders.keys()) >>> >>> train_data, sources = build_dataset() >>> dataloader = torch.utils.data.DataLoader( >>> train_data, >>> batch_size=16, >>> num_workers=0, >>> collate_fn=train_data.collate, >>> ) >>> batch = next(iter(dataloader)) >>> >>> for k in sources: >>> src = batch[k] >>> src["transformed"] = train_data.loaders[k].transform( >>> src["signal"].clone(), **src["transform_args"] >>> ) >>> >>> mixture = sum(batch[k]["transformed"] for k in sources) >>> mixture = train_data.transform(mixture, **batch["transform_args"]) ig?Nr&r'FTloadersr( n_examplesr)r,r*r+ralignedshuffle_loadersrRwithout_replacementc st|trddt|D}n t|trd|i}||_||_||_||_||_||_ ||_ ||_ | |_ | |_ | |_| r[t|} tt| djD]fdd| D}t|| qJdSdS)NcSsi|]\}}||qSrr)rr^rVrrr wz)AudioDataset.__init__..rcsg|]}|jqSr)rrUr^rrrrgz)AudioDataset.__init__..) isinstancelistr r rar*r+lengthrr(r)r,rcrdrevaluesrrrr_)r#rar(rbr)r,r*r+rrcrdrRre loaders_list input_listsrrhrr%fs,    zAudioDataset.__init__csrt|}|jdur dn|j}it|j}|jr ||||j|j |j |j |j r.|ndd}|j|d}|di||d<|ddD]3}|j|}|j rt|ddjd}|||dd|ddd|di||<qJt|j}fd d |D|d <|jdur|jj||ddd d <t|dkr|dS)N)r$r(r)r*r+r.rr'r2r,r-r)r,r-rcsi|]}||qSrr)rrArCrrrfrgz,AudioDataset.__getitem__..idx)r$r2r4r)r r"r,rjrakeysrdrr(r)r*r+rercr=updaterr?rpop)r#rpr$r,rq loader_kwargsloaderkeyrror __getitem__sH       zAudioDataset.__getitem__cCs|jSrL)rkr#rrr__len__szAudioDataset.__len__ list_of_dictsn_splitscCstj||dS)aCollates items drawn from this dataset. Uses :py:func:`audiotools.core.util.collate`. Parameters ---------- list_of_dicts : typing.Union[list, dict] Data drawn from each item. n_splits : int Number of splits to make when creating the batches (split into sub-batches). Useful for things like gradient accumulation. Returns ------- dict Dictionary of batched data. )r{)r collate)rzr{rrrr|szAudioDataset.collaterL)rErFrGrHrPrr rrr>r;rJrrKr%rwry staticmethodrjdictr|rrrrr`sTQ     )7"r`c@s*eZdZdefddZddZddZdS) ConcatDatasetdatasetscCs ||_dSrL)r)r#rrrrr%s zConcatDataset.__init__cCstdd|jDS)NcSrSrrT)rdrrrrrWz)ConcatDataset.__len__..)sumrrxrrrryrQzConcatDataset.__len__cCs&|j|t|j}||t|jSrL)rr)r#rpdatasetrrrrwszConcatDataset.__getitem__N)rErFrGrjr%ryrwrrrrrs rc4eZdZdZddeffdd ZfddZZS) ResumableDistributedSamplerzADistributed sampler that can be resumed from a given start index.N start_idxc s6tj|fi||dur||j|_dSd|_dSNr)superr% num_replicasrr#rrkwargs __class__rrr%s"z$ResumableDistributedSampler.__init__c#4ttD] \}}||jkr|Vqd|_dSrr r__iter__rr#r^rprrrr   z$ResumableDistributedSampler.__iter__rLrErFrGrHr;r%r __classcell__rrrrrrcr) ResumableSequentialSamplerz@Sequential sampler that can be resumed from a given start index.Nrc s0tj|fi||dur||_dSd|_dSr)rr%rrrrrr%sz#ResumableSequentialSampler.__init__c#rrrrrrrrrz#ResumableSequentialSampler.__iter__rLrrrrrrrr)pathlibrtypingrrrrnumpyrXtorch.utils.datartorch.utils.data.distributedrcorer r r rPr_r`rrrrrrrs&        { G