o i&+@sddlZddlmZddlZddlmmZddlZddl m Z ddl m Z ddlmZddl m Z mZddlmZddlmZdd lmZGd d d e ZGd d d e ZGdddeeeZdd ddefdededededejdBdedeeBfddZddZdS)N)files)Dataset)load_from_disk)nn)rSampler)tqdm)MelSpec)defaultc@s@eZdZ      ddefddZd d Zd d Zd dZdS) HFDataset]dvocos hf_datasetcCs,||_||_||_t||||||d|_dSN)n_fft hop_length win_lengthn_mel_channelstarget_sample_rate mel_spec_type)datarrrmel_spectrogram)selfrrrrrrrrH/home/ubuntu/.local/lib/python3.10/site-packages/f5_tts/model/dataset.py__init__s  zHFDataset.__init__cCs<|j|}|dd}|dd}|jd||j|jS)Naudioarray sampling_rate)rshaperr)rindexrowr sample_raterrr get_frame_len)s   zHFDataset.get_frame_lencC t|jSNlenrrrrr__len__/ zHFDataset.__len__c Cs|j|}|dd}|dd}|jd|}|dks |dkr,||dt|jSt|}||jkrDtj ||j}||}| d}| |}| d}|d } t|| d S) Nrrr r!333333?rtextmel_specr1)rr" __getitem__r*torch from_numpyfloatr torchaudio transformsResample unsqueezersqueezedict) rr#r$rr%duration audio_tensor resamplerr3r1rrrr42s$       zHFDataset.__getitem__N)r r r rrr)__name__ __module__ __qualname__rrr&r,r4rrrrr s  r c@sPeZdZ         dded ejdBfd d Zd d ZddZddZ dS) CustomDatasetNr r r rrFcustom_datasetmel_spec_modulec CsX||_||_||_||_||_||_||_| |_| s*t| t ||||||d|_ dSdSr) r durationsrrrrrpreprocessed_melr rr) rrErGrrrrrrrHrFrrrrSs*  zCustomDataset.__init__cCs:|jdur|j||j|jS|j|d|j|jS)Nr>)rGrrr)rr#rrrr&vs zCustomDataset.get_frame_lencCr'r(r)r+rrrr,}r-zCustomDataset.__len__c Cs |j|}|d}|d}|d}d|krdkrnnn |dt|j}q|jr4t|d}n1t|\}}|jd dkrJtj|d dd }||j kr[tj ||j } | |}| |}| d }||d S) NT audio_pathr1r>r/r.r0r3r)dimkeepdimr2)rr*rHr5tensorr8loadr"meanrr9r:rr<) rr#r$rIr1r>r3rsource_sample_rater@rrrr4s,     zCustomDataset.__getitem__) Nr r r rrrFN) rArBrCrrModulerr&r,r4rrrrrDRs$ # rDc@sPeZdZdZ ddeededefdd Zd ed dfd d ZddZ ddZ dS)DynamicBatchSamplerabExtension of Sampler that will do the following: 1. Change the batch size (essentially number of sequences) in a batch to ensure that the total number of frames are less than a certain threshold. 2. Make sure the padding efficiency in the batch is high. 3. Shuffle batches each epoch while maintaining reproducibility. rNFsamplerframes_threshold drop_residualc Cs$||_||_||_||_d|_gg}}|jj}t|jddD] } || || fq|j dddg} d} t|d|ddD]9\} } | | |jkra|dksWt | |kra| | | | 7} qBt | dkrl|| | |jkrw| g} | } qBg} d} qB|st | dkr|| ~||_ d |_ dS) NrzPSorting with sampler... if slow, check whether dataset is provided with duration)desccSs|dS)Nr0r)elemrrrsz.DynamicBatchSampler.__init__..)keyzCreating dynamic batches with z audio frames per gpuT) rRrS max_samples random_seedepoch data_sourcerappendr&sortr*batches drop_last) rrRrSrYrZrTindicesr_r\idxbatch batch_frames frame_lenrrrrs@   "       zDynamicBatchSampler.__init__r[returncCs ||_dS)z Sets the epoch for this sampler.N)r[)rr[rrr set_epochs zDynamicBatchSampler.set_epochcsdjdur+t}|jjtjtj|d}fdd|D}t |Sj}t |S)N) generatorcsg|]}j|qSr)r_).0ir+rr z0DynamicBatchSampler.__iter__..) rZr5 Generator manual_seedr[randpermr*r_tolistiter)rgrar_rr+r__iter__s zDynamicBatchSampler.__iter__cCr'r()r*r_r+rrrr,r-zDynamicBatchSampler.__len__)rNF) rArBrC__doc__rintboolrrgrsr,rrrrrQs  - rQpinyinraw dataset_name tokenizer dataset_type audio_typerFmel_spec_kwargsrfcCstd|dkrsttdd|d|}|dkr4z t|d}Wn t|d}Yd }n|d krBt|d }d }t|d ddd } t | } Wdn1s\wY| d} t |f| ||d|}|S|dkrz t|d}Wn t|d}Yt|d ddd } t | } Wdn1swY| d} t |f| |d|}|S|dkrtd| d\} } t t | d| d| ttddd}|S)a dataset_type - "CustomDataset" if you want to use tokenizer name and default data path to load for train_dataset - "CustomDatasetPath" if you just want to pass the full path to a preprocessed dataset without relying on tokenizer zLoading dataset ...rDf5_ttsz ../../data/_rxz/rawz /raw.arrowFmelz /mel.arrowTz/duration.jsonrzutf-8)encodingNr>)rGrHrFCustomDatasetPath)rGrHr zShould manually modify the path of huggingface dataset to your need. May also the corresponding script cuz different dataset may have different format./ztrain.z ../../data)split cache_dir)printstrrjoinpathrDataset_ from_fileopenjsonrMrDrr load_dataset)ryrzr{r|rFr} rel_data_path train_datasetrHf data_dictrGprepostrrrrsh   *rc Csdd|D}tdd|D}|}g}|D]}d||df}tj||dd}||qt|}dd|D}tdd|D} t|||| d S) NcSsg|] }|ddqS)r3r)r<riitemrrrrk6szcollate_fn..cSsg|]}|jdqS)r!)r")rispecrrrrk7rlrr!)valuecSsg|]}|dqS)r1rrrrrrkBcSsg|]}t|qSr)r*rrrrrkCr)r mel_lengthsr1 text_lengths) r5 LongTensoramaxsizeFpadr]stackr=) rc mel_specsrmax_mel_lengthpadded_mel_specsrpadding padded_specr1rrrr collate_fn5s"  r)rimportlib.resourcesrr5torch.nn.functionalr functionalrr8datasetsrrrtorch.utils.datarrf5_tts.model.modulesrf5_tts.model.utilsr r rDlistrurQr=rrPrrrrrrsH       ATO B