o Si @s0dZddlZddlZddlZddlmZddlmZmZm Z m Z ddl m Z ddl mZmZmZmZmZddlmZddlmZmZmZd Zd Zd d d d d d d d Zdddddddd Zdedefdede edeedeedef ddZ d#dedeedeee eefffddZ d edefd!d"Z!dS)$a The CMU_ARCTIC databases were constructed at the Language Technologies Institute at Carnegie Mellon University as phonetically balanced, US English single speaker databases designed for unit selection speech synthesis research. A detailed report on the structure and content of the database and the recording environment etc is available as a Carnegie Mellon University, Language Technologies Institute Tech Report CMU-LTI-03-177 and is also available here: http://www.festvox.org/cmu_arctic/cmu_arctic_report.pdf The databases consist of around 1150 utterances carefully selected from out-of-copyright texts from Project Gutenberg. The databses include US English male (bdl) and female (slt) speakers (both experinced voice talent) as well as other accented speakers. The 1132 sentence prompt list is available from cmuarctic.data: http://www.festvox.org/cmu_arctic/cmuarctic.data The distributions include 16KHz waveform and simultaneous EGG signals. Full phoentically labelling was perfromed by the CMU Sphinx using the FestVox based labelling scripts. Complete runnable Festival Voices are included with the database distributions, as examples though better voices can be made by improving labelling etc. Note: The Lhotse recipe is currently not downloading or using the phonetic labeling. N)Path)DictOptionalSequenceUnion)tqdm) Recording RecordingSetSupervisionSegmentSupervisionSet$validate_recordings_and_supervisions) fix_manifests)Pathlikeresumable_download safe_extractz%http://festvox.org/cmu_arctic/packed/)aewahwaupawbaxbbdlclbeeyfemgkajmkkspljmlnhrmsrxrslpsltmalefemale)rr"rrrrrz US MidwestUSzCanadian OntariozScottish South EasternIndian.F target_dirspeakersforce_downloadbase_urlreturnc Cst|}|jdddt|ddD]]}d|d}|d}||}||}||} | d} | rAtd |d | d qt|||d tj| dd t |} t | |dWdn1sewY| q|S)a Download and untar the CMU Arctic dataset. :param target_dir: Pathlike, the path of the dir to storage the dataset. :param speakers: a list of speakers to download. By default, downloads all. :param force_download: Bool, if True, download the tars no matter if the tars exist. :param base_url: str, the url of CMU Arctic download site. :return: the path to downloaded and extracted directory with data. T)parentsexist_okz)Downloading/unpacking CMU Arctic speakers)desccmu_us__arcticz.tar.bz2z .completedzSkiping z because z exists.)filenamer*) ignore_errors)pathN) rmkdirris_filelogginginforshutilrmtreetarfileopenrtouch) r(r)r*r+spknametar_namefull_urltar_pathpart_dircompleted_detectortarrFM/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/recipes/cmu_arctic.pydownload_cmu_arcticWs&     rH corpus_dir output_dirc Cs0t|}|sJd|tdd|dD}g}|dD]H}|}t|jjj }|D]6}|dd}|j dd \}} |d |}| t ||d ||j | d d d|t|dt|id q4q#t|}t||\}}t|||durt|}||d||d||dS)a. Prepares and returns the CMU Arctic manifests, which consist of Recordings and Supervisions. :param corpus_dir: Pathlike, the path of the data dir. :param output_dir: Pathlike, the path where to write the manifests. :return: a dict of {'recordings': ..., 'supervisions': ...} zNo such directory: css4|]}tj|t|jjjd|jdVqdS)-) recording_idN)r from_file _get_speakerparentr?stem).0wavrFrFrG s z%prepare_cmu_arctic..z*.wavz txt.done.data)maxsplitrKr"Englishaccent) idrLstartdurationtextlanguagespeakergendercustomNz"cmu-arctic_recordings_all.jsonl.gzz$cmu-arctic_supervisions_all.jsonl.gz) recordings supervisions)ris_dirr from_recordingsrglob read_text splitlinesrNrOr?splitappendr r^replace GENDER_MAPget ACCENT_MAPr from_segmentsr r to_file) rIrJrdrer4linesralseg_idr_rFrFrGprepare_cmu_arctic|sF         rvdirnamecCs|ddS)N_rT)rk)rwrFrFrGrNsrN)N)"__doc__r7r9r;pathlibrtypingrrrr tqdm.autorlhotserr r r r lhotse.qar lhotse.utilsrrrBASE_URLSPEAKERSrnrpstrboolrHrvrNrFrFrFrGsh      ' 9