o Si@s>dZddlZddlZddlZddlmZddlmZmZm Z m Z ddl m Z ddl mZmZmZmZmZddlmZddlmZmZmZd Zd Zd d d d d d d d d d d d d d ZdddddddddZdedefdede edeedeedef ddZ d'd ed!eedeee eefffd"d#Z d$edefd%d&Z!dS)(a The CMU_INDIC databases were constructed at the Language Technologies Institute at Carnegie Mellon University as phonetically balanced, single speaker databases designed for corpus based speech synthesis research. They are covering major languages spoken in the Indian subcontinet. The distributions include the raw waveform files, with transcriptions in the language's native script (etc/txt.done.data file), and also complete built synthesis voices from these databases using CMU Clustergen statistical parameteric speech synthesizer. Complete android voices for CMU Flite are voice built from these databases are available in the Google Play store. You can hear voices built from these databases here CMU INDIC Databases All 13 voices are available from packed do_indic a script to download and build a full voice from these databases (assuming FestVox build tools are all installed. These packed versions contain only the waveform files, and the txt.done.data file. Acknowledgements These datasets were collected and developed with help from Hear2Read. We acknowledge their contributions to making these practical languages for festvox. Special Thanks for to Suresh Bazaj. Source: http://festvox.org/cmu_indic/ N)Path)DictOptionalSequenceUnion)tqdm) Recording RecordingSetSupervisionSegmentSupervisionSet$validate_recordings_and_supervisions) fix_manifests)Pathlikeresumable_download safe_extractzhttp://festvox.org/h2r_indic/) ben_rmguj_adguj_dpguj_kthin_abkan_plvmar_aupmar_slppan_amptam_sdrtel_kpntel_sktel_ssFM) rrrrrrrrrrrrrBengaliGujaratiKannadaHindiMarathiPunjabiTamilTelugu)bengujkanhinmarpantamtel.F target_dirspeakersforce_downloadbase_urlreturnc Cst|}|jdddt|ddD]\}d|}|d}||}||}||} | d} | r@td|d | d qt|||d tj| dd t |} t | |d Wdn1sdwY| q|S)a Download and untar the CMU Indic dataset. :param target_dir: Pathlike, the path of the dir to storage the dataset. :param speakers: a list of speakers to download. By default, downloads all. :param force_download: Bool, if True, download the tars no matter if the tars exist. :param base_url: str, the url of CMU Arctic download site. :return: the path to downloaded and extracted directory with data. T)parentsexist_okz(Downloading/unpacking CMU Indic speakers)desc cmu_indic_z.tar.bz2z .completedzSkiping z because z exists.)filenamer3) ignore_errors)pathN) rmkdirris_filelogginginforshutilrmtreetarfileopenrtouch) r1r2r3r4spknametar_namefull_urltar_pathpart_dircompleted_detectortarrNL/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/recipes/cmu_indic.pydownload_cmu_indicTs&     rP corpus_dir output_dircCst|}|sJd|tdd|dD}g}|dD]}|}t|jjj }| dd}zt |jdd  d d  }Wnd }Y|D]U} | d d} | j d d\} } |d| } t|} d| v} d }| s~|d uri}| r| |d<|d ur||d<|t| | d|| j| dd | rdn| |t||d qWq#t|}t||\}}t|||d urt|}||d||d||dS)a- Prepares and returns the CMU Indic manifests, which consist of Recordings and Supervisions. :param corpus_dir: Pathlike, the path of the data dir. :param output_dir: Pathlike, the path where to write the manifests. :return: a dict of {'recordings': ..., 'supervisions': ...} zNo such directory: css4|]}tj|t|jjjd|jdVqdS)-) recording_idN)r from_file _get_speakerparentrGstem).0wavrNrNrO s z$prepare_cmu_indic..z*.wavz txt.done.data_rz voice.featszage NmaxsplitrSarcticaccentage"English) idrTstartdurationtextlanguagespeakergendercustomz!cmu-indic_recordings_all.jsonl.gzz#cmu-indic_supervisions_all.jsonl.gz) recordings supervisions)ris_dirr from_recordingsrglob read_text splitlinesrVrWrGsplitintreplacestrip LANGUAGE_MAPappendr rj GENDER_MAPgetr from_segmentsr r to_file)rQrRrprqr<linesrm lang_coderelseg_idrkrl is_englishrorNrNrOprepare_cmu_indicysp          rdirnamecCs|jddddS)Nr\r_ra)rw)rrNrNrOrVsrV)N)"__doc__r?rArCpathlibrtypingrrrr tqdm.autorlhotserr r r r lhotse.qar lhotse.utilsrrrBASE_URLSPEAKERSr}r{strboolrPrrVrNrNrNrOsv     ' W