o Si @sdZddlZddlmZddlmZddlmZmZm Z ddl m Z ddl m Z ddlmZmZdd lmZdd lmZmZdd lmZd efd dZ ddedeedeeeee eeffffddZdS)a About the CDSD (Chinese Dysarthric Speech Database) dataset: This database comprises speech data from 24 participants with dysarthria. Among these participants, one recorded an additional 10 hours of speech data, while each recorded one hour, resulting in 34 hours of speech material. To accommodate participants with varying cognitive levels, the text pool primarily consists of content from the AISHELL-1 dataset and speeches by primary and secondary school students. When participants read these texts, they must use a mobile device or the ZOOM F8n multi-track field recorder to record their speeches. In this paper, the authors elucidate the data collection and annotation processes and present an approach for establishing a baseline for dysarthric speech recognition. Furthermore, the authors conducted a speaker-dependent dysarthric speech recognition experiment using an additional 10 hours of speech data from one of the participants. arXiv link: https://arxiv.org/abs/2310.15930v1 N) defaultdict)Path)DictOptionalUnion)tqdm)$validate_recordings_and_supervisions) Recording RecordingSet) fix_manifests)SupervisionSegmentSupervisionSet)PathlikelinecCsH|dd}|dd}|dd}|dd}|d d }|}|S) u Modified from https://github.com/wenet-e2e/wenet/blob/main/examples/multi_cn/s0/local/aishell_data_prep.sh#L54 sed 's/ａ/a/g' | sed 's/ｂ/b/g' | sed 's/ｃ/c/g' | sed 's/ｋ/k/g' | sed 's/ｔ/t/g' > $dir/transcripts.t uａauｂbuｃcuｋkuｔt)replaceupper)rrG/home/ubuntu/.local/lib/python3.10/site-packages/lhotse/recipes/cdsd.pytext_normalizes     r corpus_dir output_dirreturncCs2t|}|sJd||durt|}|jdddtt}ddg}|D]}td|g}g}|d|d }i}|d D]2} t| d d d !} | D]} | j dd\} } t | } | || <qTWdn1sswYqF|d|d}t |dddD]R}|j}|jd}||vrtd|t|dq||}|std|qt|}||t||d|jdd|| ddd}||qt|}t|}t||\}}t|||dur||d|d||d |d||d!||<q(|S)"aL Returns the manifests which consist of the Recordings and Supervisions :param corpus_dir: Pathlike, the path of the data dir. :param output_dir: Pathlike, the path where to write the manifests. :return: a Dict whose key is the dataset part, and the value is Dicts with the keys 'recordings' and 'supervisions'. zNo such directory: NT)parentsexist_ok1h10hzProcessing CDSD subset: after_cattingTextz**/*.txtrzutf-8)encoding)maxsplitAudioz**/*.wavzProcessing audio)desczNo transcript: z has no transcript.zNo such file: grChinese )id recording_idstartdurationchannellanguagespeakertextcdsd_supervisions_z .jsonl.gzcdsd_recordings_) recordings supervisions)ris_dirmkdirrdictlogginginforglobopen readlinesstripsplitrrstempartswarningis_filer from_fileappendr r0rr from_recordingsr from_segmentsr rto_file)rr manifests dataset_partspartr7r8txt_pathtranscript_dict text_pathfridx_transcriptcontentwav_path audio_pathidxr3r4 recordingsegment recording_setsupervision_setrrr prepare_cdsd.sp          r\)N)__doc__r< collectionsrpathlibrtypingrrr tqdm.autorlhotser lhotse.audior r lhotse.qar lhotse.supervisionr r lhotse.utilsrstrrr\rrrrs(