o
…wÖiD ã @ sp d dl Z d dlZd dlZd dlZd dlZd dlZd dlmZ d dlZdd„ Z d de
dee
dee
fd d
„Z dS ) é N)ÚOptionalc C s^ |r|n|}t | dƒš}t |dƒ|}|D ]q}|d | d¡d … ¡ }| dd¡ dd¡}| ¡ }|| d¡d d… } tj ||| | d ¡d | d ¡… | d
¡}
tj ||| | d ¡d | d ¡… | d
¡} d d l } | j
|
¡}
| |
|d œ}t
||¡ | d
¡ qW d ƒ n1 sw Y W d ƒ d S W d ƒ d S 1 s¨w Y d S )NÚrÚwú(é zÚ zéþÿÿÿú-ú.wavr )Úaudio_filepathÚdurationÚtextÚ
)ÚopenÚfindÚlowerÚreplaceÚstripÚosÚpathÚjoinÚrfindÚsoxÚ file_infor ÚjsonÚdumpÚwrite)Útranscripts_pathÚ
manifest_pathÚdata_dirÚ mount_dirÚwav_pathÚfinÚfoutÚlineÚ
transcriptÚfile_idÚ
audio_pathÚmounted_audio_pathr r Úmetadata© r* úV/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/utils/notebook_utils.pyÚbuild_manifest s0 $ÿ$ÿ é ÿÿ"ÿr, ú./r Útrain_mount_dirÚtest_mount_dirc C sH t dƒ tj| dd tj | d ¡s#d}t || ¡}t d|› ƒ nt dƒ | d }tj | d ¡sbt |¡}|j | d t d
ƒ t
j
| d dd }|D ]}|d
d… d }d||g} t | ¡ qMt dƒ t dƒ | d }
| d } tj
| ¡s„t|
| | |dƒ t dƒ | d } | d }
tj
|
¡sžt| |
| |dƒ t dƒ t dƒ d
S )aÛ
Function to download the AN4 dataset. This hides pre-processing boilerplate for notebook ASR examples.
Args:
data_dir: Path to store the data.
train_mount_dir: If you plan to mount the dataset, use this to prepend the mount directory to the
audio filepath in the train manifest.
test_mount_dir: If you plan to mount the dataset, use this to prepend the mount directory to the
audio filepath in the test manifest.
z******T)Úexist_okz/an4_sphere.tar.gzzBhttps://dldata-public.s3.us-east-2.amazonaws.com/an4_sphere.tar.gzzDataset downloaded at: zTarfile already exists.z/an4/)r zConverting .sph to .wav...z
/an4/**/*.sph)Ú recursiveNéüÿÿÿr
r zFinished conversion.
******z /an4/etc/an4_train.transcriptionz/an4/train_manifest.jsonzan4/wav/an4_clstkzTraining manifest created.z /an4/etc/an4_test.transcriptionz/an4/test_manifest.jsonzan4/wav/an4test_clstkzTest manifest created.z
***Done***)Úprintr Úmakedirsr ÚexistsÚwgetÚdownloadÚtarfiler Ú
extractallÚglobÚ
subprocessÚrunÚisfiler, )r r. r/ Úan4_urlÚan4_pathÚtarÚsph_listÚsph_pathr! ÚcmdÚtrain_transcriptsÚtrain_manifestÚtest_transcriptsÚ
test_manifestr* r* r+ Ú download_an49 s<
rH )r- NN) r: r r Úos.pathr; r8 Útypingr r6 r, ÚstrrH r* r* r* r+ Ú s
$