o XiZ@sdZddlZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZdd lmZmZmZmZmZmZmZmZgd Zd d d ddZdddddZdededeeeffddZdeefddZ ddededefddZdS)zRModified from https://github.com/openai/whisper/blob/main/whisper/__init__.py N)ListUnion)tqdm) S3TokenizerV2) S3TokenizerV3) S3Tokenizer) load_audiolog_mel_spectrogrammake_non_pad_mask mask_to_biasmerge_tokenized_segments onnx2torch onnx2torch_v3padding)r r r r rrrr z[https://www.modelscope.cn/models/iic/cosyvoice-300m/resolve/master/speech_tokenizer_v1.onnxz`https://www.modelscope.cn/models/iic/CosyVoice-300M-25Hz/resolve/master/speech_tokenizer_v1.onnxz\https://www.modelscope.cn/models/iic/CosyVoice2-0.5B/resolve/master/speech_tokenizer_v2.onnxzmhttps://www.modelscope.cn/models/FunAudioLLM/Fun-CosyVoice3-0.5B-2512/resolve/master/speech_tokenizer_v3.onnx)speech_tokenizer_v1speech_tokenizer_v1_25hzspeech_tokenizer_v2_25hzspeech_tokenizer_v3_25hz@23b5a723ed9143aebfd9ffda14ac4c21231f31c35ef837b6a13bb9e5488abb1e@56285ddd4a83e883ee0cb9f8d69c1089b53a94b1f78ff7e4a0224a27eb4cb486@d43342aa12163a80bf07bffb94c9de2e120a8df2f9917cd2f642e7f4219c6f71@23236a74175dbdda47afc66dbadd5bcb41303c467a57c261cb8539ad9db9208dnamerootreturnc Cstj|ddt|}t|}tj||d}tj|r,tj|s,t|dtj|r^t |d }| }Wdn1sFwYt | |krV|St|dtj|T}t |d?}tt|dd d dd d d } | d} | sn || | t| q~Wdn1swYWdn1swYWdn1swYt |d }t | |krtd|S)NT)exist_okz.onnxz! exists and is not a regular filerbzH exists, but the SHA256 checksum does not match; re-downloading the filewbzContent-LengthPiBizDownloading onnx checkpoint)totalncolsunit unit_scale unit_divisordesci zeModel has been downloaded but the SHA256 checksum does not not match. Please retry loading the model.)osmakedirs_SHA256S_MODELSpathjoinexistsisfile RuntimeErroropenreadhashlibsha256 hexdigestwarningswarnurllibrequesturlopenrintinfogetwriteupdatelen) rrexpected_sha256urldownload_targetf model_bytessourceoutputloopbufferrIH/home/ubuntu/.local/lib/python3.10/site-packages/s3tokenizer/__init__.py _downloadBsb     rKcCs ttS)z%Returns the names of available models)listr*keysrIrIrIrJavailable_modelsrs rN download_rootcCs|durtjtjdd}tjtd|d}|tvr$t||}ntj|r-|}n td|dt d|vrAt |}n d |vrJt |}nt |}| ||S) a Load a S3Tokenizer ASR model Parameters ---------- name : str one of the official model names listed by `s3tokenizer.available_models()`, or path to a model checkpoint containing the model dimensions and the model state_dict. download_root: str path to download the model files; by default, it uses "~/.cache/s3tokenizer" Returns ------- model : S3Tokenizer The S3Tokenizer model instance N~z.cacheXDG_CACHE_HOME s3tokenizerzModel z not found; available models = v3v2)r'r+r, expandusergetenvr*rKr.r/rNrrrinit_from_onnx)rrOdefaultcheckpoint_filemodelrIrIrJ load_modelws&     r[)N) __doc__r2r'r7r5typingrrrs3tokenizer.model_v2rs3tokenizer.model_v3rrZrutilsr r r r r rrr__all__r*r)strbytesrKrNr[rIrIrIrJs@    ( 0