o i@s8ddlmZeddddZeddddZdS))tablestokenizer_classesWhisperTokenizercKsfzddlm}WntdY|dd}|dd}|dd}|d d }|||||d }|S) Nr get_tokenizerJNotice: If you want to use whisper, please `pip install -U openai-whisper`languagetask transcribeis_multilingualT num_languagesc) multilingualr rr )whisper.tokenizerrprintget)kwargsrrr r r tokenizerrV/home/ubuntu/.local/lib/python3.10/site-packages/funasr/tokenizer/whisper_tokenizer.pyrs     SenseVoiceTokenizercKstzddlm}WntdY|dd}|dd}|dd}|dd }|d d}||||||d }|S) Nrrrrr r Tr i-" vocab_path)rr rr r)/funasr.models.sense_voice.whisper_lib.tokenizerrrr)rrrr r r rrrrrrs"      N)funasr.registerrregisterrrrrrrs