o %ݫi59@sndZddlmZddlZddlmZmZGdddeeZGdddeZGd d d eZ Gd d d eZ dS) aM Specifies the inference interfaces for text-processing modules. Authors: * Aku Rouhe 2021 * Peter Plantinga 2021 * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Titouan Parcollet 2021 * Abdel Heba 2021 * Andreas Nautsch 2022, 2023 * Pooneh Mousavi 2023 * Sylvain de Langen 2023 * Adel Moumen 2023 * Pradnya Kandarkar 2023 )chainN)EncodeDecodePipelineMixin PretrainedcsveZdZdZdgZdgZfddZeddZedd Z d d Z d d Z ddZ ddZ ddZdddZZS)GraphemeToPhonemea A pretrained model implementation for Grapheme-to-Phoneme (G2P) models that take raw natural language text as an input and Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> text = ("English is tough. It can be understood " ... "through thorough thought though") >>> from speechbrain.inference.text import GraphemeToPhoneme >>> tmpdir = getfixture('tmpdir') >>> g2p = GraphemeToPhoneme.from_hparams('path/to/model', savedir=tmpdir) # doctest: +SKIP >>> phonemes = g2p.g2p(text) # doctest: +SKIP txtphonemescs&tj|i|||dSN)super__init__create_pipelinesload_dependenciesselfargskwargs __class__N/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/inference/text.pyr 3s zGraphemeToPhoneme.__init__cC|jjS)zReturns the available phonemes)hparamsrrrrrr8zGraphemeToPhoneme.phonemescCr)z6Returns the language for which this model is available)rlanguagerrrrr=rzGraphemeToPhoneme.languagecst|t}|r |g}|d|i}|||t|jdr)fdd|jjD|jjdi}| |}|d}| |}|rF|d}|S) asPerforms the Grapheme-to-Phoneme conversion Arguments --------- text: str or list[str] a single string to be encoded to phonemes - or a sequence of strings Returns ------- result: list if a single example was provided, the return value is a single list of phonemes rmodel_input_keyscsi|]}||qSrr).0k model_inputsrr Zs z)GraphemeToPhoneme.g2p..rrNr) isinstancestr encode_input_update_graphemeshasattrrrmodsmodel decode_output _remove_eos)rtextsingleencoded_inputs model_outputsdecoded_outputrrrrg2pBs"      zGraphemeToPhoneme.g2pcCsdd|DS)aRemoves the EOS character from the end of the sequence, if encountered Arguments --------- phonemes : list a list of phomemic transcriptions Returns ------- result : list phonemes, without cSs,g|]}|r|ddkr|ddn|qS)zNr)ritemrrr tsz1GraphemeToPhoneme._remove_eos..r)rrrrrr(fszGraphemeToPhoneme._remove_eoscCsFt|jd}|r|dkrd|}||vr!|||d<dSdSdSdS)Ngrapheme_sequence_moderawgrapheme_encoded_grapheme_encoded)getattrr)rrr2grapheme_encoded_keyrrrr#ys    z#GraphemeToPhoneme._update_graphemescCs*t|jdd}|r||dSdS)z%Loads any relevant model dependenciesdeps_pretrainerN)r6r collect_filesload_collected)rr8rrrr s  z#GraphemeToPhoneme.load_dependenciescCs ||S)atA convenience callable wrapper - same as G2P Arguments --------- text: str or list[str] a single string to be encoded to phonemes - or a sequence of strings Returns ------- result: list if a single example was provided, the return value is a single list of phonemes )r.)rr)rrr__call__s zGraphemeToPhoneme.__call__NcCs |||S)z#Runs enhancement on the noisy input) enhance_batch)rnoisylengthsrrrforwards zGraphemeToPhoneme.forwardr)__name__ __module__ __qualname____doc__INPUT_STATIC_KEYS OUTPUT_KEYSr propertyrrr.r(r#r r;r? __classcell__rrrrrs   $ rcs>eZdZdZdgZfddZddZddZd d ZZ S) ResponseGeneratoraA ready-to-use Response Generator model The class can be used to generate and continue dialogue given the user input. The given YAML must contain the fields specified in the *_NEEDED[] lists. It needs to be used with custom.py to load the expanded model with added tokens like bos,eos, and speaker's tokens. Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. r&csBtj|i||jj|_|jj|_d|jjd|_g|_dS)N)r r rr& tokenizer max_historyhistory_windowhistoryr rrrr s    zResponseGenerator.__init__cCsd|j||}||}|jjj|dd|djddfddd}|d}|j||S)a5 Complete a dialogue given the user's input. Arguments --------- turn: str User input which is the last turn of the dialogue. Returns ------- response Generated response for the user input based on the dialogue history. NrrJT)skip_special_tokensclean_up_tokenization_spaces)rNappend prepare_inputgenerater&rK batch_decodeshape)rturninputshypspredicted_wordsresponserrrgenerate_responses   z#ResponseGenerator.generate_responsecCtz?Users should modify this function according to their own tasks.NotImplementedErrorrrrrrRzResponseGenerator.prepare_inputcCr\r]r^rrrrrSr`zResponseGenerator.generate) r@rArBrCMODULES_NEEDEDr r[rRrSrGrrrrrHs rHc0eZdZdZfddZddZddZZS)GPTResponseGeneratoraaA ready-to-use Response Generator model The class can be used to generate and continue dialogue given the user input. The given YAML must contain the fields specified in the *_NEEDED[] lists. It needs to be used with custom.py to load the expanded GPT model with added tokens like bos,eos, and speaker's tokens. Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> from speechbrain.inference.text import GPTResponseGenerator >>> tmpdir = getfixture("tmpdir") >>> res_gen_model = GPTResponseGenerator.from_hparams(source="speechbrain/MultiWOZ-GPT-Response_Generation", ... pymodule_file="custom.py") # doctest: +SKIP >>> response = res_gen_model.generate_response("I want to book a table for dinner") # doctest: +SKIP cs8tj|i||jj|jj\|_|_|_ |_ dSr) r r r&rKconvert_tokens_to_idsrspecial_tokensboseossystemuserr rrrr szGPTResponseGenerator.__init__cCsB|\}}|jj||jjjd}|j|||d}|S)a Complete a dialogue given the user's input. Arguments --------- inputs: tuple history_bos which is the tokenized history+input values with appropriate speaker token appended before each turn and history_token_type which determines the type of each token based on who is uttered that token (either User or System). Returns ------- response Generated hypothesis for the user input based on the dialogue history. pad_idxbeam)r padding_maskr&rK unk_token_idrSdetach)rrW history_boshistory_token_typermrXrrrrSs zGPTResponseGenerator.generatecsfddjD}fddt|D}|j d}ttt|}ttj g|tj gf}fddt|D}tttj gg|j dj gg}| d| dfS)amConvert user input and previous histories to the format acceptable for GPT model. It appends all previous history and input and truncates it based on max_history value. It then tokenizes the input and generates additional input that determines the type of each token (System or User). Returns ------- history_bos: torch.Tensor Tokenized history+input values with appropriate speaker token appended before each turn. history_token_type: torch.LongTensor Type of each token based on who is uttered that token (either User or System) csg|] }jj|qSr)r&rKencoderrVrrrr1%sz6GPTResponseGenerator.prepare_input..cs.g|]\}}|ddkrjnjg|qSrIr)rirhri encoded_turnrrrr1+sNcs2g|]\}}|ddkrjnjgt|qSrt)rirhlenrurrrr1;s r) rN enumeraterMtorch LongTensorlistrcattensorrfrh unsqueeze)rhistory_tokens_listshistory_input_lists history_idsrphistory_token_type_listsrqrrrrRs4    z"GPTResponseGenerator.prepare_inputr@rArBrCr rSrRrGrrrrrcs   rccrb)Llama2ResponseGeneratoramA ready-to-use Response Generator model The class can be used to generate and continue dialogue given the user input. The given YAML must contain the fields specified in the *_NEEDED[] lists. It needs to be used with custom.py to load the expanded Llama2 model with added tokens like bos,eos, and speaker's tokens. Arguments --------- *args : tuple **kwargs : dict Arguments are forwarded to ``Pretrained`` parent class. Example ------- >>> from speechbrain.inference.text import Llama2ResponseGenerator >>> tmpdir = getfixture("tmpdir") >>> res_gen_model = Llama2ResponseGenerator.from_hparams(source="speechbrain/MultiWOZ-Llama2-Response_Generation", ... pymodule_file="custom.py") # doctest: +SKIP >>> response = res_gen_model.generate_response("I want to book a table for dinner") # doctest: +SKIP cs"ddi}tj|d|i|dS)Ndevicecudarun_opts)r r )rrrrrrrr dsz Llama2ResponseGenerator.__init__cCsF|d|jjj}|jj||jjd}|j||d}|S)aJ Complete a dialogue given the user's input. Arguments --------- inputs: prompt_bos prompted inputs to be passed to llama2 model for generation. Returns ------- response Generated hypothesis for the user input based on the dialogue history. rrjrl) tor&rrrmrK pad_token_idrSro)rrW prompt_bosrmrXrrrrSis z Llama2ResponseGenerator.generatecs|dd}tt|tj}fdd|D}|j d}ttt|}tt j j g|f}| dj ddS)aConvert user input and previous histories to the format acceptable for Llama2 model. It appends all previous history and input and truncates it based on max_history value. It then tokenizes the input and add prompts. Returns ------- prompt_bos: torch.Tensor Tokenized history+input values with appropriate prompt. cSs$|\}}|ddkrd|dS|S)a|add [INST] and [/INST] prompt to the start and end ogf item. Arguments --------- idx_and_item: tuple id and its corresponding text. If the id is even, it is user turn and [ INST] is added. Returns ------- prompt_bos: torch.LongTensor prompted text for one item. rIrz[INST] z [/INST]r) idx_and_itemindexr0rrrgenerate_prompts  z>Llama2ResponseGenerator.prepare_input..generate_promptcsg|]}j|qSr)rKrrrsrrrr1sz9Llama2ResponseGenerator.prepare_input..Nr)dim) r|mapryrNrMrzr{rr}r~rK bos_token_idr)rrpromptsprompt_tokens_lists prompt_idsrrrrrRs z%Llama2ResponseGenerator.prepare_inputrrrrrrMs  r) rC itertoolsrrz speechbrain.inference.interfacesrrrrHrcrrrrrs ;s