o ei%F@s4dZddlZddlmZddlZddlmZddlZddlm Z e dZ ddZ d:d d Z d;ddZddZddZddZddZdd8d9Z$dS)?z Data pipeline elements for the G2P pipeline Authors * Loren Lugosch 2020 * Mirco Ravanelli 2020 * Artem Ploujnikov 2021 (minor refactoring only) N)reduce)nn)expand_to_charsz\s{2,}cs0|}dfdd|D}td|}|S)aM Cleans incoming text, removing any characters not on the accepted list of graphemes and converting to uppercase Arguments --------- txt: str the text to clean up graphemes: list a list of graphemes Returns ------- item: DynamicItem A wrapped transformation function c3s|] }|vr|VqdSN.0char graphemesra/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/lobes/models/g2p/dataio.py (sz!clean_pipeline.. )upperjoinRE_MULTI_SPACEsub)txtr resultrr r clean_pipelines rTc#sJ|r|}fdd|D}|V|}|Vt|}|VdS)aEncodes a grapheme sequence Arguments --------- char: str A list of characters to encode. grapheme_encoder: speechbrain.dataio.encoder.TextEncoder a text encoder for graphemes. If not provided, uppercase: bool whether or not to convert items to uppercase Yields ------ grapheme_list: list a raw list of graphemes, excluding any non-matching labels grapheme_encoded_list: list a list of graphemes encoded as integers grapheme_encoded: torch.Tensor csg|] }|jvr|qSr)lab2ind)r graphemegrapheme_encoderrr Dsz%grapheme_pipeline..N)rencode_sequencetorch LongTensor)r r uppercase grapheme_listgrapheme_encoded_listgrapheme_encodedrrr grapheme_pipeline-s    r#rc #stfdd|D}|Vd|durt||n|}|r&t||||} n|j|} | Vt| } | VdS)aA pipeline element that uses a pretrained tokenizer Arguments --------- seq: list List of tokens to encode. tokenizer: speechbrain.tokenizer.SentencePiece a tokenizer instance tokens: str available tokens wordwise: str whether tokenization is performed on the whole sequence or one word at a time. Tokenization can produce token sequences in which a token may span multiple words word_separator: str The substring to use as a separator between words. token_space_index: int the index of the space token char_map: dict a mapping from characters to tokens. This is used when tokenizing sequences of phonemes rather than sequences of characters. A sequence of phonemes is typically a list of one or two-character tokens (e.g. ["DH", "UH", " ", "S", "AW", "N", "D"]). The character map makes it possible to map these to arbitrarily selected characters Yields ------ token_list: list a list of raw tokens encoded_list: list a list of tokens, encoded as a list of integers encoded: torch.Tensor a list of tokens, encoded as a tensor csg|]}|vr|qSrrr tokentokensrr rzz-tokenizer_encode_pipeline..rN)r_map_tokens_item_wordwise_tokenizesp encode_as_idsrr) seq tokenizerr(wordwiseword_separatortoken_space_indexchar_map token_listtokenizer_input encoded_listencodedrr'r tokenizer_encode_pipelineNs ,    r8csL||vr j|Stt||}fdd|D}|gtfdd|S)aTokenizes a sequence wordwise Arguments --------- tokenizer: speechbrain.tokenizers.SentencePiece.SentencePiece a tokenizer instance sequence: iterable the original sequence input_separator: str the separator used in the input sequence token_separator: str the token separator used in the output sequence Returns ------- result: str the resulting tensor cg|]}j|qSr)r,r-r word_tokensr/rr r z&_wordwise_tokenize..cs ||Srr)leftright)sep_listrr s z$_wordwise_tokenize..)r,r-list _split_listr)r/sequenceinput_separatortoken_separatorwords encoded_wordsr)r@r/r r+s  r+csjt|tr |dkr dS||vr t|tr|n|}j|Stt||}fdd|D}||S)aDetokenizes a sequence wordwise Arguments --------- tokenizer: speechbrain.tokenizers.SentencePiece.SentencePiece a tokenizer instance sequence: iterable the original sequence output_separator: str the separator used in the output sequence token_separator: str the token separator used in the output sequence Returns ------- result: torch.Tensor the result rcr9rr, decode_idsr:r<rr rr=z(_wordwise_detokenize..) isinstancestrrBtolistr,rJrCr)r/rDoutput_separatorrF sequence_listrGrHrr<r _wordwise_detokenizes   rPccsh|dur0d}t|D]\}}||kr||d|V|}q ||dkr2||ddVdSdSdS)z Splits a sequence (such as a tensor) by the specified separator Arguments --------- items: sequence any sequence that supports indexing separator: str the separator token Yields ------ item N) enumerate)items separatorlast_idxidxitemrrr rCs rCcCsx|dur tjj}||krd|jvr|jdd|dnd|jvr*|jdd||dd|jvr3||j|dd |S) a' Initializes the phoneme encoder with EOS/BOS sequences Arguments --------- tokens: list a list of tokens encoder: speechbrain.dataio.encoder.TextEncoder. a text encoder instance. If none is provided, a new one will be instantiated bos_index: int the position corresponding to the Beginning-of-Sentence token eos_index: int the position corresponding to the End-of-Sentence Returns ------- encoder: speechbrain.dataio.encoder.TextEncoder an encoder Nz ) bos_label eos_label bos_indexzz)rYrZr[ eos_indexzF)sequence_input)sbdataioencoder TextEncoderrinsert_bos_eosadd_unkupdate_from_iterable)r(r`r[r\rrr enable_eos_boss*    reccs,|V||}|Vt|}|VdS)aEncodes a sequence of phonemes using the encoder provided Arguments --------- phn: list List of phonemes phoneme_encoder: speechbrain.datio.encoder.TextEncoder a text encoder instance (optional, if not provided, a new one will be created) Yields ------ phn: list the original list of phonemes phn_encoded_list: list encoded phonemes, as a list phn_encoded: torch.Tensor encoded phonemes, as a tensor N)rrr)phnphoneme_encoderphn_encoded_list phn_encodedrrr phoneme_pipelines    rjccsv||}t|st|}|Vtt|V||}t|s,t|}|Vtt|VdS)a}Adds BOS and EOS tokens to the sequence provided Arguments --------- seq: torch.Tensor the source sequence encoder: speechbrain.dataio.encoder.TextEncoder an encoder instance Yields ------ seq_eos: torch.Tensor the sequence, with the EOS token added seq_bos: torch.Tensor the sequence, with the BOS token added N)prepend_bos_indexr is_tensortensorlonglenappend_eos_index)r.r`seq_bosseq_eosrrr add_bos_eos1s        rscCs |||S)aPerforms a Beam Search on the phonemes. This function is meant to be used as a component in a decoding pipeline Arguments --------- char_lens: torch.Tensor the length of character inputs encoder_out: torch.Tensor Raw encoder outputs beam_searcher: speechbrain.decoders.seq2seq.S2SBeamSearcher a SpeechBrain beam searcher instance Returns ------- hyps: list hypotheses scores: list confidence scores associated with each hypotheses r) char_lens encoder_out beam_searcherrrr beam_search_pipelineNs rwcC ||S)a#Decodes a sequence of phonemes Arguments --------- hyps: list hypotheses, the output of a beam search phoneme_encoder: speechbrain.datio.encoder.TextEncoder a text encoder instance Returns ------- phonemes: list the phoneme sequence  decode_ndim)hypsrgrrr phoneme_decoder_pipelinees r|cCs ddtt|t|dDS)zProduces a list of consecutive characters Arguments --------- start_char: str the starting character end_char: str the ending characters Returns ------- char_range: str the character range cSsg|]}t|qSrchr)r rWrrr rzchar_range..rR)rangeord) start_charend_charrrr char_rangews rcCsLtddtdd}ttdd|}tt||dt|}d|d<|S) awBuilds a map that maps arbitrary tokens to arbitrarily chosen characters. This is required to overcome the limitations of SentencePiece. Arguments --------- tokens: list a list of tokens for which to produce the map Returns ------- token_map: dict a dictionary with original tokens as keys and new mappings as values AZazcSs|dkS)Nrrr}rrr rAsz&build_token_char_map..Nr)rrBfilterdictzipro)r(charsvalues token_maprrr build_token_char_maps rcCsdd|DS)zExchanges keys and values in a dictionary Arguments --------- map_dict: dict a dictionary Returns ------- reverse_map_dict: dict a dictionary with keys and values flipped cSsi|]\}}||qSrr)r keyvaluerrr szflip_map..)rT)map_dictrrr flip_maps rcCrx)aDDecodes a sequence using a tokenizer. This function is meant to be used in hparam files Arguments --------- seq: torch.Tensor token indexes encoder: sb.dataio.encoder.TextEncoder a text encoder instance Returns ------- output_seq: list a list of lists of tokens ry)r.r`rrr text_decodes rcs8fdd}fdd}|r|n|fdd}|S)aReturns a function that recovers the original sequence from one that has been tokenized using a character map Arguments --------- char_map: dict a character-to-output-token-map tokenizer: speechbrain.tokenizers.SentencePiece.SentencePiece a tokenizer instance token_space_index: int the index of the "space" token wordwise: bool Whether to apply detokenize per word. Returns ------- f: callable the tokenizer function cst|dS)z+Detokenizes the sequence one word at a timer)rPrX)r2r/rr detokenize_wordwisesz0char_map_detokenize..detokenize_wordwisecsj|S)zDetokenizes the entire sequencerIrr<rr detokenize_regularsz/char_map_detokenize..detokenize_regularcs fdd|D}t|}|S)zThe tokenizer functioncsg|]}|qSrrr rX) detokenizerr rrz2char_map_detokenize..f..)_map_tokens_batch)r(decoded_tokens mapped_tokens)r3rrr fs zchar_map_detokenize..fr)r3r/r2r0rrrr)r3rr2r/r char_map_detokenizes   rcfdd|DS)aPerforms token mapping, in batch mode Arguments --------- tokens: iterable a list of token sequences char_map: dict a token-to-character mapping Returns ------- result: list a list of lists of characters csg|] }fdd|DqS)cg|]}|qSrrrr3rr rrz0_map_tokens_batch...rrrrr rsz%_map_tokens_batch..rr(r3rrr rrcr)zMaps tokens to characters, for a single item Arguments --------- tokens: iterable a single token sequence char_map: dict a token-to-character mapping Returns ------- result: list a list of tokens crrrrrrr r rz$_map_tokens_item..rrrrr r*rr*cs4eZdZdZfddZddZfddZZS)LazyInitzA lazy initialization wrapper Arguments --------- init : callable The function to initialize the underlying object cs td|_||_d|_dSr)super__init__instanceinitdevice)selfr __class__rr rs  zLazyInit.__init__cCs|jdur ||_|jS)zEInitializes the object instance, if necessary and returns it.N)rr)rrrr __call__s  zLazyInit.__call__cs>t||jdur||_t|jdr|j||_|S)zMoves the underlying object to the specified device Arguments --------- device : str | torch.device the device Returns ------- self Nto)rrrrhasattr)rrrrr r&s   z LazyInit.to)__name__ __module__ __qualname____doc__rrr __classcell__rrrr rs  rcCst|S)aLA wrapper to ensure that the specified object is initialized only once (used mainly for tokenizers that train when the constructor is called Arguments --------- init: callable a constructor or function that creates an object Returns ------- instance: object the object instance )r)rrrr lazy_init:srcCs|dkr|S|d|S)aHDetermines the key to be used for sequences (e.g. graphemes/phonemes) based on the naming convention Arguments --------- key: str the key (e.g. "graphemes", "phonemes") mode: str the mode/suffix (raw, eos/bos) Returns ------- key if ``mode=="raw"`` else ``f"{key}_{mode}"`` raw_r)rmoderrr get_sequence_keyLsrcCs||}dd|DS)aConverts a batch of phoneme sequences (a single tensor) to a list of space-separated phoneme label strings, (e.g. ["T AY B L", "B UH K"]), removing any special tokens Arguments --------- phns: torch.Tensor a batch of phoneme sequences decoder: Callable Converts tensor to phoneme label strings. Returns ------- result: list a list of strings corresponding to the phonemes provided cSsg|] }dt|qS)r)rremove_specialrrrr rqsz%phonemes_to_label..r)phnsdecoder phn_decodedrrr phonemes_to_label^srcCsdd|DS)aRemoves any special tokens from the sequence. Special tokens are delimited by angle brackets. Arguments --------- phn: list a list of phoneme labels Returns ------- result: list the original list, without any special tokens cSsg|]}d|vr|qS).r)rfrrr rtsrc CsJd}|r#||}|jd}t|d|d|d|dd}|S)aApplies word embeddings, if applicable. This function is meant to be used as part of the encoding pipeline Arguments --------- txt: str the raw text grapheme_encoded: torch.Tensor the encoded graphemes grapheme_encoded_len: torch.Tensor encoded grapheme lengths grapheme_encoder: speechbrain.dataio.encoder.TextEncoder the text encoder used for graphemes word_emb: callable the model that produces word embeddings use_word_emb: bool a flag indicated if word embeddings are to be applied Returns ------- char_word_emb: torch.Tensor Word embeddings, expanded to the character dimension Nrr)embr.seq_lenr1) embeddingsrr unsqueezesqueeze) rr"grapheme_encoded_lenrword_emb use_word_emb char_word_emb raw_word_embword_separator_idxrrr word_emb_pipelines  r)NT)Trr$Nr)NN)NNN)%rre functoolsrrr speechbrainr^speechbrain.wordemb.utilrcompilerrr#r8r+rPrCrerjrsrwr|rrrrrrr*Modulerrrrrrrrrr sP     % ?# -  **