o iw0@sdZddlmZddlmZmZmZddlZddlm Z ddl m Z ddl mZddlmZdd lmZdd lmZdd lmZe eje d krPdd lmZnedddZGdddeZdS)zText-to-speech ESPnet model.)contextmanager)DictOptionalTupleN)parse)check_argument_types) AbsNormalize)InversibleInterface)AbsESPnetModel)AbsTTS)AbsFeatsExtractz1.6.0)autocastTccs dVdS)N)enabledrrL/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/tts/espnet_model.pyr s r c!s eZdZdZdeedeedeedeeoedeeoedeeo#edeffd d Z d d e j d e j de j de j dee j dee j dee j dee j dee j dee j dee j dee j dee j de e j e ee j fe j ffddZ d d e j d e j de j de j dee j dee j dee j dee j dee j dee j dee j dee j dee j de ee j ffddZ d!d e j dee j dee j dee j dee j dee j dee j dee j de ee j ffddZZS)"ESPnetTTSModelz%ESPnet model for text-to-speech task. feats_extract pitch_extractenergy_extract normalizepitch_normalizeenergy_normalizettscsBtsJt||_||_||_||_||_||_||_ dS)z!Initialize ESPnetTTSModel module.N) rsuper__init__rrrrrrr)selfrrrrrrr __class__rrrs  zESPnetTTSModel.__init__Ntext text_lengthsspeechspeech_lengths durationsdurations_lengthspitch pitch_lengthsenergyenergy_lengthsspembssidslidsreturncKstdl|jdur|||\}}n||}}|jdur-|dur-|j|||||d\}}|jdurB| durB|j|||||d\} } |jdurO|||\}}|jdur\|||\}}|jduri|| | \} } Wdn1sswYt||||d}| dur|j| d| dur|j| d| dur|j| d|dur|j||d|jdur|dur|j||d |jdur| dur|j| | d |j j r|j||d |j d i|S) acCaclualte outputs and return the loss tensor. Args: text (Tensor): Text index tensor (B, T_text). text_lengths (Tensor): Text length tensor (B,). speech (Tensor): Speech waveform tensor (B, T_wav). speech_lengths (Tensor): Speech length tensor (B,). duration (Optional[Tensor]): Duration tensor. duration_lengths (Optional[Tensor]): Duration length tensor (B,). pitch (Optional[Tensor]): Pitch tensor. pitch_lengths (Optional[Tensor]): Pitch length tensor (B,). energy (Optional[Tensor]): Energy tensor. energy_lengths (Optional[Tensor]): Energy length tensor (B,). spembs (Optional[Tensor]): Speaker embedding tensor (B, D). sids (Optional[Tensor]): Speaker ID tensor (B, 1). lids (Optional[Tensor]): Language ID tensor (B, 1). kwargs: "utt_id" is among the input. Returns: Tensor: Loss scalar tensor. Dict[str, float]: Statistics to be monitored. Tensor: Weight tensor to summarize losses. FN feats_lengthsr"r#)rrfeatsr-r(r)r*)r"r#r$r%r&r')r r!r) r rrrrrrdictupdaterrequire_raw_speech)rrrr r!r"r#r$r%r&r'r(r)r*kwargsr.r-batchrrrforward4sb )      #   zESPnetTTSModel.forwardcKs|jdur|||\}}n||}}|jdur$|j|||||d\}}|jdur5|j|||||d\} } t||d}|durF|j||d| durQ|j| | d|S)aCaclualte features and return them as a dict. Args: text (Tensor): Text index tensor (B, T_text). text_lengths (Tensor): Text length tensor (B,). speech (Tensor): Speech waveform tensor (B, T_wav). speech_lengths (Tensor): Speech length tensor (B,). durations (Optional[Tensor): Duration tensor. durations_lengths (Optional[Tensor): Duration length tensor (B,). pitch (Optional[Tensor): Pitch tensor. pitch_lengths (Optional[Tensor): Pitch length tensor (B,). energy (Optional[Tensor): Energy tensor. energy_lengths (Optional[Tensor): Energy length tensor (B,). spembs (Optional[Tensor]): Speaker embedding tensor (B, D). sids (Optional[Tensor]): Speaker ID tensor (B, 1). lids (Optional[Tensor]): Language ID tensor (B, 1). Returns: Dict[str, Tensor]: Dict of features. Nr,)r.r-r2r3)rrrr4r5)rrrr r!r"r#r$r%r&r'r(r)r*r7r.r- feats_dictrrr collect_featss2 '    zESPnetTTSModel.collect_featsc Ks$t|d} | dst|jddrK|durtd|jdur)||ddd} n|} |jdur;|| ddd} | j| d|jjrK| j|d | dr|durY| j|d |jdurt|j|dt t | g|dd dd}|j dur| |ddd}|dur| j|d |j dur|j |dt t | g|dd dd}|jdur||ddd}|dur| j|d |dur| j|d|dur| j|d|dur| j|d|jjdi| | } |jdur| ddur|j| dddd} | j| d| S)a_Caclualte features and return them as a dict. Args: text (Tensor): Text index tensor (T_text). speech (Tensor): Speech waveform tensor (T_wav). spembs (Optional[Tensor]): Speaker embedding tensor (D,). sids (Optional[Tensor]): Speaker ID tensor (1,). lids (Optional[Tensor]): Language ID tensor (1,). durations (Optional[Tensor): Duration tensor. pitch (Optional[Tensor): Pitch tensor. energy (Optional[Tensor): Energy tensor. Returns: Dict[str, Tensor]: Dict of outputs. )ruse_teacher_forcinguse_gstFNz#missing required argument: 'speech'r)r.)r )r")r-r")r$)r&r/r0r1feat_gen)feat_gen_denormr)r4getattrr RuntimeErrorrrr5r6rtorch LongTensorlenrrr inferencegetinverseclone)rrr r(r)r*r"r$r& decode_config input_dictr. output_dictr?rrrrEsx                zESPnetTTSModel.inference) NNNNNNNNN)NNNNNNN)__name__ __module__ __qualname____doc__rr rr r rrBTensorrrstrr9r;rE __classcell__rrrrrs         k       I  r)T)rO contextlibrtypingrrrrBpackaging.versionrV typeguardrespnet2.layers.abs_normalizer#espnet2.layers.inversible_interfacer espnet2.train.abs_espnet_modelr espnet2.tts.abs_ttsr +espnet2.tts.feats_extract.abs_feats_extractr __version__torch.cuda.ampr rrrrrs