o i$@sdZddlmZddlmZmZmZddlZddlm Z ddl m Z ddl mZddlmZdd lmZdd lmZdd lmZe eje d krPdd lmZnedddZGdddeZdS)z&GAN-based text-to-speech ESPnet model.)contextmanager)AnyDictOptionalN)parse)check_argument_types) AbsGANTTS) AbsNormalize)InversibleInterface)AbsGANESPnetModel)AbsFeatsExtractz1.6.0)autocastTccs dVdS)N)enabledrrP/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/espnet_model.pyr s r c!seZdZdZdeedeeoedeedeeoedeedeeo#edeffd d Z d d e j de j de j de j dee j dee j dee j dee j dee j dee j dee j dee j dee j de de eeffddZ d!d e j de j de j de j dee j dee j dee j dee j dee j dee j dee j dee j dee j de ee j ffddZZS)"ESPnetGANTTSModelz/ESPnet model for GAN-based text-to-speech task. feats_extract normalize pitch_extractpitch_normalizeenergy_extractenergy_normalizettscsftsJt||_||_||_||_||_||_||_ t |ds(Jdt |ds1JddS)z$Initialize ESPnetGANTTSModel module. generatorz4generator module must be registered as tts.generator discriminatorz| dur>|j|||||d\} } |jdurK|||\}}|jdurX|||\}}|jdure|| | \} } Wdn1sowYt|||d}|dur|j||d|j j r|j||d|dur|j||d|jdur|dur|j||d|jdur| dur|j| | d | dur|j| d | dur|j| d | dur|j| d |j d i|S)aBReturn generator or discriminator loss with dict format. Args: text (Tensor): Text index tensor (B, T_text). text_lengths (Tensor): Text length tensor (B,). speech (Tensor): Speech waveform tensor (B, T_wav). speech_lengths (Tensor): Speech length tensor (B,). duration (Optional[Tensor]): Duration tensor. duration_lengths (Optional[Tensor]): Duration length tensor (B,). pitch (Optional[Tensor]): Pitch tensor. pitch_lengths (Optional[Tensor]): Pitch length tensor (B,). energy (Optional[Tensor]): Energy tensor. energy_lengths (Optional[Tensor]): Energy length tensor (B,). spembs (Optional[Tensor]): Speaker embedding tensor (B, D). sids (Optional[Tensor]): Speaker ID tensor (B, 1). lids (Optional[Tensor]): Language ID tensor (B, 1). forward_generator (bool): Whether to forward generator. kwargs: "utt_id" is among the input. Returns: Dict[str, Any]: - loss (Tensor): Loss scalar tensor. - stats (Dict[str, float]): Statistics to be monitored. - weight (Tensor): Weight tensor to summarize losses. - optim_idx (int): Optimizer index (0 for G and 1 for D). FN feats_lengthsr%r&)r!r"r.featsr1)r#r$)r%r&r'r(r)r*)r+)r,)r-r) r rrrrrrdictupdaterrequire_raw_speech)rr!r"r#r$r%r&r'r(r)r*r+r,r-r.kwargsr3r1batchrrrforward:sj -     "   zESPnetGANTTSModel.forwardcKsd}|jdur|||\}}|jdur |j|||||d\}}|jdur1|j|||||d\} } i}|dur>|j||d|durI|j||d| durT|j| | d|S)aCalculate features and return them as a dict. Args: text (Tensor): Text index tensor (B, T_text). text_lengths (Tensor): Text length tensor (B,). speech (Tensor): Speech waveform tensor (B, T_wav). speech_lengths (Tensor): Speech length tensor (B, 1). durations (Optional[Tensor): Duration tensor. durations_lengths (Optional[Tensor): Duration length tensor (B,). pitch (Optional[Tensor): Pitch tensor. pitch_lengths (Optional[Tensor): Pitch length tensor (B,). energy (Optional[Tensor): Energy tensor. energy_lengths (Optional[Tensor): Energy length tensor (B,). spembs (Optional[Tensor]): Speaker embedding tensor (B, D). sids (Optional[Tensor]): Speaker index tensor (B, 1). lids (Optional[Tensor]): Language ID tensor (B, 1). Returns: Dict[str, Tensor]: Dict of features. Nr0r2r4r5)rrrr7)rr!r"r#r$r%r&r'r(r)r*r+r,r-r9r3r1 feats_dictrrr collect_featss<&     zESPnetGANTTSModel.collect_feats) NNNNNNNNNT) NNNNNNNNN)__name__ __module__ __qualname____doc__rr r r rrtorchTensorboolrstrrr;r= __classcell__rrrrrs   !       o      r)T)rA contextlibrtypingrrrrBpackaging.versionrV typeguardrespnet2.gan_tts.abs_gan_ttsrespnet2.layers.abs_normalizer #espnet2.layers.inversible_interfacer "espnet2.train.abs_gan_espnet_modelr +espnet2.tts.feats_extract.abs_feats_extractr __version__torch.cuda.ampr rrrrrs