o i @sPdZddlZddlmZddlZddlmZddlmZGdddej j Z dS)z[Text encoder module in VITS. This code is based on https://github.com/jaywalnut310/vits. N)Tuple)Encoder)make_non_pad_maskc#seZdZdZ          d(dedededededededededededededededed ef"fd!d" Zd#e j d$e j d%e e j e j e j e j ffd&d'Z Z S)) TextEncoderaText encoder module in VITS. This is a module of text encoder described in `Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech`_. Instead of the relative positional Transformer, we use conformer architecture as the encoder module, which contains additional convolution layers. .. _`Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech`: https://arxiv.org/abs/2006.04558 conv1drel_pos rel_selfattnswishTF皙?vocabs attention_dimattention_heads linear_unitsblockspositionwise_layer_typepositionwise_conv_kernel_sizepositional_encoding_layer_typeself_attention_layer_typeactivation_typenormalize_beforeuse_macaron_styleuse_conformer_convconformer_kernel_size dropout_ratepositional_dropout_rateattention_dropout_ratecst||_tj|||_tjj|jj d|dt diddddd|d|d |d |d |d |d |d| d|d|d| d|d| d| d| d||_ tj ||dd|_ dS)aInitialize TextEncoder module. Args: vocabs (int): Vocabulary size. attention_dim (int): Attention dimension. attention_heads (int): Number of attention heads. linear_units (int): Number of linear units of positionwise layers. blocks (int): Number of encoder blocks. positionwise_layer_type (str): Positionwise layer type. positionwise_conv_kernel_size (int): Positionwise layer's kernel size. positional_encoding_layer_type (str): Positional encoding layer type. self_attention_layer_type (str): Self-attention layer type. activation_type (str): Activation function type. normalize_before (bool): Whether to apply LayerNorm before attention. use_macaron_style (bool): Whether to use macaron style components. use_conformer_conv (bool): Whether to use conformer conv layers. conformer_kernel_size (int): Conformer's conv kernel size. dropout_rate (float): Dropout rate. positional_dropout_rate (float): Dropout rate for positional encoding. attention_dropout_rate (float): Dropout rate for attention. rgidim input_layerNrrr num_blocksr r!r"rrr macaron_stylepos_enc_layer_typeselfattention_layer_typeruse_cnn_modulecnn_module_kernelr)super__init__rtorchnn Embeddingembinitnormal_weightrencoderConv1dproj)selfrrrrrrrrrrrrrrr r!r" __class__r-U/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/vits/text_encoder.pyr/!sT *     zTextEncoder.__init__x x_lengthsreturncCs||t|j}t|j|j|jdd}| ||\}}| dd}| ||}|j | dddd\}}||||fS)aCalculate forward propagation. Args: x (Tensor): Input index tensor (B, T_text). x_lengths (Tensor): Length tensor (B,). Returns: Tensor: Encoded hidden representation (B, attention_dim, T_text). Tensor: Projected mean tensor (B, attention_dim, T_text). Tensor: Projected scale tensor (B, attention_dim, T_text). Tensor: Mask tensor for input tensor (B, 1, T_text). )devicedtyper,r)dim)r3mathsqrtrrtorArB unsqueezer7 transposer9splitsize)r:r>r?x_mask_statsmlogsr-r-r=forwardhs  zTextEncoder.forward)rrrr r r r r rTFFrrrr)__name__ __module__ __qualname____doc__intstrboolfloatr/r0TensorrrP __classcell__r-r-r;r=rsx     Gr) rTrDtypingrr0-espnet.nets.pytorch_backend.conformer.encoderr&espnet.nets.pytorch_backend.nets_utilsrr1Modulerr-r-r-r=s