o ¡¿¯il#ã@szdZddlZddlmmZddlmZmZddl m Z Gdd„dejj ƒZ Gdd„dejj ƒZ Gd d „d ejj ƒZdS) zCBHG related modules.éN)Úpack_padded_sequenceÚpad_packed_sequence)Úmake_non_pad_maskcs*eZdZdZd‡fdd„ Zdd„Z‡ZS)ÚCBHGLosszLoss function module for CBHG.Tcstt|ƒ ¡||_dS)z†Initialize CBHG loss module. Args: use_masking (bool): Whether to mask padded part in loss calculation. N)ÚsuperrÚ__init__Ú use_masking)Úselfr©Ú __class__©ú^/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/pytorch_backend/tacotron2/cbhg.pyrs zCBHGLoss.__init__cCsP|jrt|ƒ d¡ |j¡}| |¡}| |¡}t ||¡}t ||¡}||fS)a†Calculate forward propagation. Args: cbhg_outs (Tensor): Batch of CBHG outputs (B, Lmax, spc_dim). spcs (Tensor): Batch of groundtruth of spectrogram (B, Lmax, spc_dim). olens (LongTensor): Batch of the lengths of each sequence (B,). Returns: Tensor: L1 loss value Tensor: Mean square error loss value. éÿÿÿÿ) rrÚ unsqueezeÚtoÚdeviceÚ masked_selectÚFÚl1_lossÚmse_loss)r Ú cbhg_outsÚspcsÚolensÚmaskÚ cbhg_l1_lossÚ cbhg_mse_lossr r r Úforwards    zCBHGLoss.forward)T©Ú__name__Ú __module__Ú __qualname__Ú__doc__rrÚ __classcell__r r r r rs rcsPeZdZdZ       d‡fdd„ Zd d „Zd d „Zd d„Zdd„Z‡Z S)ÚCBHGa`CBHG module to convert log Mel-filterbanks to linear spectrogram. This is a module of CBHG introduced in `Tacotron: Towards End-to-End Speech Synthesis`_. The CBHG converts the sequence of log Mel-filterbanks into linear spectrogram. .. _`Tacotron: Towards End-to-End Speech Synthesis`: https://arxiv.org/abs/1703.10135 éé€éééc stt|ƒ ¡||_||_||_||_||_||_||_ ||_ | |_ t j  ¡|_td|jdƒD]D} | ddkr?| dd} n| dd| dddf} |jt j  t j  | d¡t j j||j| ddddt j  |j¡t j  ¡¡g7_q0t j  t j  dd¡t j jddd¡|_t j  t j j|j|j|j|jd|jddddt j  |j¡t j  ¡t j j|j|j|jd|jddddt j  |j¡¡|_t j  ¡|_|jt j  ||j ¡g7_t|j ƒD] } |jt|j ƒg7_qàt j j|j | ddddd |_t j j| |dd |_d S) aInitialize CBHG module. Args: idim (int): Dimension of the inputs. odim (int): Dimension of the outputs. conv_bank_layers (int, optional): The number of convolution bank layers. conv_bank_chans (int, optional): The number of channels in convolution bank. conv_proj_filts (int, optional): Kernel size of convolutional projection layer. conv_proj_chans (int, optional): The number of channels in convolutional projection layer. highway_layers (int, optional): The number of highway network layers. highway_units (int, optional): The number of highway network units. gru_units (int, optional): The number of GRU units (for both directions). éérgT)ÚstrideÚpaddingÚbias)rr))r+)Ú num_layersÚ batch_firstÚ bidirectional)r-N)rr#rÚidimÚodimÚconv_bank_layersÚconv_bank_chansÚconv_proj_filtsÚconv_proj_chansÚhighway_layersÚ highway_unitsÚ gru_unitsÚtorchÚnnÚ ModuleListÚ conv_bankÚrangeÚ SequentialÚ ConstantPad1dÚConv1dÚ BatchNorm1dÚReLUÚ MaxPool1dÚmax_poolÚ projectionsÚhighwaysÚLinearÚ HighwayNetÚGRUÚgruÚoutput) r r1r2r3r4r5r6r7r8r9Úkr,Ú_r r r rCsz   ÿ ú ÿ ÿ  ú  ú í û z CBHG.__init__c Cs"| dd¡}g}t|jƒD] }||j||ƒg7}q tj|dd}| |¡}| |¡ dd¡}| dd¡|}t|jdƒD] }|j ||ƒ}q>|  ||¡\}}}|  d¡}t |tj ƒsat |¡}t|| ¡dd}|j ¡| |¡\}}t|d|d\}}| |||¡\}}| |¡}||fS)a|Calculate forward propagation. Args: xs (Tensor): Batch of the padded sequences of inputs (B, Tmax, idim). ilens (LongTensor): Batch of lengths of each input sequence (B,). Return: Tensor: Batch of the padded sequence of outputs (B, Tmax, odim). LongTensor: Batch of lengths of each output sequence (B,). r)r*)ÚdimT)r/)r/Ú total_length)Ú transposer>r3r=r:ÚcatrErFr7rGÚ_sort_by_lengthÚsizeÚ isinstanceÚTensorÚtensorrÚcpurKÚflatten_parametersrÚ_revert_sort_by_lengthrL) r ÚxsÚilensÚconvsrMÚiÚsort_idxrPrNr r r rªs*       z CBHG.forwardcCsHt| ¡ƒdks J‚| d¡}| | d¡g¡ ¡}| ||¡ddS)z¦Inference. Args: x (Tensor): The sequences of inputs (T, idim). Return: Tensor: The sequence of outputs (T, odim). r*r)ÚlenrTrÚnewÚlongr)r Úxr[r\r r r Ú inferenceÖs zCBHG.inferencecCs$|jddd\}}|||||fS)NrT)Ú descending©Úsort)r r[r\Ú sort_ilensr_r r r rSæszCBHG._sort_by_lengthcCs| d¡\}}||||fS)Nrrf)r r[r\r_rNÚ revert_idxr r r rZêszCBHG._revert_sort_by_length)r$r%r&r'r(r%r') rrr r!rrrdrSrZr"r r r r r#7sög,r#cs(eZdZdZ‡fdd„Zdd„Z‡ZS)rIz§Highway Network module. This is a module of Highway Network introduced in `Highway Networks`_. .. _`Highway Networks`: https://arxiv.org/abs/1505.00387 csXtt|ƒ ¡||_tj tj ||¡tj ¡¡|_ tj tj ||¡tj  ¡¡|_ dS)zlInitialize Highway Network module. Args: idim (int): Dimension of the inputs. N) rrIrr1r:r;r?rHrCÚ projectionÚSigmoidÚgate)r r1r r r røs ÿ$zHighwayNet.__init__cCs(| |¡}| |¡}|||d|S)zÚCalculate forward propagation. Args: x (Tensor): Batch of inputs (B, ..., idim). Returns: Tensor: Batch of outputs, which are the same shape as inputs (B, ..., idim). gð?)rjrl)r rcÚprojrlr r r rs zHighwayNet.forwardrr r r r rIïs rI)r!r:Útorch.nn.functionalr;Ú functionalrÚtorch.nn.utils.rnnrrÚ&espnet.nets.pytorch_backend.nets_utilsrÚModulerr#rIr r r r Ús '9