o i6A@s|dZddlZddlmZmZmZddlZddlZddl m Z Gdddej j Z Gdddej j ZGd d d ej j ZdS) z]MelGAN Modules. This code is modified from https://github.com/kan-bayashi/ParallelWaveGAN. N)AnyDictList) ResidualStackcseZdZdZdddddgdddd d d id iddfd edededededeededededeee fdedeee fdedeffdd Z de j de j fdd Z d!d"Zd#d$Zd%d&Zde j de j fd'd(ZZS))MelGANGeneratorzMelGAN generator module.PiT)r r  LeakyReLUnegative_slope皙?ReflectionPad1d in_channels out_channels kernel_sizechannelsbiasupsample_scalesstack_kernel_sizestacksnonlinear_activationnonlinear_activation_paramspad pad_paramsuse_final_nonlinear_activationuse_weight_normc st|t|ksJ|dt|dksJ|dddks&Jdg}|ttj| |ddfi| tjj||||dg7}t t|||_ t |D]T\}}|ttj| d i| g7}|tjj |d||d|d|d||d|d|d|dg7}t |D]}|t||d|d|||| | | | dg7}qqR|ttj| d i| g7}|ttj| |ddfi| tjj|d|d|||dg7}| r|tjg7}tjj||_|r||dS) aInitialize MelGANGenerator module. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. kernel_size (int): Kernel size of initial and final conv layer. channels (int): Initial number of channels for conv layer. bias (bool): Whether to add bias parameter in convolution layers. upsample_scales (List[int]): List of upsampling scales. stack_kernel_size (int): Kernel size of dilated conv layers in residual stack. stacks (int): Number of stacks in a single residual stack. nonlinear_activation (str): Activation function module name. nonlinear_activation_params (Dict[str, Any]): Hyperparameters for activation function. pad (str): Padding function module name before dilated convolution layer. pad_params (Dict[str, Any]): Hyperparameters for padding function. use_final_nonlinear_activation (torch.nn.Module): Activation function for the final layer. use_weight_norm (bool): Whether to use weight norm. If set to true, it will be applied to all of the conv layers. r rrz$Not support even number kernel size.r)stridepaddingoutput_paddingr)rrdilationrrrrrN)super__init__npprodlengetattrtorchnnConv1dintupsample_factor enumerateConvTranspose1drangerTanh Sequentialmelganapply_weight_normreset_parameters)selfrrrrrrrrrrrrrrlayersiupsample_scalej __class__r$Q/home/ubuntu/.local/lib/python3.10/site-packages/espnet2/gan_tts/melgan/melgan.pyr&sl (   zMelGANGenerator.__init__creturncCs ||S)zCalculate forward propagation. Args: c (Tensor): Input tensor (B, channels, T). Returns: Tensor: Output tensor (B, 1, T ** prod(upsample_scales)). )r5r8r@r$r$r?forwards zMelGANGenerator.forwardcC dtjjfdd}||dS):Remove weight normalization module from all of the layers.mcS<ztd|dtjj|WdStyYdSwNzWeight norm is removed from .loggingdebugr+r,utilsremove_weight_norm ValueErrorrFr$r$r?_remove_weight_norm  z?MelGANGenerator.remove_weight_norm.._remove_weight_normNr+r,Moduleapplyr8rQr$r$r?rNz"MelGANGenerator.remove_weight_normcCrD)9Apply weight normalization module from all of the layers.rFcSDt|tjjst|tjjr tjj|td|ddSdSNzWeight norm is applied to rI isinstancer+r,r-r1rM weight_normrKrLrPr$r$r?_apply_weight_norm z=MelGANGenerator.apply_weight_norm.._apply_weight_normNrSr8r^r$r$r?r6rWz!MelGANGenerator.apply_weight_normcCsdd}||dS)Reset parameters. This initialization follows official implementation manner. https://github.com/descriptinc/melgan-neurips/blob/master/mel2wav/modules.py cSFt|tjjst|tjjr!|jjddt d|ddSdSNgg{Gz?zReset parameters in rI r\r+r,r-r1weightdatanormal_rKrLrPr$r$r?_reset_parameters z;MelGANGenerator.reset_parameters.._reset_parametersN)rUr8rhr$r$r?r7sz MelGANGenerator.reset_parameterscCs*||ddd}|dddS)zPerform inference. Args: c (Tensor): Input tensor (T, in_channels). Returns: Tensor: Output tensor (T ** prod(upsample_scales), out_channels). rr)r5 transpose unsqueezesqueezerBr$r$r? inferences zMelGANGenerator.inference)__name__ __module__ __qualname____doc__r.boolrstrrrr&r+TensorrCrNr6r7rn __classcell__r$r$r=r?rsd     n   rcseZdZdZddddgdddgdd d d id if d ededeededededeededeee fdedeee fffdd Z de j dee j fddZ ZS)MelGANDiscriminatorzMelGAN discriminator module.rr Tr|r|r|r rrrrr kernel_sizesrmax_downsample_channelsrdownsample_scalesrrrrc sttj|_t|dksJ|dddksJ|dddks'J|jtjttj| t |ddfi| tjj ||t ||dttj|d i| g7_|} |D]4} t | | |}|jtjtjj | || dd| | d| d|dttj|d i| g7_|} q^t | d|}|jtjtjj | ||d|ddd|d ttj|d i| g7_|jtjj |||d|ddd|d g7_d S) aInitilize MelGANDiscriminator module. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. kernel_sizes (List[int]): List of two kernel sizes. The prod will be used for the first conv layer, and the first and the second kernel sizes will be used for the last two layers. For example if kernel_sizes = [5, 3], the first layer kernel size will be 5 * 3 = 15, the last two layers' kernel size will be 5 and 3, respectively. channels (int): Initial number of channels for conv layer. max_downsample_channels (int): Maximum number of channels for downsampling layers. bias (bool): Whether to add bias parameter in convolution layers. downsample_scales (List[int]): List of downsampling scales. nonlinear_activation (str): Activation function module name. nonlinear_activation_params (Dict[str, Any]): Hyperparameters for activation function. pad (str): Padding function module name before dilated convolution layer. pad_params (Dict[str, Any]): Hyperparameters for padding function. r rrr rxr|)rr r!groupsr)r!rNr$) r%r&r+r, ModuleListr9r)r4r*r'r(r-min)r8rrr}rr~rrrrrrin_chsdownsample_scaleout_chsr=r$r?r&sp $ $     zMelGANDiscriminator.__init__xrAcCs&g}|jD] }||}||g7}q|S)zCalculate forward propagation. Args: x (Tensor): Input noise signal (B, 1, T). Returns: List[Tensor]: List of output tensors of each layer. )r9r8routsfr$r$r?rC/s   zMelGANDiscriminator.forward)rorprqrrr.rrsrtrrr&r+rurCrvr$r$r=r?rwsJ    "erwcseZdZdZdddddddddd dgd d d gd dddidid fdededededeeefdeededede deededeeefdedeeefd e ffd!d" Z d#e j d$eee j fd%d&Z d'd(Zd)d*Zd+d,ZZS)-MelGANMultiScaleDiscriminatorz(MelGAN multi-scale discriminator module.rr AvgPool1dr|r F)rr r!count_include_padrxryrzTr{r rrrrrscalesdownsample_poolingdownsample_pooling_paramsr}rr~rrrrrrrcs~ttj|_t|D]}|jt|||||| | | | | |d g7_qttj|di||_ |r9| | dS)aInitilize MelGANMultiScaleDiscriminator module. Args: in_channels (int): Number of input channels. out_channels (int): Number of output channels. scales (int): Number of multi-scales. downsample_pooling (str): Pooling module name for downsampling of the inputs. downsample_pooling_params (Dict[str, Any]): Parameters for the above pooling module. kernel_sizes (List[int]): List of two kernel sizes. The sum will be used for the first conv layer, and the first and the second kernel sizes will be used for the last two layers. channels (int): Initial number of channels for conv layer. max_downsample_channels (int): Maximum number of channels for downsampling layers. bias (bool): Whether to add bias parameter in convolution layers. downsample_scales (List[int]): List of downsampling scales. nonlinear_activation (str): Activation function module name. nonlinear_activation_params (Dict[str, Any]): Hyperparameters for activation function. pad (str): Padding function module name before dilated convolution layer. pad_params (Dict[str, Any]): Hyperparameters for padding function. use_weight_norm (bool): Whether to use weight norm. ) rrr}rr~rrrrrrNr$) r%r&r+r,rdiscriminatorsr2rwr*poolingr6r7)r8rrrrrr}rr~rrrrrrr_r=r$r?r&Ds0 2    z&MelGANMultiScaleDiscriminator.__init__rrAcCs,g}|jD]}|||g7}||}q|S)aCalculate forward propagation. Args: x (Tensor): Input noise signal (B, 1, T). Returns: List[List[Tensor]]: List of list of each discriminator outputs, which consists of each layer output tensors. )rrrr$r$r?rCs   z%MelGANMultiScaleDiscriminator.forwardcCrD)rErFcSrGrHrJrPr$r$r?rQrRzMMelGANMultiScaleDiscriminator.remove_weight_norm.._remove_weight_normNrSrVr$r$r?rNrWz0MelGANMultiScaleDiscriminator.remove_weight_normcCrD)rXrFcSrYrZr[rPr$r$r?r^r_zKMelGANMultiScaleDiscriminator.apply_weight_norm.._apply_weight_normNrSr`r$r$r?r6rWz/MelGANMultiScaleDiscriminator.apply_weight_normcCrD)rarFcSrbrcrdrPr$r$r?rhrizIMelGANMultiScaleDiscriminator.reset_parameters.._reset_parametersNrSrjr$r$r?r7sz.MelGANMultiScaleDiscriminator.reset_parameters)rorprqrrr.rtrrrrsr&r+rurCrNr6r7rvr$r$r=r?rAsp    Q  r)rrrKtypingrrrnumpyr'r+%espnet2.gan_tts.melgan.residual_stackrr,rTrrwrr$r$r$r?s 5z