o Si@sddlmZmZddlZddlmZddlmZmZGdddejZ GdddejZ Gd d d ejZ dd ej d e dej fddZd ej dej fddZd ej dej fddZdS))OptionalTupleN)nn) weight_normremove_weight_normc s\eZdZdZ ddedededeeffdd Zdd ej d eej d ej fd d Z Z S) ConvNeXtBlockaConvNeXt Block adapted from https://github.com/facebookresearch/ConvNeXt to 1D audio signal. Args: dim (int): Number of input channels. intermediate_dim (int): Dimensionality of the intermediate layer. layer_scale_init_value (float, optional): Initial value for the layer scale. None means no scaling. Defaults to None. adanorm_num_embeddings (int, optional): Number of embeddings for AdaLayerNorm. None means non-conditional LayerNorm. Defaults to None. Ndimintermediate_dimlayer_scale_init_valueadanorm_num_embeddingscsttj||dd|d|_|du|_|r t||dd|_ntj|dd|_t |||_ t |_ t |||_ |dkrNtj|t|dd|_dSd|_dS) N) kernel_sizepaddinggroupsư>epsrT requires_grad)super__init__rConv1ddwconvadanorm AdaLayerNormnorm LayerNormLinearpwconv1GELUactpwconv2 Parametertorchonesgamma)selfrr r r  __class__A/home/ubuntu/.local/lib/python3.10/site-packages/vocos/modules.pyrs   zConvNeXtBlock.__init__xcond_embedding_idreturncCs|}||}|dd}|jr|dusJ|||}n||}||}||}||}|jdur;|j|}|dd}||}|S)N)r transposerrrr!r"r&)r'r,r-residualr*r*r+forward+s          zConvNeXtBlock.forwardN) __name__ __module__ __qualname____doc__intfloatrrr$Tensorr3 __classcell__r*r*r(r+rs*rcsLeZdZdZd dededeffdd Zdejd ejd ejfd d Z Z S)rz Adaptive Layer Normalization module with learnable embeddings per `num_embeddings` classes Args: num_embeddings (int): Number of embeddings. embedding_dim (int): Dimension of the embeddings. rnum_embeddings embedding_dimrcs^t||_||_tj||d|_tj||d|_tjj |jj tjj |jj dS)N)r=r>) rrrrr Embeddingscaleshiftr$initones_weightzeros_)r'r=r>rr(r*r+rHs zAdaLayerNorm.__init__r,r-r.cCs<||}||}tjj||jf|jd}|||}|S)Nr)r@rAr functional layer_normrr)r'r,r-r@rAr*r*r+r3Qs   zAdaLayerNorm.forward)r) r5r6r7r8r9r:rr$r;r3r<r*r*r(r+r?s$ rc seZdZdZ    ddededeeeefd ed eef fd d Zd e j de j fddZ ddZ e ddededefddZZS) ResBlock1a ResBlock adapted from HiFi-GAN V1 (https://github.com/jik876/hifi-gan) with dilated 1D convolutions, but without upsampling layers. Args: dim (int): Number of input channels. kernel_size (int, optional): Size of the convolutional kernel. Defaults to 3. dilation (tuple[int], optional): Dilation factors for the dilated convolutions. Defaults to (1, 3, 5). lrelu_slope (float, optional): Negative slope of the LeakyReLU activation function. Defaults to 0.1. layer_scale_init_value (float, optional): Initial value for the layer scale. None means no scaling. Defaults to None. r r/r 皙?Nrrdilation lrelu_sloper cs|t||_tttj|||d|d|||ddttj|||d|d|||ddttj|||d|d|||ddg|_tttj|||dd||ddttj|||dd||ddttj|||dd||ddg|_ t |durtj |t |dddnd|durtj |t |dddnd|durtj |t |dddndg|_dS)Nr/r)rLrr0Tr)rrrMr ModuleListrr get_paddingconvs1convs2 ParameterListr#r$r%r&)r'rrrLrMr r(r*r+risj   #    zResBlock1.__init__r,r.cCsrt|j|j|jD]-\}}}tjjj||jd}||}tjjj||jd}||}|dur2||}||}q |S)N)negative_slope) ziprPrQr&r$rrF leaky_relurM)r'r,c1c2r&xtr*r*r+r3s zResBlock1.forwardcCs,|jD]}t|q|jD]}t|q dSr4)rPrrQ)r'lr*r*r+rs    zResBlock1.remove_weight_normr/cCst|||dS)Nr0)r9)rrLr*r*r+rOszResBlock1.get_padding)r rIrKN)r/)r5r6r7r8r9rr:rrr$r;r3r staticmethodrOr<r*r*r(r+rHYs* C "rHHz>r,clip_valr.cCsttj||dS)aU Computes the element-wise logarithm of the input tensor with clipping to avoid near-zero values. Args: x (Tensor): Input tensor. clip_val (float, optional): Minimum value to clip the input tensor. Defaults to 1e-7. Returns: Tensor: Element-wise logarithm of the input tensor with clipping applied. )min)r$logclip)r,r\r*r*r+safe_logs r`cCst|t|Sr4)r$signlog1pabsr,r*r*r+symlogsrecCst|t|dS)Nr/)r$raexprcrdr*r*r+symexpsrg)r[)typingrrr$rtorch.nn.utilsrrModulerrrHr;r:r`rergr*r*r*r+s 7i