o ¡¿¯ir6ã@s²dZddlZddlZddlZddlZddlZddlmm Z ddlmZddd„Z ddd„Z d d „Z Gd d „d ejƒZGd d„dejƒZGdd„dejƒZGdd„dejƒZdS)zKThis code is based on https://github.com/kan-bayashi/PytorchWaveNetVocoder.éN)ÚnnécCsZ|d}t |¡t d|t |¡¡t d|¡}t |dd|d¡ tj¡S)zâPerform mu-law encoding. Args: x (ndarray): Audio signal with the range from -1 to 1. mu (int): Quantized level. Returns: ndarray: Quantized audio signal with the range from 0 to mu - 1. ééçà?)ÚnpÚsignÚlogÚabsÚfloorÚastypeÚint64)ÚxÚmuÚfx©rúW/home/ubuntu/.local/lib/python3.10/site-packages/espnet/nets/pytorch_backend/wavenet.pyÚ encode_mu_laws 0"rcCsD|d}|d|dd}t |¡|d|t |¡d}|S)zâPerform mu-law decoding. Args: x (ndarray): Quantized audio signal with the range from 0 to mu - 1. mu (int): Quantized level. Returns: ndarray: Audio signal with the range from -1 to 1. rrr)rrr )ÚyrrrrrrÚ decode_mu_law"s $rcCs^t|tjƒrtj |j¡tj |jd¡t|tjƒr-tj |jd¡tj |jd¡dSdS)z^Initilize conv layers with xavier. Args: m (torch.nn.Module): Torch module. ggð?N) Ú isinstancerÚConv1dÚinitÚxavier_uniform_ÚweightÚ constant_ÚbiasÚConvTranspose2d)ÚmrrrÚ initialize3s  þrcs(eZdZdZ‡fdd„Zdd„Z‡ZS)ÚOneHotz]Convert to one-hot vector. Args: depth (int): Dimension of one-hot vector. cstt|ƒ ¡||_dS©N)Úsuperr Ú__init__Údepth)Úselfr$©Ú __class__rrr#Ks zOneHot.__init__cCsD||j}t |d¡}| | d¡| d¡|j¡ ¡}| d|d¡S)zØCalculate forward propagation. Args: x (LongTensor): long tensor variable with the shape (B, T) Returns: Tensor: float tensor variable with the shape (B, depth, T) rrr)r$ÚtorchÚ unsqueezeÚ new_zerosÚsizeÚfloatÚscatter_)r%rÚx_onehotrrrÚforwardOs  zOneHot.forward©Ú__name__Ú __module__Ú __qualname__Ú__doc__r#r/Ú __classcell__rrr&rr Cs r cs*eZdZdZd‡fdd„ Zdd„Z‡ZS) Ú CausalConv1dz1D dilated causal convolution.rTcsTtt|ƒ ¡||_||_||_||_|d||_}tj ||||||d|_ dS)Nr)ÚpaddingÚdilationr) r"r6r#Ú in_channelsÚ out_channelsÚ kernel_sizer8r7rrÚconv)r%r9r:r;r8rr7r&rrr#cs úzCausalConv1d.__init__cCs6| |¡}|jdkr|dd…dd…d|j …f}|S)zÑCalculate forward propagation. Args: x (Tensor): Input tensor with the shape (B, in_channels, T). Returns: Tensor: Tensor with the shape (B, out_channels, T) rN)r<r7©r%rrrrr/ss zCausalConv1d.forward)rTr0rrr&rr6`sr6cs*eZdZdZd‡fdd„ Zdd„Z‡ZS)Ú UpSamplingziUpsampling layer with deconvolution. Args: upsampling_factor (int): Upsampling factor. TcsBtt|ƒ ¡||_||_tjddd|jfd|jf|jd|_dS)Nr)r;Ústrider)r"r>r#Úupsampling_factorrrrr<)r%r@rr&rrr#‹s ûzUpSampling.__init__cCs| d¡}| |¡}| d¡S)zßCalculate forward propagation. Args: x (Tensor): Input tensor with the shape (B, C, T) Returns: Tensor: Tensor with the shape (B, C, T') where T' = T * upsampling_factor. r)r)r<Úsqueezer=rrrr/—s  zUpSampling.forward)Tr0rrr&rr>ƒs r>csdeZdZdZ        d‡fd d „ Zd d „Zddd„Zdd„Zdd„Zdd„Z dd„Z ‡Z S)ÚWaveNeta;Conditional wavenet. Args: n_quantize (int): Number of quantization. n_aux (int): Number of aux feature dimension. n_resch (int): Number of filter channels for residual block. n_skipch (int): Number of filter channels for skip connection. dilation_depth (int): Number of dilation depth (e.g. if set 10, max dilation = 2^(10-1)). dilation_repeat (int): Number of dilation repeat. kernel_size (int): Filter size of dilated causal convolution. upsampling_factor (int): Upsampling factor. rééé érrc sÖtt|ƒ ¡||_||_||_||_||_||_||_ ||_ dd„t |jƒDƒ|j |_ |jdt |j ƒd|_t|jƒ|_t|j|j|jƒ|_|j dkrUt|j ƒ|_t ¡|_t ¡|_t ¡|_t ¡|_t ¡|_t ¡|_|j D]^} |jt|j|j|j| ƒg7_|jt|j|j|j| ƒg7_|jt |j|jd¡g7_|jt |j|jd¡g7_|jt |j|jd¡g7_|jt |j|jd¡g7_qvt |j|jd¡|_t |j|jd¡|_dS)NcSsg|]}d|‘qS)rr)Ú.0ÚirrrÚ Ësÿz$WaveNet.__init__..rr) r"rBr#Ún_auxÚ n_quantizeÚn_reschÚn_skipchr;Údilation_depthÚdilation_repeatr@ÚrangeÚ dilationsÚsumÚreceptive_fieldr Úonehotr6Úcausalr>Ú upsamplingrÚ ModuleListÚ dil_sigmoidÚdil_tanhÚaux_1x1_sigmoidÚ aux_1x1_tanhÚskip_1x1Úres_1x1rÚ conv_post_1Ú conv_post_2) r%rKrJrLrMrNrOr;r@Údr&rrr#¶sL ÿþ          ÿÿ zWaveNet.__init__c Cs–| |¡}|jdkr| |¡}g}tt|jƒƒD]'}| |||j||j||j ||j ||j ||j |¡\}}|  |¡qt|ƒ}| |¡}|S)a+Calculate forward propagation. Args: x (LongTensor): Quantized input waveform tensor with the shape (B, T). h (Tensor): Auxiliary feature tensor with the shape (B, n_aux, T). Returns: Tensor: Logits with the shape (B, T, n_quantize). r)Ú _preprocessr@rVrPÚlenrQÚ_residual_forwardrXrYrZr[r\r]ÚappendrRÚ _postprocess)r%rÚhÚoutputÚskip_connectionsrHÚskiprrrr/ís&  ø  zWaveNet.forwardNÚsamplingc CsÖt|jƒdks J‚t|jƒdkr|jd|jksJ‚| d¡}| dd¡ d¡}|jdkr2| |¡}||jdkrGt |d||jdfd¡}|j |  d¡}|dkrit ||dfd|j d¡}t ||dfd¡}|  |¡}|dd…dd…d|  d¡…f}g} g} t |jƒD]W\} } | |||j| |j| |j| |j| |j| |j| ¡\}} | d|jdkr½|  |jd¡n |  | d|jd¡|  |dd…dd…| |  dd…f¡q‡|d}t ¡}t|ƒD]ô} ||j ddd… d¡}|  |¡}|dd…dd…|  d¡df ¡ d|jd¡}g}g}t |jƒD]H\}} | |||j||j||j||j||j||j|¡\}}tj| ||gdd}| |dd…dd…| | d…f¡| |¡q"|} t |ƒ}| !|¡d}|d kr”tj"|ddd}tj# $|¡}| %¡ d¡}n|d krŸ| &d¡}n t' (d ¡t) *d¡tj||gdd}|durß| d|dkrßt ¡||}t' +d | d||| d||f¡t ¡}që|| d… ,¡ -¡S) a‡Generate a waveform with fast genration algorithm. This generation based on `Fast WaveNet Generation Algorithm`_. Args: x (LongTensor): Initial waveform tensor with the shape (T,). h (Tensor): Auxiliary feature tensor with the shape (n_samples + T, n_aux). n_samples (int): Number of samples to be generated. interval (int, optional): Log interval. mode (str, optional): "sampling" or "argmax". Return: ndarray: Generated quantized waveform (n_samples). .. _`Fast WaveNet Generation Algorithm`: https://arxiv.org/abs/1611.09482 rrrÚ replicateÚconstantNéÿÿÿÿ)ÚdimrjÚargmaxz!mode should be sampling or argmaxz3%d/%d estimated time = %.3f sec (%.3f sec / sample)).rbÚshaperJr)Ú transposer@rVÚFÚpadrSr+rKraÚ enumeraterQrcrXrYrZr[r\r]rNrdr;ÚtimerPÚ contiguousÚviewÚ_generate_residual_forwardr(ÚcatrRreÚsoftmaxÚ distributionsÚ CategoricalÚsampleroÚloggingÚerrorÚsysÚexitÚinfoÚcpuÚnumpy)r%rrfÚ n_samplesÚintervalÚmodeÚn_padrgÚh_Ú output_bufferÚ buffer_sizerHr`Ú_ÚsamplesÚ start_timeÚoutput_buffer_nextrhÚjriÚ posteriorÚdistr}Úelapsed_time_per_samplerrrÚgeneratesž"     ø ,  0ø &      üÿÿ €zWaveNet.generatecCs | |¡ dd¡}| |¡}|S©Nrr)rTrqrU©r%rrgrrrra‡s zWaveNet._preprocesscCs4t |¡}| |¡}t |¡}| |¡ dd¡}|Sr•)rrÚrelur^r_rqr–rrrreŒs   zWaveNet._postprocessc Cs\||ƒ} ||ƒ} ||ƒ} ||ƒ} t | | ¡t | | ¡} || ƒ}|| ƒ} | |} | |fSr!©r(ÚsigmoidÚtanh©r%rrfrXrYrZr[r\r]Úoutput_sigmoidÚ output_tanhÚaux_output_sigmoidÚaux_output_tanhrgrirrrrc“s ÿzWaveNet._residual_forwardc Csž||ƒdd…dd…dd…f} ||ƒdd…dd…dd…f} ||ƒ} ||ƒ} t | | ¡t | | ¡} || ƒ}|| ƒ} | |dd…dd…dd…f} | |fS)Nrmr˜r›rrrrxªs ÿz"WaveNet._generate_residual_forward)rrCrDrrErFrr)Nrj) r1r2r3r4r#r/r”rarercrxr5rrr&rrB¦s"÷7 %urB)r)r4r~r€rur„rr(Útorch.nn.functionalrÚ functionalrrrrrÚModuler r6r>rBrrrrÚs   ##