o 9wirB@sddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZGdd d ejZd d Zd d ZGdddejZGdddejZGdddejZdS)N)nn) functional) capture_init) center_trimunfold) LayerScalecs*eZdZdZd fdd ZddZZS) BLSTMz BiLSTM with same hidden units as input dim. If `max_steps` is not None, input will be splitting in overlapping chunks and the LSTM applied separately on each chunk. rNFcsXt|dus|ddksJ||_tjd|||d|_td|||_||_dS)NrT) bidirectional num_layers hidden_size input_size) super__init__ max_stepsrLSTMlstmLinearlinearskip)selfdimlayersrr __class__J/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/demucs/demucs.pyrs  zBLSTM.__init__c Cs|j\}}}|}d}|jdur5||jkr5|j}|d}t|||} | jd} d}| ddddd||}|ddd}||d}||}|ddd}|rg} ||d||} |d} t| D]C} | dkr|| | dd| ddd| fqc| | dkr| | dd| dd| dfqc| | dd| dd| | fqct | d} | dd|f} | }|j r||}|S) NFrTrr.) shaperrpermutereshaperrrangeappendtorchcatr)rxBCTyframedwidthstrideframesnframesoutlimitkrrrforward"s<     & $& z BLSTM.forward)rNF)__name__ __module__ __qualname____doc__rr5 __classcell__rrrrr sr cCsL|j}||d}|jj|_|jdur$|jj|_dSdS)zTRescale initial weight scale. It is unclear why it helps but it certainly does. ?N)weightstddetachdatabias)conv referencer=scalerrr rescale_convFs   rDcCs6|D]}t|tjtjtjtjfrt||qdSN)modules isinstancerConv1dConvTranspose1dConv2dConvTranspose2drD)modulerBsubrrrrescale_modulePs  rNc sBeZdZdZ   dded ed ed effd d ZddZZS)DConva  New residual branches in each encoder layer. This alternates dilated convolutions, potentially with LSTMs and attention. Also before entering each residual branch, dimension is projected on a smaller subspace, e.g. of dim `channels // compress`. r r-C6?TFrchannelscompressdepthinitc  s<t| ddks J||_||_t||_|dk} dd} |r&dd} t||}| r2tj}ntj }t g|_ t |jD][}| rHd|nd}|| d}tj ||| ||d| ||t |d|d| d|tdt||g}|r|dt|||d | r|dt|dd d d tj|}|j |q@d S)a Args: channels: input/output channels for residual branch. compress: amount of channel compression inside the branch. depth: number of layers in the residual branch. Each layer has its own projection, and potentially LSTM and attention. init: initial scale for LayerNorm. norm: use GroupNorm. attn: use LocalAttention. heads: number of heads for the LocalAttention. ndecay: number of decay controls in the LocalAttention. lstm: use LSTM. gelu: Use GELU activation. kernel: kernel size for the (dilated) convolutions. dilate: if true, use dilation, increasing with the depth. rrrcStSrErIdentitydrrryz DConv.__init__..cSs td|S)Nrr GroupNormrXrrrrZ{ )dilationpaddingr)headsndecayT)rrrN)rrrQrRabsrSintrGELUReLU ModuleListrr$rHGLUrinsert LocalStater Sequentialr%)rrQrRrSrTnormattnrarbrgelukerneldilatenorm_fnhiddenactrYr_r`modslayerrrrr]s<       zDConv.__init__cCs|jD]}|||}q|SrE)r)rr(rvrrrr5s z DConv.forward) r rrPTFr r FTrT) r6r7r8r9refloatrr5r:rrrrrOVs :rOc s<eZdZdZd dedededeffdd Zd d ZZS) rkaLocal state allows to have attention based only on data (no positional embedding), but while setting a constraint on the time window (e.g. decaying penalty term). Also a failed experiments with trying to provide some frequency based attention. r rrQranfreqsrbcst||dksJ||f||_||_||_t||d|_t||d|_t||d|_ |r>t|||d|_ |rdt|||d|_ |j j j d9_ |j jdus[Jd|j jj dd<t||||d|_dS)Nrrg{Gz?)rrrarxrbrrHcontentquerykey query_freqs query_decayr<r?r@proj)rrQrarxrbrrrrs zLocalState.__init__cCs|j\}}}|j}tj||j|jd}|dddf|dddf}||||d|}||||d|} t d| |} | | jdd} |j rtjd|j d|j|jd} t dt j || ddd} ||||d||j d} | t d| | 7} |jrtjd|jd|j|jd}||||d|}t|d}|ddd ||jd}| t d||7} | tj|| jtjddtj| dd }||||d|}t d ||}|j rt d || }t||gd}||d|}|||S) N)devicedtyper zbhct,bhcs->bhtsrr;rzfts,bhfs->bhtsirzbhts,bhct->bhcszbhts,fts->bhfs)r!rar&arangerrr{viewr|einsumrxcosmathpir}rbr~sigmoidrd masked_fill_eyeboolsoftmaxrzr'r#r)rr(r)r*r+raindexesdeltaquerieskeysdotsperiods freq_kernelfreq_qdecaysdecay_q decay_kernelweightsrzresulttime_sigrrrr5s8  " "zLocalState.forward)r rr )r6r7r8r9rerr5r:rrrrrks rkcspeZdZe                   dfdd ZddZddZdfdd ZZS)Demucsr@@Trr rrP皙?D(c& sht||_||_||_| |_| |_||_||_||_ ||_ ||_ ||_ t |_t |_t |_| r@t jdd}d}nt }d}| rLt j}nt j}|}d}t|D]}dd} || krgfdd} g}!|!t |||| | ||g7}!||k}"||k}#|d@r|!t|||||"|#dg7}!|r|!t |||d| |||g7}!|jt j|!g}$|dkr|}%nt|j|}%|r|$t j|||d| d| d | |||g7}$|d@r|$t|||||"|#dg7}$|$t j||%|| |d g7}$|dkr|$| |%|g7}$|jdt j|$|}t||}qW|}|r$t|||_nd |_|r2t ||d d Sd S) ab Args: sources (list[str]): list of source names audio_channels (int): stereo or mono channels (int): first convolution channels depth (int): number of encoder/decoder layers growth (float): multiply (resp divide) number of channels by that for each layer of the encoder (resp decoder) depth (int): number of layers in the encoder and in the decoder. rewrite (bool): add 1x1 convolution to each layer. lstm_layers (int): number of lstm layers, 0 = no lstm. Deactivated by default, as this is now replaced by the smaller and faster small LSTMs in the DConv branches. kernel_size (int): kernel size for convolutions stride (int): stride for convolutions context (int): kernel size of the convolution in the decoder before the transposed convolution. If > 1, will provide some context from neighboring time steps. gelu: use GELU activation function. glu (bool): use glu instead of ReLU for the 1x1 rewrite conv. norm_starts: layer at which group norm starts being used. decoder layers are numbered in reverse order. norm_groups: number of groups for group norm. dconv_mode: if 1: dconv in encoder only, 2: decoder only, 3: both. dconv_depth: depth of residual DConv branch. dconv_comp: compression of DConv branch. dconv_attn: adds attention layers in DConv branch starting at this layer. dconv_lstm: adds a LSTM layer in DConv branch starting at this layer. dconv_init: initial scale for the DConv branch LayerScale. normalize (bool): normalizes the input audio on the fly, and scales back the output by the same amount. resample (bool): upsample x2 the input and downsample /2 the output. rescale (float): rescale initial weights of convolutions to get their standard deviation closer to `rescale`. samplerate (int): stored as meta information for easing future evaluations of the model. segment (float): duration of the chunks of audio to ideally evaluate the model on. This is used by `demucs.apply.apply_model`. rrrrcSrUrErVrXrrrrZFr[z!Demucs.__init__..cs t|SrEr\rX norm_groupsrrrZHr^)rSrTrRrnr)r`N)rB)!rraudio_channelssources kernel_sizecontextr/rSresamplerQ normalize sampleratesegmentrrhencoderdecoder skip_scalesrirgrfr$rHrOr%rllenrIrjrer rrN)&rrrrQgrowthrSrewrite lstm_layersrr/rroglu norm_startsr dconv_mode dconv_depth dconv_comp dconv_attn dconv_lstm dconv_initrrrescalerr activationch_scaleact2 in_channelsr`indexrrencodernrdecode out_channelsrrrrs L             zDemucs.__init__cCs|jr|d9}t|jD]}t||j|jd}td|}q t|jD] }|d|j|j}q&|jr=t|d}t|S)aX Return the nearest valid length to use with the model so that there is no time steps left over in a convolution, e.g. for all layers, size of the input - kernel_size % stride = 0. Note that input are automatically padded if necessary to ensure that the output has the same length as the input. rr) rr$rSrceilrr/maxre)rlength_idxrrr valid_lengthxs  zDemucs.valid_lengthc CsF|}|jd}|jr(|jddd}|jddd}|jddd}||d|}nd}d}|||}t||d||df}|jrKt |dd}g}|j D] } | |}| |qP|j rd| |}|j D]} |d} t| |} | || }qg|jrt |dd}|||}t||}||dt|j|j|d}|S)Nr rT)rkeepdimgh㈵>rr)r!rmeanr=rFpadrjulius resample_fracrr%rrpoprrsizerrr) rmixr(rmonorr=rsavedrrrrrrr5s:         $zDemucs.forwardcs~t|jD]/}dD]*}dD]%}|d|d|}|d|d|}||vr2||vr2||||<q q qtj||ddS)N)rr)r@r<.z.3.z.2.)strict)r$rSrrload_state_dict)rstaterrabnewoldrrrrszDemucs.load_state_dict)rrrrTrrr rTTr r rrr r r rPTTrrr)T) r6r7r8rrrr5rr:rrrrrs>'r)rtypingtprr&rtorch.nnrrstatesrutilsrr transformerrModuler rDrNrOrkrrrrrs    2 G>