o si@s&ddlmZddlZddlmZddlmZddlmZddlm Z ddl m Z d Z d$d ee d efd d Ze d$d efddZe d$d efddZGdddejZGdddeZGdddeZGdddeZGdddeZGdddeZeZeZeZeZeZddZd d!Zd"d#Z dS)%)partialN)nn) _BatchNorm)List) complex_nn)script_if_tracing:0yE>dimsepscCs:|j|dd}tj||ddd}||t||}|S)NTdimkeepdimFr runbiased)meantorchvarsqrt)xr r rvar2valuerI/home/ubuntu/.local/lib/python3.10/site-packages/asteroid/masknn/norms.pyz_norm srcC"tdt|j}t|||S)Nrarangelenshapetolistrrr r rrr _glob_norm r#cCr)Nrrr"rrr_feat_glob_normr$r%cs(eZdZdZfddZddZZS) _LayerNormzLayer Normalization base class.csDtt|||_tjt|dd|_tjt |dd|_ dS)NT) requires_grad) superr&__init__ channel_sizer Parameterronesgammazerosbeta)selfr* __class__rrr)#sz_LayerNorm.__init__cCs |j|dd|jddS)z+Assumes input of size `[batch, chanel, *]`.r)r- transposer/)r0normed_xrrrapply_gain_and_bias)s z_LayerNorm.apply_gain_and_bias)__name__ __module__ __qualname____doc__r)r6 __classcell__rrr1rr& s r&c@ eZdZdZddefddZdS)GlobLNz$Global Layer Normalization (globLN).r EPScCt||d}||S)zApplies forward pass. Works for any input size > 2D. Args: x (:class:`torch.Tensor`): Shape `[batch, chan, *]` Returns: :class:`torch.Tensor`: gLN_x `[batch, chan, *]` r )r#r6r0rr>rrrrforward1 zGlobLN.forwardNr r7r8r9r:floatrBrrrrr=.r=c@r<)ChanLNz*Channel-wise Layer Normalization (chanLN).r r>cCs<tj|ddd}tj|dddd}|||||S)zApplies forward pass. Works for any input size > 2D. Args: x (:class:`torch.Tensor`): `[batch, chan, *]` Returns: :class:`torch.Tensor`: chanLN_x `[batch, chan, *]` rTr Fr)rrrr6r)r0rr>rrrrrrBCs zChanLN.forwardNrDrErrrrrH@rGrHc@r<)CumLNz-Cumulative Global layer normalization(cumLN).r r>c Cs|\}}}tj|jddddd}tj|djddddd}tj|||d||j|jdddd}||} ||| d} | || | | S)z Args: x (:class:`torch.Tensor`): Shape `[batch, channels, length]` Returns: :class:`torch.Tensor`: cumLN_x `[batch, channels, length]` rT)rr3)r r)startendstepdtypedevice) sizercumsumsumpowrrMrNviewr6r) r0rr>batchchanspec_lencum_sum cum_pow_sumcntcum_meancum_varrrrrBVs z CumLN.forwardNrDrErrrrrISrGrIc@r<) FeatsGlobLNznFeature-wise global Layer Normalization (FeatsGlobLN). Applies normalization over frames for each channel.r r>cCr?)zApplies forward pass. Works for any input size > 2D. Args: x (:class:`torch.Tensor`): `[batch, chan, time]` Returns: :class:`torch.Tensor`: chanLN_x `[batch, chan, time]` r@)r%r6rArrrrBmrCzFeatsGlobLN.forwardNrDrErrrrr\isr\c@seZdZdZddZdS) BatchNormz5Wrapper class for pytorch BatchNorm1D and BatchNorm2DcCs.|dks |dkrtd|dS)Nrz'expected 4D or 3D input (got {}D input))r ValueErrorformat)r0inputrrr_check_input_dimszBatchNorm._check_input_dimN)r7r8r9r:rbrrrrr]|s r]cCsL|jtvs|jtvrtd|jdt|j|idS)zqRegister a custom norm, gettable with `norms.get`. Args: custom_norm: Custom norm to register. zNorm z% already exists. Choose another name.N)r7globalskeyslowerr_update) custom_normrrr register_norms$rhcCsZ|durdSt|r |St|tr%t|}|dur#tdt||Stdt|)zReturns a norm class from a string. Returns its input if it is callable (already a :class:`._LayerNorm` for example). Args: identifier (str or Callable or None): the norm identifier. Returns: :class:`._LayerNorm` or None Nz.Could not interpret normalization identifier: )callable isinstancestrrcgetr_) identifierclsrrrrls   rlcCs t|}|dur dSttj|S)zQLike `.get` but returns a complex norm created with `asteroid.complex_nn.OnReIm`.N)rlrrOnReIm)rmnormrrr get_complexs rqrD)! functoolsrrrtorch.nn.modules.batchnormrtypingrrutils.torch_utilsrr>intrFrr#r%Moduler&r=rHrIr\r]gLNfgLNcLNcgLNbNrhrlrqrrrrs6