o åÊi½ã@sPddlZddlmZmZmZddlmZGdd„dejƒZGdd„dejƒZdS)éN)ÚnnÚsinÚpow)Ú Parametercó,eZdZdZ d ‡fdd„ Zdd„Z‡ZS) ÚSnakeaÓ Implementation of a sine-based periodic activation function Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter References: - This activation function is from this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snake(256) >>> x = torch.randn(256) >>> x = a1(x) çð?TFcs\tt|ƒ ¡||_||_|jrtt |¡|ƒ|_n tt  |¡|ƒ|_||j_ d|_ dS)a  Initialization. INPUT: - in_features: shape of the input - alpha: trainable parameter alpha is initialized to 1 by default, higher values = higher-frequency. alpha will be trained along with the rest of your model. ç•Ö&è .>N) ÚsuperrÚ__init__Ú in_featuresÚalpha_logscalerÚtorchÚzerosÚalphaÚonesÚ requires_gradÚno_div_by_zero©Úselfr rÚalpha_trainabler ©Ú __class__©úH/home/ubuntu/.local/lib/python3.10/site-packages/neucodec/activations.pyr s  zSnake.__init__cCsJ|j d¡ d¡}|jrt |¡}|d||jtt||ƒdƒ}|S)uŽ Forward pass of the function. Applies the function to the input elementwise. Snake ∶= x + 1/a * sin^2 (xa) réÿÿÿÿré)rÚ unsqueezer rÚexprrr)rÚxrrrrÚforward3s  $z Snake.forward©rTF©Ú__name__Ú __module__Ú __qualname__Ú__doc__r r Ú __classcell__rrrrr s ÿrcr) Ú SnakeBetaam A modified Snake function which uses separate parameters for the magnitude of the periodic components Shape: - Input: (B, C, T) - Output: (B, C, T), same shape as the input Parameters: - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude References: - This activation function is a modified version based on this paper by Liu Ziyin, Tilman Hartwig, Masahito Ueda: https://arxiv.org/abs/2006.08195 Examples: >>> a1 = snakebeta(256) >>> x = torch.randn(256) >>> x = a1(x) rTFcsŒtt|ƒ ¡||_||_|jr%tt |¡|ƒ|_tt |¡|ƒ|_ ntt  |¡|ƒ|_tt  |¡|ƒ|_ ||j_ ||j _ d|_ dS)aÍ Initialization. INPUT: - in_features: shape of the input - alpha - trainable parameter that controls frequency - beta - trainable parameter that controls magnitude alpha is initialized to 1 by default, higher values = higher-frequency. beta is initialized to 1 by default, higher values = higher-magnitude. alpha will be trained along with the rest of your model. r N) r r(r r r rrrrÚbetarrrrrrrr Ss  zSnakeBeta.__init__cCsf|j d¡ d¡}|j d¡ d¡}|jrt |¡}t |¡}|d||jtt||ƒdƒ}|S)u’ Forward pass of the function. Applies the function to the input elementwise. SnakeBeta ∶= x + 1/b * sin^2 (xa) rrrr) rrr)r rrrrr)rrrr)rrrr qs  $zSnakeBeta.forwardr!r"rrrrr(As ÿr() rrrrÚtorch.nnrÚModulerr(rrrrÚs  8