o ei@srdZddlZddlmmZddlmZeeZ Gdddejj Z Gdddejj Z Gdd d ejj Z dS) zcLibrary implementing activation functions. Authors * Mirco Ravanelli 2020 * Jianyuan Zhong 2020 N) get_loggercs4eZdZdZdddejffdd ZddZZS) SoftmaxaComputes the softmax of a 2d, 3d, or 4d input tensor. Arguments --------- apply_log : bool Whether to apply the log function before softmax. dim : int If the dimension where softmax is applied. reshape: bool whether to apply reshaping (true by default) dtype: torch.dtype dtype of the output tensor Example ------- >>> classifier = Softmax() >>> inputs = torch.rand(10, 50, 40) >>> output = classifier(inputs) >>> output.shape torch.Size([10, 50, 40]) FTcs6t|r tj|_ntj|_||_||_||_dSN) super__init__F log_softmaxactsoftmaxdimreshapedtype)self apply_logr r r __class__Z/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/nnet/activations.pyr's   zSoftmax.__init__cCs|j}|jr1t|dkr||d|d|d}t|dkr1||d|d|d|d}|j||j|jd}|jrgt|dkrQ||d|d|d}t|dkrg||d|d|d|d}|S)zReturns the softmax of the input tensor. Arguments --------- x : torch.Tensor Input tensor. Returns ------- x_act : torch.Tensor The softmax outputs. r)r r)shaper lenr r r)rxdimsx_actrrrforward5s  "   zSoftmax.forward) __name__ __module__ __qualname____doc__torchfloat32rr __classcell__rrrrrs  rcs*eZdZdZdfdd ZddZZS) GumbelSoftmaxaZSamples from the Gumbel-Softmax distribution and optionally discretizes. Reference: https://arxiv.org/abs/1611.00712, https://arxiv.org/abs/1611.01144 Arguments --------- tau: float non-negative scalar temperature hard: bool if True, the returned samples will be discretized as one-hot vectors, but will be differentiated as if it is the soft sample in autograd apply_log: bool if True, returns the log of the softmax outputs. Example ------- >>> x = torch.randn((8, 40, 120)) >>> act = GumbelSoftmax(0.8, True) >>> x = act(x) Fcs t||_||_||_dSr)rrtauhardr)rr'r(rrrrrns  zGumbelSoftmax.__init__cCs4|jrttj||j|jdStj||j|jdS)zReturns the Gumbel softmax of the input tensor. Arguments --------- x : torch.Tensor Input tensor. Returns ------- The Gumbel softmax output. )r'r()rr#logrgumbel_softmaxr'r(rrrrrrts zGumbelSoftmax.forward)FF)rr r!r"rrr%rrrrr&Ysr&cs0eZdZdZddeffdd ZddZZS) SwishaEThe class implements the Swish activation function from https://arxiv.org/pdf/2005.03191.pdf given input x. Swish(x) = x / (1 + exp(beta * x)) Arguments --------- beta: float Beta value. Example ------- >>> x = torch.randn((8, 40, 120)) >>> act = Swish() >>> x = act(x) ?betacs t||_tj|_dSr)rrr.r#nnSiLUsilu)rr.rrrrs zSwish.__init__cCs|jdkr ||j}||S)zReturns the Swished input tensor. Arguments --------- x : torch.Tensor Input tensor. Returns ------- The swished output. r)r.r1r+rrrrs  z Swish.forward)r-)rr r!r"floatrrr%rrrrr,sr,)r"r#torch.nn.functionalr/ functionalrspeechbrain.utils.loggerrrloggerModulerr&r,rrrrs I,