o %ݫi@sbdZddlmZddlZddlmZddlmZmZm Z ddl m Z e e Z ddZdd ZGd d d ejjZGd d d ejjZGdddejjZGdddejZGdddejZGdddejZGdddejZGdddejjZGdddejjZGdddejjZGdddejjZGd d!d!ejjZGd"d#d#ejZd$d%ZdS)&zLibrary implementing recurrent neural networks. Authors * Adel Moumen 2023 * Mirco Ravanelli 2020 * Ju-Chieh Chou 2020 * Jianyuan Zhong 2020 * Loren Lugosch 2020 )OptionalN)ContentBasedAttentionKeyValueAttentionLocationAwareAttention) get_loggercCs*||d}tjjjj||dddS)zReturns packed speechbrain-formatted tensors. Arguments --------- inputs : torch.Tensor The sequences to pack. lengths : torch.Tensor The length of each sequence. Returns ------- The packed sequences. TF) batch_firstenforce_sorted)sizecputorchnnutilsrnnpack_padded_sequence)inputslengthsrH/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/nnet/RNN.pyrs rcCstjjjj|dd\}}|S)aReturns speechbrain-formatted tensor from packed sequences. Arguments --------- inputs : torch.nn.utils.rnn.PackedSequence An input set of sequences to convert to a tensor. Returns ------- outputs : torch.Tensor The padded sequences. T)r)r r rrpad_packed_sequence)routputsrrrrr.s  rcs<eZdZdZ        d fdd Zd d d ZZS)RNNaThis function implements a vanilla RNN. It accepts in input tensors formatted as (batch, time, fea). In the case of 4d inputs like (batch, time, fea, channel) the tensor is flattened as (batch, time, fea*channel). Arguments --------- hidden_size : int Number of output neurons (i.e, the dimensionality of the output). values (i.e, time and frequency kernel sizes respectively). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. nonlinearity : str Type of nonlinearity (tanh, relu). num_layers : int Number of layers to employ in the RNN architecture. bias : bool If True, the additive bias b is adopted. dropout : float It is the dropout factor (must be between 0 and 1). re_init : bool If True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. bidirectional : bool If True, a bidirectional model that scans the sequence both right-to-left and left-to-right is used. Example ------- >>> inp_tensor = torch.rand([4, 10, 20]) >>> net = RNN(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor, _ = net(inp_tensor) >>> torch.Size([4, 10, 5]) NrelurTFc std|_|dur|durtd|dur-t|dkr!d|_tt|dd}tjj ||||| |d|d|_ |rEt |j dSdS)NF*Expected one of input_shape or input_size.T) input_size hidden_size num_layersdropout bidirectionalbiasr nonlinearity) super__init__reshape ValueErrorlenr prodtensorr rrrnn_init) selfr input_shaperr#rr"r re_initr! __class__rrr%is*   z RNN.__init__cC|jr|jdkr||jd|jd|jd|jd}|j|dur+t||}|dur9|j||d\}}n||\}}|durHt|}||fS)aReturns the output of the vanilla RNN. Arguments --------- x : torch.Tensor Input tensor. hx : torch.Tensor Starting hidden state. lengths : torch.Tensor Relative lengths of the input signals. Returns ------- output : torch.Tensor The output of the vanilla RNN hn : torch.Tensor The hidden states. rrrrNhxr&ndimshaperflatten_parametersrrr,xr4routputhnrrrforward *  z RNN.forward)NNrrTrTFNN__name__ __module__ __qualname____doc__r%r= __classcell__rrr/rrAs*&rc:eZdZdZ       d fdd Zd d d ZZS) LSTMaHThis function implements a basic LSTM. It accepts in input tensors formatted as (batch, time, fea). In the case of 4d inputs like (batch, time, fea, channel) the tensor is flattened as (batch, time, fea*channel). Arguments --------- hidden_size : int Number of output neurons (i.e, the dimensionality of the output). values (i.e, time and frequency kernel sizes respectively). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. num_layers : int Number of layers to employ in the RNN architecture. bias : bool If True, the additive bias b is adopted. dropout : float It is the dropout factor (must be between 0 and 1). re_init : bool It True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. bidirectional : bool If True, a bidirectional model that scans the sequence both right-to-left and left-to-right is used. Example ------- >>> inp_tensor = torch.rand([4, 10, 20]) >>> net = LSTM(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor = net(inp_tensor) >>> torch.Size([4, 10, 5]) NrTrFc td|_|dur|durtd|dur/t|dkr!d|_tt|dd}tj j ||||||dd|_ |rFt |j dSdSNFrrTr)rrrr r!r"r) r$r%r&r'r(r r)r*itemr rGrr+ r,rr-rrr"r r.r!r/rrr%(   z LSTM.__init__cCr1)aReturns the output of the LSTM. Arguments --------- x : torch.Tensor Input tensor. hx : torch.Tensor Starting hidden state. lengths : torch.Tensor Relative length of the input signals. Returns ------- output : torch.Tensor The output of the LSTM. hn : torch.Tensor The hidden states. r2rrrrNr3r5r9rrrr=r>z LSTM.forwardNNrTrTFr?r@rrr/rrG($rGcrF) GRUaEThis function implements a basic GRU. It accepts input tensors formatted as (batch, time, fea). In the case of 4d inputs like (batch, time, fea, channel) the tensor is flattened as (batch, time, fea*channel). Arguments --------- hidden_size : int Number of output neurons (i.e, the dimensionality of the output). values (i.e, time and frequency kernel sizes respectively). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. num_layers : int Number of layers to employ in the RNN architecture. bias : bool If True, the additive bias b is adopted. dropout: float It is the dropout factor (must be between 0 and 1). re_init : bool If True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. bidirectional : bool If True, a bidirectional model that scans the sequence both right-to-left and left-to-right is used. Example ------- >>> inp_tensor = torch.rand([4, 10, 20]) >>> net = GRU(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor, _ = net(inp_tensor) >>> torch.Size([4, 10, 5]) NrTrFc rHrI) r$r%r&r'r(r r)r*rJr rOrr+rKr/rrr%WrLz GRU.__init__cCr1)aReturns the output of the GRU. Arguments --------- x : torch.Tensor Input tensor. hx : torch.Tensor Starting hidden state. lengths : torch.Tensor Relative length of the input signals. Returns ------- output : torch.Tensor Output of GRU. hn : torch.Tensor Hidden states. r2rrrrNr3r5r9rrrr={r>z GRU.forwardrMr?r@rrr/rrO1rNrOcrF) RNNCella]This class implements a basic RNN Cell for a timestep of input, while RNN() takes the whole sequence as input. It is designed for an autoregressive decoder (ex. attentional decoder), which takes one input at a time. Using torch.nn.RNNCell() instead of torch.nn.RNN() to reduce VRAM consumption. It accepts in input tensors formatted as (batch, fea). Arguments --------- hidden_size : int Number of output neurons (i.e, the dimensionality of the output). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. num_layers : int Number of layers to employ in the RNN architecture. bias : bool If True, the additive bias b is adopted. dropout : float It is the dropout factor (must be between 0 and 1). re_init : bool It True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. nonlinearity : str Type of nonlinearity (tanh, relu). Example ------- >>> inp_tensor = torch.rand([4, 20]) >>> net = RNNCell(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor, _ = net(inp_tensor) >>> out_tensor.shape torch.Size([4, 5]) NrTrtanhc st||_||_|dur|durtd|dur0t|dkr$d|_tt |dd}||j||d} t tj j d i| g|_ |j| d<t|jdD]} |j tj j d i| qRt fddt|jdD|_|r}t|j dSdS) NrrTr)rrr"r#rcg|] }tjjdqSpr r Dropout.0_r rr z$RNNCell.__init__..r)r$r%rrr'r(r&r r)r*r ModuleListrP rnn_cellsrangeappenddropout_layersr+) r,rr-rrr"r r.r#kwargsir/r[rr%s0   zRNNCell.__init__cC|dur||j|jd|j}|jd||d}|g}td|jD]}|j|d|}|j||||}||q#tj |dd}||fS)abReturns the output of the RNNCell. Arguments --------- x : torch.Tensor The input of RNNCell. hx : torch.Tensor The hidden states of RNNCell. Returns ------- h : torch.Tensor Outputs of RNNCell. hidden : torch.Tensor Hidden states. Nrrdim new_zerosrr7rr_r`rbrar stackr,r:r4h hidden_lstrddrop_hhiddenrrrr=s zRNNCell.forward)NNrTrTrQNr@rrr/rrPs*,rPc8eZdZdZ      d fdd Zd dd ZZS) GRUCellaThis class implements a basic GRU Cell for a timestep of input, while GRU() takes the whole sequence as input. It is designed for an autoregressive decoder (ex. attentional decoder), which takes one input at a time. Using torch.nn.GRUCell() instead of torch.nn.GRU() to reduce VRAM consumption. It accepts in input tensors formatted as (batch, fea). Arguments --------- hidden_size: int Number of output neurons (i.e, the dimensionality of the output). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. num_layers : int Number of layers to employ in the GRU architecture. bias : bool If True, the additive bias b is adopted. dropout : float It is the dropout factor (must be between 0 and 1). re_init : bool It True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. Example ------- >>> inp_tensor = torch.rand([4, 20]) >>> net = GRUCell(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor, _ = net(inp_tensor) >>> out_tensor.shape torch.Size([4, 5]) NrTrc t||_||_|dur|durtd|dur0t|dkr$d|_tt |dd}||j|d}t tj j d i|g|_ |j|d<t|jdD]} |j tj j d i|qQt fddt|jdD|_|r|t|j dSdS) NrrTrrrr"rcrRrSrVrXr[rrr\dr]z$GRUCell.__init__..r)r$r%rrr'r(r&r r)r*r r^rrr_r`rarbr+ r,rr-rrr"r r.rcrdr/r[rr%@.   zGRUCell.__init__cCre)aaReturns the output of the GRUCell. Arguments --------- x : torch.Tensor The input of GRUCell. hx : torch.Tensor The hidden states of GRUCell. Returns ------- h : torch.Tensor Outputs of GRUCell hidden : torch.Tensor Hidden states. Nrrrfrhrkrrrr=js zGRUCell.forwardNNrTrTrpr@rrr/rrr'*rrcrq) LSTMCellaThis class implements a basic LSTM Cell for a timestep of input, while LSTM() takes the whole sequence as input. It is designed for an autoregressive decoder (ex. attentional decoder), which takes one input at a time. Using torch.nn.LSTMCell() instead of torch.nn.LSTM() to reduce VRAM consumption. It accepts in input tensors formatted as (batch, fea). Arguments --------- hidden_size: int Number of output neurons (i.e, the dimensionality of the output). input_shape : tuple The shape of an example input. Alternatively, use ``input_size``. input_size : int The size of the input. Alternatively, use ``input_shape``. num_layers : int Number of layers to employ in the LSTM architecture. bias : bool If True, the additive bias b is adopted. dropout : float It is the dropout factor (must be between 0 and 1). re_init : bool If True, orthogonal initialization is used for the recurrent weights. Xavier initialization is used for the input connection weights. Example ------- >>> inp_tensor = torch.rand([4, 20]) >>> net = LSTMCell(hidden_size=5, input_shape=inp_tensor.shape) >>> out_tensor, _ = net(inp_tensor) >>> out_tensor.shape torch.Size([4, 5]) NrTrc rs) NrrTrrtrcrRrSrVrXr[rrr\r]z%LSTMCell.__init__..r)r$r%rrr'r(r&r r)r*r r^ryr_r`rarbr+rur/r[rr%rvzLSTMCell.__init__c Cs|dur||j|jd|j||j|jd|jf}|jd||dd|ddf\}}|g}|g}td|jD])}|j|d|}|j|||d||d|f\}}||||q