o %ݫi")@sZdZddlZddlZGdddejjjZGdddejjjZGdddejjjZ dS) zA combination of Convolutional, Recurrent, and Fully-connected networks. Authors * Mirco Ravanelli 2020 * Peter Plantinga 2020 * Ju-Chieh Chou 2020 * Titouan Parcollet 2020 * Abdel 2020 Ncs^eZdZdZddejjddddgddddejj j ddgdd d d ddd d dffd d Z Z S)CRDNNa6 This model is a combination of CNNs, RNNs, and DNNs. This model expects 3-dimensional input [batch, time, feats] and by default produces output of the size [batch, time, dnn_neurons]. One exception is if ``using_2d_pooling`` or ``time_pooling`` is True. In this case, the time dimension will be downsampled. Arguments --------- input_size : int The length of the expected input at the third dimension. input_shape : tuple While input_size will suffice, this option can allow putting CRDNN into a sequential with other classes. activation : torch class A class used for constructing the activation layers for CNN and DNN. dropout : float Neuron dropout rate as applied to CNN, RNN, and DNN. cnn_blocks : int The number of convolutional neural blocks to include. cnn_channels : list of ints A list of the number of output channels for each CNN block. cnn_kernelsize : tuple of ints The size of the convolutional kernels. time_pooling : bool Whether to pool the utterance on the time axis before the RNN. time_pooling_size : int The number of elements to pool on the time axis. freq_pooling_size : int The number of elements to pool on the frequency axis. rnn_class : torch class The type of RNN to use in CRDNN network (LiGRU, LSTM, GRU, RNN) inter_layer_pooling_size : list of ints A list of the pooling sizes for each CNN block. using_2d_pooling: bool Whether using a 2D or 1D pooling after each CNN block. rnn_layers : int The number of recurrent RNN layers to include. rnn_neurons : int Number of neurons in each layer of the RNN. rnn_bidirectional : bool Whether this model will process just forward or in both directions. rnn_re_init : bool, If True, an orthogonal initialization will be applied to the recurrent weights. dnn_blocks : int The number of linear neural blocks to include. dnn_neurons : int The number of neurons in the linear layers. projection_dim : int The number of neurons in the projection layer. This layer is used to reduce the size of the flattened representation obtained after the CNN blocks. use_rnnp: bool If True, a linear projection layer is added between RNN layers. Example ------- >>> inputs = torch.rand([10, 15, 60]) >>> model = CRDNN(input_shape=inputs.shape) >>> outputs = model(inputs) >>> outputs.shape torch.Size([10, 15, 512]) N333333?)rFiTc s|dur |dur td|durdd|g}tj|d|dkr*|jtjjjddt|D]}|j jt |||| | |||d|dq.|rX|jtjj j dd | d d d d|d kr|jtjjjdd|j jtjjj|dddd|j jtjjjdd|j j|dd|dkr|r|jtjjjddt|D]"}|j| |d ||d|jtjjj|ddd|tjj|dqn |j| d|||||d|dkr|jtjjjddt|D]}|jjt|||d|dqdS)Nz-Must specify one of input_size or input_shape input_shaperCNN layer_nameblock_)channels kernel_size using_2d_pool pooling_size activationdropoutrmaxr pool_type input_dimsr pool_axis time_poolingr projectionTlinear) n_neuronsbias combine_dimsrnormactRNN) hidden_size num_layers bidirectionalre_init)rr r!p)rr%r&rr'r(DNN)neuronsrrr) ValueErrorsuper__init__appendsbnnet containers Sequentialranger CNN_Blockpooling Pooling1drrLinear normalization LayerNormtorchnnDropoutr+ DNN_Block)self input_sizer rr cnn_blocks cnn_channelscnn_kernelsizertime_pooling_sizefreq_pooling_size rnn_classinter_layer_pooling_sizeusing_2d_pooling rnn_layers rnn_neuronsrnn_bidirectional rnn_re_init dnn_blocks dnn_neuronsprojection_dimuse_rnnp block_index_ __class__R/home/ubuntu/.local/lib/python3.10/site-packages/speechbrain/lobes/models/CRDNN.pyr/Ss       zCRDNN.__init__) __name__ __module__ __qualname____doc__r<r= LeakyReLUr1r2r$LiGRUr/ __classcell__rVrVrTrWrs0Drcs4eZdZdZddgejjdddffdd ZZS)r6a/CNN Block, based on VGG blocks. Arguments --------- input_shape : tuple Expected shape of the input. channels : int Number of convolutional channels for the block. kernel_size : tuple Size of the 2d convolutional kernel activation : torch.nn.Module class A class to be used for instantiating an activation layer. using_2d_pool : bool Whether to use 2d pooling or only 1d pooling. pooling_size : int Size of pooling kernel, duplicated for 2d pooling. dropout : float Rate to use for dropping channels. Example ------- >>> inputs = torch.rand(10, 15, 60) >>> block = CNN_Block(input_shape=inputs.shape, channels=32) >>> outputs = block(inputs) >>> outputs.shape torch.Size([10, 15, 30, 32]) rFrrcstj|d|jtjjj||dd|jtjjjdd|j|dd|jtjjj||dd|jtjjjdd|j|d d|rW|jtjj j d ||fd d d dn|jtjj j d d|ddd d|jtjj j |ddddS)Nr conv_1) out_channelsrrnorm_1r act_1conv_2norm_2act_2r)rr)rrrr7rrr) drop_ratedrop)r.r/r0r1r2r Conv2dr:r;r7 Pooling2dr8r Dropout2d)r@r rrrrrrrTrVrWr/sL    zCNN_Block.__init__ rXrYrZr[r<r=r\r/r^rVrVrTrWr6s r6cs*eZdZdZejjdffdd ZZS)r?a*Block for linear layers. Arguments --------- input_shape : tuple Expected shape of the input. neurons : int Size of the linear layers. activation : torch.nn.Module class Class definition to use for constructing activation layers. dropout : float Rate to use for dropping neurons. Example ------- >>> inputs = torch.rand(10, 15, 128) >>> block = DNN_Block(input_shape=inputs.shape, neurons=64) >>> outputs = block(inputs) >>> outputs.shape torch.Size([10, 15, 64]) rcsdtj|d|jtjjj|dd|jtjjjdd|j|dd|jt j j |ddddS) Nr r)rrr"r r#r)r) r.r/r0r1r2rr9r: BatchNorm1dr<r=r>)r@r r,rrrTrVrWr/0szDNN_Block.__init__rkrVrVrTrWr?sr?) r[r< speechbrainr1r2r3r4rr6r?rVrVrVrWs 8R