o ½e¦iø6ã@s¾dZddlZddlZddlmZddlmmZddlm Z ddl m Z ddd„Z ddd „ZGd d „d ejƒZGd d „d ejƒZGdd„dejƒZGdd„dejƒZGdd„dejjƒZdS)zNResNet PreActivated for speaker verification Authors * Mickael Rouvier 2022 éN)ÚLinear)Ú BatchNorm1décCstj||d|dddS)z#2D convolution with kernel_size = 3érF)Ú kernel_sizeÚstrideÚpaddingÚbias©ÚnnÚConv2d©Ú in_planesÚ out_planesr©rú]/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/lobes/models/ResNet.pyÚconv3x3súrcCstj||d|ddS)z#2D convolution with kernel_size = 1rF©rrr r r rrrÚconv1x1s ÿrcs0eZdZdZdejf‡fdd„ Zdd„Z‡ZS)ÚSEBlockaÙAn implementation of Squeeze-and-Excitation Block. Arguments --------- channels : int The number of channels. reduction : int The reduction factor of channels. activation : Callable The function to apply between layers. Example ------- >>> inp_tensor = torch.rand([1, 64, 80, 40]) >>> se_layer = SEBlock(64) >>> out_tensor = se_layer(inp_tensor) >>> out_tensor.shape torch.Size([1, 64, 80, 40]) rcsNtt|ƒ ¡t d¡|_t t |||¡|ƒt |||¡t ¡¡|_ dS)Nr) ÚsuperrÚ__init__r ÚAdaptiveAvgPool2dÚavg_poolÚ SequentialrÚSigmoidÚfc)ÚselfÚchannelsÚ reductionÚ activation©Ú __class__rrr;s  üzSEBlock.__init__cCs@| ¡\}}}}| |¡ ||¡}| |¡ ||dd¡}||S)ú^Intermediate step. Processes the input tensor x and returns an output tensor. r)ÚsizerÚviewr)rÚxÚbÚcÚ_ÚyrrrÚforwardGszSEBlock.forward© Ú__name__Ú __module__Ú __qualname__Ú__doc__r ÚReLUrr+Ú __classcell__rrr!rr&s rcs2eZdZdZddejf‡fdd„ Zdd„Z‡ZS)Ú BasicBlocka–An implementation of ResNet Block. Arguments --------- in_channels : int Number of input channels. out_channels : int The number of output channels. stride : int Factor that reduce the spatial dimensionality downsample : torch function A function for downsample the identity of block when stride != 1 activation : torch class A class for constructing the activation layers. Example ------- >>> inp_tensor = torch.rand([1, 64, 80, 40]) >>> layer = BasicBlock(64, 64, stride=1) >>> out_tensor = layer(inp_tensor) >>> out_tensor.shape torch.Size([1, 64, 80, 40]) rNcsptt|ƒ ¡|ƒ|_t |¡|_t|||ƒ|_t |¡|_ t||ƒ|_ t |¡|_ t ||ƒ|_ ||_||_dS©N)rr3rr r Ú BatchNorm2dÚbn1rÚconv1Úbn2Úconv2Úbn3rÚconv3Ú downsampler)rÚ in_channelsÚ out_channelsrr<r r!rrrjs      zBasicBlock.__init__cCs~|}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}|jdur9| |¡}||7}|S©r#N)r6r r7r8r9r:r;r<©rr&ÚresidualÚoutrrrr+s           zBasicBlock.forwardr,rrr!rr3Qsúr3cs4eZdZdZdddejf‡fdd„ Zdd„Z‡ZS)Ú SEBasicBlockaíAn implementation of Squeeze-and-Excitation ResNet Block. Arguments --------- in_channels : int Number of input channels. out_channels : int The number of output channels. reduction : int The reduction factor of channels. stride : int Factor that reduce the spatial dimensionality downsample : torch function A function for downsample the identity of block when stride != 1 activation : torch class A class for constructing the activation layers. Example ------- >>> inp_tensor = torch.rand([1, 64, 80, 40]) >>> layer = SEBasicBlock(64, 64, stride=1) >>> out_tensor = layer(inp_tensor) >>> out_tensor.shape torch.Size([1, 64, 80, 40]) rNcs|tt|ƒ ¡|ƒ|_t |¡|_t|||ƒ|_t |¡|_ t||ƒ|_ t |¡|_ t ||ƒ|_ ||_||_t||ƒ|_dSr4)rrCrr r r5r6rr7r8r9r:rr;r<rrÚse)rr=r>rrr<r r!rrrµs      zSEBasicBlock.__init__cCsˆ|}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}|jdur>| |¡}||7}|Sr?) r6r r7r8r9r:r;rDr<r@rrrr+Ïs            zSEBasicBlock.forwardr,rrr!rrCšsùrCcs^eZdZdZddejjgd¢gd¢gd¢df‡fdd „ Zdd d „Zdd d„Z ddd„Z ‡Z S)ÚResNetaAn implementation of ResNet Arguments --------- input_size : int Expected size of the input dimension. device : str Device used, e.g., "cpu" or "cuda". activation : torch class A class for constructing the activation layers. channels : list of ints List of number of channels used per stage. block_sizes : list of ints List of number of groups created per stage. strides : list of ints List of stride per stage. lin_neurons : int Number of neurons in linear layers. Example ------- >>> input_feats = torch.rand([2, 400, 80]) >>> compute_embedding = ResNet(lin_neurons=256) >>> outputs = compute_embedding(input_feats) >>> outputs.shape torch.Size([2, 256]) éPÚcpu)é€rHérI)réér)rérLrLrIc stƒ ¡t|ƒdks J‚t|ƒdksJ‚t|ƒdksJ‚t ||d|d|d|d¡}tjd|dddddd|_t |d¡|_ |ƒ|_ |j |d|d|d|dd|_ |j |d|d|d|dd|_ |j|d|d|d|dd|_|j|d|d|d|dd|_tj d||d ¡|_t tj|d |d dd t ¡t d ¡tjd |d |dd tjdd ¡|_t d||d |¡|_tj |¡|_| ¡D]*} t| tjƒrótjj| j d ddqàt| tjƒr tj !| j d¡tj !| j"d¡qàdS)NrJrrrLrF)r )réÿÿÿÿrH)r©ÚdimÚfan_outÚrelu)ÚmodeÚ nonlinearity)#rrÚlenÚmathÚceilr r r7r5r6Ú activation1Ú_make_layer_seÚlayer1Úlayer2Ú _make_layerÚlayer3Úlayer4ÚtorchrÚ norm_statsrÚConv1dr1ÚSoftmaxÚ attentionrÚfc_embedÚ norm_embedÚmodulesÚ isinstanceÚinitÚkaiming_normal_ÚweightÚ constant_r ) rÚ input_sizeÚdevicer rÚ block_sizesÚstridesÚ lin_neuronsÚ input_outÚmr!rrrsT "ÿÿÿÿÿ û  ÿ€ùzResNet.__init__rc Cs|d}|dks ||krt tj||d|ddt |¡¡}g}| t||d||ƒ¡td|ƒD] }| t||dƒ¡q-tj|ŽS)aùConstruct the squeeze-and-excitation block layer. Arguments --------- in_channels : int Number of input channels. out_channels : int The number of output channels. block_num: int Number of ResNet blocks for the network. stride : int Factor that reduce the spatial dimensionality. Default is 1 Returns ------- se_block : nn.Sequential Squeeze-and-excitation block NrFr)r rr r5ÚappendrCÚrange©rr=r>Ú block_numrr<ÚlayersÚirrrrXCs&ûø ÿ zResNet._make_layer_sec Csxd}|dks ||krt tj||d|ddt |¡¡}g}| t||||ƒ¡td|ƒD] }| t||ƒ¡q,tj|ŽS)aß Construct the ResNet block layer. Arguments --------- in_channels : int Number of input channels. out_channels : int The number of output channels. block_num: int Number of ResNet blocks for the network. stride : int Factor that reduce the spatial dimensionality. Default is 1 Returns ------- block : nn.Sequential ResNet block NrFr)r rr r5rrr3rsrtrrrr[ls"ûø  zResNet._make_layerNcCsä| d¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| |¡}| dd¡}|  dd¡}|  |¡}t j ||dd}t   t j |d|dd|djdd¡}t j||gdd}| |¡}| |¡}| |¡}|S)aMReturns the embedding vector. Arguments --------- x : torch.Tensor Tensor of shape (batch, time, channel). lengths : torch.Tensor Corresponding relative lengths of the inputs. Returns ------- x : torch.Tensor The embedding vector. rrLrrNgñh㈵øä>)Úmin)Ú unsqueezer7r6rWrYrZr\r]Ú transposeÚflattenrbr^ÚsumÚsqrtÚclampÚcatr_rcrd)rr&ÚlengthsÚwÚmuÚsgrrrr+”s$           ,   zResNet.forward©rr4) r-r.r/r0r^r r1rrXr[r+r2rrr!rrEësø ; )(rEcs2eZdZdZ    d ‡fdd„ Zdd „Z‡ZS) Ú ClassifieraßThis class implements the cosine similarity on the top of features. Arguments --------- input_size : int Expected size of the inputs. device : str Device used, e.g., "cpu" or "cuda". lin_blocks : int Number of linear layers. lin_neurons : int Number of neurons in linear layers. out_neurons : int Number of classes. Example ------- >>> classify = Classifier(input_size=2, lin_neurons=2, out_neurons=2) >>> outputs = torch.tensor([ [1., -1.], [-9., 1.], [0.9, 0.1], [0.1, 0.9] ]) >>> outputs = outputs.unsqueeze(1) >>> cos = classify(outputs) >>> (cos < -1.0).long().sum() tensor(0) >>> (cos > 1.0).long().sum() tensor(0) rGrrIé»csntƒ ¡t ¡|_t|ƒD]}|j t|dt||dg¡|}qt  t j |||d¡|_ tj  |j ¡dS)N)rk)rkÚ n_neurons)rl)rrr Ú ModuleListÚblocksrsÚextendÚ _BatchNorm1drÚ Parameterr^Ú FloatTensorrirgÚxavier_uniform_)rrkrlÚ lin_blocksroÚ out_neuronsÚ block_indexr!rrrÚs    þÿÿzClassifier.__init__cCs>|jD]}||ƒ}qt t | d¡¡t |j¡¡}| d¡S)zúReturns the output probabilities over speakers. Arguments --------- x : torch.Tensor Torch tensor. Returns ------- x : torch.Tensor Output probabilities over speakers. r)r‰ÚFÚlinearÚ normalizeÚsqueezeriry)rr&Úlayerrrrr+ôs   zClassifier.forward)rGrrIr†)r-r.r/r0rr+r2rrr!rr…¾súr…r„)r0rUr^Útorch.nnr Útorch.nn.functionalÚ functionalr’Úspeechbrain.nnet.linearrÚspeechbrain.nnet.normalizationrr‹rrÚModulerr3rCrEr…rrrrÚs     +IQT