o }oi@svddlmZddlZddlZddlmZddlmZmZddl m Z m Z m Z m Z ddlmZdgZGdddeZdS) )DictN)ConformerEncoder) NeuralModule typecheck) ChannelType LengthsType NeuralTypeSpectrogramType)loggingSpectrogramConformercseZdZdZddedeffdd ZeddZejd dZed e e e ffd d Z ed e e e ffd dZ e dddZZS)r a'A Conformer-based model for processing complex-valued spectrograms. This model processes complex-valued inputs by stacking real and imaginary components along the channel dimension. The stacked tensor is processed using Conformer layers, and the output is projected back to generate real and imaginary components of the output channels. Args: in_channels: number of input complex-valued channels out_channels: number of output complex-valued channels kwargs: additional arguments for ConformerEncoder  in_channels out_channelsc st|dkrtd|||_|dkrtd|||_|}d|j|d|d<|d<td|td i||_ t j j d|jd|jdd|_ td |jjtd |jtd |jdS) Nr zKNumber of input channels needs to be larger or equal to one, current value zLNumber of output channels needs to be larger or equal to one, current value feat_infeat_outzConformer params: %s)r r kernel_sizezInitialized %s withz in_channels: %sz out_channels: %s)super__init__ ValueErrorr rcopyr debugr conformertorchnnConv2doutput_projection __class____name__)selfr rkwargsconformer_paramsrre/home/ubuntu/.local/lib/python3.10/site-packages/nemo/collections/audio/parts/submodules/conformer.pyr*s,  zSpectrogramConformer.__init__cCs|jjS)axReturns the attention context size used by the conformer encoder. The context size is a list of two integers [left_context, right_context] that defines how many frames to the left and right each frame can attend to in the self-attention layers. Returns: List[int]: The attention context size [left_context, right_context] )ratt_context_sizer rrr$ context_sizeNs z!SpectrogramConformer.context_sizecCs|j|dS)azSets the attention context size used by the conformer encoder. The context size is a list of two integers [left_context, right_context] that defines how many frames to the left and right each frame can attend to in the self-attention layers. Args: value (List[int]): The attention context size [left_context, right_context] N)rset_default_att_context_size)r valuerrr$r'[s returnc CLtdttdtddtdtddtdtddttdtdddS) +Returns definitions of module output ports.BCDTr.Toptionalr0r.r1r0r0r.r0r1r.)input input_lengthcache_last_channelcache_last_timecache_last_channel_lenrr rrtupler&rrr$ input_typesh z SpectrogramConformer.input_typesc Cr+) r,r-r2Tr3r5r6r.)output output_lengthcache_last_channel_nextcache_last_time_nextcache_last_channel_next_lenr<r&rrr$ output_typestr?z!SpectrogramConformer.output_typesNc Cs|j\}}}} ||jkrtd|d|jtj|j|jgdd} t| d}|dur6|j ||d\} } n|j |||||d\} } }}}tj| d |jd|d } | | } tj| d |j d|d } t | } |duro| | fS| | |||fS) a4Forward pass for the SpectrogramConformer model. This method processes complex-valued inputs by stacking real and imaginary components, passing the stacked tensor through Conformer layers, and projecting back to generate real and imaginary components of the output channels. zUnexpected input channel size z , expected r)dimzB C RI D T -> B (C RI D) TN) audio_signallength)rGrHr9r:r;zB (C RI D) T -> B (C RI) D T)r/RIr0zB (C RI) D T -> B C D T RI)shaper RuntimeErrorrstackrealimageinops rearrangerrrview_as_complex contiguous) r r7r8r9r:r;r.C_inr0r1input_real_imagr@rArrr$forwards*   zSpectrogramConformer.forward)r r )NNNN)r __module__ __qualname____doc__intrpropertyr'setterrstrrr>rErrU __classcell__rrr#r$r s $     )typingrrOr.nemo.collections.asr.modules.conformer_encoderrnemo.core.classesrrnemo.core.neural_typesrrrr nemo.utilsr __all__r rrrr$s