o i_@sddlZddlmZmZmZddlZddlZddlm Z ddlm Z ddl m Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZdd lmZmZdd lm Z ddl!m"Z"m#Z#m$Z$ddl%m&Z&e#ryddl'm(Z(e$)e*Z+Gddde j,Z-Gddde j,Z.GdddeZ/GdddeZ0GdddeZ1Gddde j,Z2Gddde j,Z3  dLd!e j,d"ej4d#ej4d$ej4d%eej4d&ee5d'e5d(eej4fd)d*Z6Gd+d,d,e j,Z7Gd-d.d.e j,Z8Gd/d0d0eZ9Gd1d2d2e j,Z:Gd3d4d4e j,Z;Gd5d6d6eZ  dMd;e?e@e@fde@d?ejBf d@dAZCe"GdBdCdCe>ZDdZEe"dDdEGdFdGdGe>ZFe"dHdEGdIdJdJe>ZGgdKZHdS)NN)CallableOptionalUnion)CrossEntropyLoss)ACT2FN)is_deepspeed_zero3_enabled)is_fsdp_managed_module)_prepare_4d_attention_mask#_prepare_4d_attention_mask_for_sdpa)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringis_torch_flex_attn_availablelogging) HubertConfig)make_flex_block_causal_maskc$eZdZfddZddZZS)HubertPositionalConvEmbeddingcs@ttj|j|j|j|jd|jd|_d|_|j r%t |j|_nmtj j }t tj jdr5tj jj }trddl}|jj|jjdd||jddd|_Wdn1sZwYt |jdrr|jjjj}|jjjj}n|jj}|jj}|j|||j||n ||jddd|_t|j|_t|j|_dS) N) kernel_sizepaddinggroups weight_normr modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normconv_pos_batch_norm BatchNorm1dutilsr hasattrr&r deepspeedzeroGatheredParametersr# original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerrrfeat_extract_activation activation)selfconfigr r4r9r: __class__g/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/transformers/models/hubert/modeling_hubert.pyr(3s:     z&HubertPositionalConvEmbedding.__init__cCsN|dd}|jdur||}||}||}||}|dd}|S)Nrr) transposer/r.rr>r? hidden_statesrCrCrDforwardXs       z%HubertPositionalConvEmbedding.forward__name__ __module__ __qualname__r(rH __classcell__rCrCrArDr2s %rcr)r<cs*t|ddkrd|_dSd|_dS)Nrrr)r'r(num_pad_remove)r?r,rArCrDr(es  zHubertSamePadLayer.__init__cCs,|jdkr|ddddd|j f}|SNr)rNrFrCrCrDrHis zHubertSamePadLayer.forwardrIrCrCrArDr<ds r<c&eZdZdfdd ZddZZS)HubertNoLayerNormConvLayerrcsjt|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ t |j |_ dS)Nrrrstridebias)r'r(conv_dim in_conv_dim out_conv_dimr)r* conv_kernel conv_stride conv_biasr.rr=r>r?r@layer_idrArCrDr(ps  z#HubertNoLayerNormConvLayer.__init__cCs||}||}|SN)r.r>rFrCrCrDrH~s  z"HubertNoLayerNormConvLayer.forwardrrIrCrCrArDrQosrQcrP)HubertLayerNormConvLayerrcs|t|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ tj |jdd|_ t |j|_dS)NrrrRT)elementwise_affine)r'r(rUrVrWr)r*rXrYrZr. LayerNorm layer_normrr=r>r[rArCrDr(s  z!HubertLayerNormConvLayer.__init__cCs:||}|dd}||}|dd}||}|S)N)r.rErbr>rFrCrCrDrHs     z HubertLayerNormConvLayer.forwardr^rIrCrCrArDr_sr_crP)HubertGroupNormConvLayerrcst|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ t |j |_ tj|j|jdd|_dS)NrrrRT) num_groups num_channelsaffine)r'r(rUrVrWr)r*rXrYrZr.rr=r> GroupNormrbr[rArCrDr(s   z!HubertGroupNormConvLayer.__init__cCs"||}||}||}|Sr])r.rbr>rFrCrCrDrHs   z HubertGroupNormConvLayer.forwardr^rIrCrCrArDresrecs0eZdZdZfddZddZddZZS)HubertFeatureEncoderz.Construct the features from raw audio waveformcstjdkr tddgfddtjdD}njdkr2fddtjD}n td jd t||_ d |_ d |_ dS) Ngrouprr\csg|] }t|ddqS)rrl)rQ.0ir@rCrD sz1HubertFeatureEncoder.__init__..rlayercsg|]}t|dqS)rl)r_rmrprCrDrqsz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FT) r'r(feat_extract_normrerangenum_feat_extract_layers ValueErrorr) ModuleList conv_layersgradient_checkpointing_requires_grad)r?r@rxrArprDr(s       zHubertFeatureEncoder.__init__cCs|D]}d|_qd|_dSNF) parameters requires_gradrzr?paramrCrCrD_freeze_parameterss  z'HubertFeatureEncoder._freeze_parameterscCs:|dddf}|jr|jrd|_|jD]}||}q|S)NT)rztrainingr}rx)r? input_valuesrG conv_layerrCrCrDrHs    zHubertFeatureEncoder.forward)rJrKrL__doc__r(rrHrMrCrCrArDrjs  rjcr)HubertFeatureProjectioncsXt|j|_|jrtj|jd|jd|_t|jd|j |_ t |j |_ dS)Nrdeps)r'r(feat_proj_layer_normr)rarUlayer_norm_epsrbLinearr+ projectionDropoutfeat_proj_dropoutdropoutr?r@rArCrDr(s z HubertFeatureProjection.__init__cCs(|jr||}||}||}|Sr])rrbrrrFrCrCrDrHs    zHubertFeatureProjection.forwardrIrCrCrArDrs rmodulequerykeyvalueattention_maskscalingr head_maskc Ks|dur |dd}t||dd|} |dur| |} tjj| dd} |dur5| |dddd} tjj| ||j d} t| |} | dd } | | fS)Nrdrrr%r)pr) sizetorchmatmulrEr) functionalsoftmaxviewrr contiguous) rrrrrrrrkwargs attn_weights attn_outputrCrCrDeager_attention_forwards  rcseZdZdZ     ddededed ed ed ed eeffd d Z    dde j dee j dee j dee j deede e dee j ee j eee j ffddZZS)HubertAttentionz=Multi-headed attention from 'Attention Is All You Need' paperrFTN embed_dim num_headsr is_decoderrT is_causalr@cst||_||_||_|||_||_|j||jkr*td|jd|d|jd|_||_ ||_ t j |||d|_ t j |||d|_t j |||d|_t j |||d|_dS)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r)rT)r'r(rrrhead_dimr@rvrrrr)rk_projv_projq_projout_proj)r?rrrrrTrr@rArCrDr(s&    zHubertAttention.__init__rGkey_value_statesrlayer_head_maskoutput_attentionsrreturncKs|du}|jdd\}} |r|jdn| } || d|jf} || d|jf} ||j| dd} |r4|n|}||j| dd}||j| dd}t}|jj dkr\t |jj }||| |||f|j shdn|j |j ||d|\}}||| d}||}||dfS)z#Input shape: Batch x Time x ChannelNrdrreagerr)rrrr)shaperrrrErrrr@_attn_implementationrrrrreshaperr)r?rGrrrrris_cross_attentionbsztgt_lensrc_len q_input_shapekv_input_shape query_statescurrent_states key_states value_statesattention_interfacerrrCrCrDrH/s:      zHubertAttention.forward)rFTFN)NNNF)rJrKrLrintfloatboolrrr(rTensorrr tuplerHrMrCrCrArDr sR"  rcr)HubertFeedForwardcsptt|j|_t|j|j|_ t |j t r"t |j |_n|j |_t|j|j|_t|j|_dSr])r'r(r)ractivation_dropoutintermediate_dropoutrr+intermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrrArCrDr(fs  zHubertFeedForward.__init__cCs6||}||}||}||}||}|Sr])rrrrrrFrCrCrDrHss     zHubertFeedForward.forwardrIrCrCrArDres  rcs&eZdZfddZdddZZS)HubertEncoderLayercshtt|j|j|jd|d|_t|j |_ tj |j|j d|_ t||_tj |j|j d|_dS)NFrrrrr@r)r'r(rr+num_attention_headsattention_dropout attentionr)rrrrarrbr feed_forwardfinal_layer_normrrArCrDr(~s  zHubertEncoderLayer.__init__NFcCsf|}|j|||d\}}}||}||}||}|||}||}|f}|r1||f7}|SNrr)rrrbrrr?rGrr attn_residualr_outputsrCrCrDrHs    zHubertEncoderLayer.forwardr{rIrCrCrArDr}s rc sjeZdZfddZ    ddejdeejded ed ef d d Z de ejdfd ejfddZ Z S) HubertEncodercft|_t|_tjjjd|_ t j |_ t fddtjD|_d|_dS)Nrcg|]}tqSrC)rrnrrprCrDrqz*HubertEncoder.__init__..Fr'r(r@rpos_conv_embedr)rar+rrbrrrrwrtnum_hidden_layerslayersryrrArprDr(s    zHubertEncoder.__init__NFTrGrroutput_hidden_states return_dictcCs*|rdnd}|r dnd}|dur"|ddd|jd}d||<|||}||} || }||}||}tpAt|} |j D]3} |rN||f}t g} |j o[| |j jk} | r`| rk| |||d}|d}| rod}|rx||df}qE|r||f}|stdd |||fDSt|||d S) NrCrdrrrrNNcs|] }|dur|VqdSr]rCrnvrCrCrD z(HubertEncoder.forward..last_hidden_staterG attentions) unsqueezerepeatr_update_full_maskrrbrrr rrrandrr@ layerdroprrr?rGrrrrall_hidden_statesall_self_attentionsexpand_attention_maskposition_embeddings synced_gpusrrdropout_probabilityskip_the_layer layer_outputsrCrCrDrHsL          zHubertEncoder.forward inputs_embedscC|dur>|jjdkrd|vr|}|Sd}|S|jjdkr$t||j}|S|jjdkr8t|tjr6t|dd}|St||j}|SNflash_attention_2rsdpaflex_attentionF)r r@rr dtyperrrrr r?rrrCrCrDr       zHubertEncoder._update_full_maskNFFT) rJrKrLr(rtensorrrrrHrrrMrCrCrArDrs,   < rcs,eZdZfddZdejfddZZS)HubertAttnAdapterLayercsZt|j|_|j|_t|j|_t |j|j|_ t |_ t |j|j|_ dS)z Implements adapter modules directly with 3D tensor weight as parameters and without using ModuleList to speed up training throughput. N)r'r(adapter_attn_dim input_dimr+ hidden_dimr)ranormrlinear_1ReLUact_fnlinear_2rrArCrDr(s  zHubertAttnAdapterLayer.__init__rGcCs,||}||}||}||}|Sr])rrrrrFrCrCrDrH s    zHubertAttnAdapterLayer.forward)rJrKrLr(r FloatTensorrHrMrCrCrArDrs rcs@eZdZfddZ  d dejdeejdefdd ZZ S) !HubertEncoderLayerStableLayerNormcstt|j|j|jd|d|_t|j |_ tj |j|j d|_ t||_tj |j|j d|_t|dddurAt||_dSd|_dS)NFrrr)r'r(rr+rrrr)rrrrarrbrrrgetattrr adapter_layerrrArCrDr(s   z*HubertEncoderLayerStableLayerNorm.__init__NFrGrrcCsz|}||}|j|||d\}}}||}||}||||}|jdur1|||}|f}|r;||f7}|Sr)rbrrrrrrrCrCrDrH+s    z)HubertEncoderLayerStableLayerNorm.forwardr{) rJrKrLr(rrrrrHrMrCrCrArDrs rcsLeZdZfddZ    d ddZdeejdfd ejfd d ZZ S) HubertEncoderStableLayerNormcr)NrcrrC)rrrprCrDrqMrz9HubertEncoderStableLayerNorm.__init__..FrrrArprDr(Fs   z%HubertEncoderStableLayerNorm.__init__NFTcCs*|rdnd}|r dnd}|dur"|ddd|jd}d||<|||}||} || }||}tp.r)rrrrrrrr rrrrr@rrbrrrrCrCrDrHQsL          z$HubertEncoderStableLayerNorm.forwardrrcCrrr r rCrCrDrr z.HubertEncoderStableLayerNorm._update_full_maskr ) rJrKrLr(rHrrrrrMrCrCrArDrEs  > rc@sbeZdZUeed<dZdZdZdZdZ dZ ddZ de e jeffdd Zd ed e jfd d ZdS)HubertPreTrainedModelr@hubertrTcCst|tjr |jjjd|jjd|jdur|jj dSdSt|tj tj tj fr:|jj |jj ddSt|tjrtrddl}t|drvt|drv|jj|j|jgddtj|jjWdn1spwYn*|jj|jddtj|jjWdn1swYntj|jj|jdur|jj dSdSt|trt|d r|jjdSdSt|trt|d r|jj d|jjd dSdSdS) zInitialize the weightsr)meanstdNg?rr:r9r!masked_spec_embed layer_weightsr)rr)rr#datanormal_r@initializer_rangerTzero_rarir1fill_r*rr4r3r5r6r:r9initkaiming_normal_ HubertModelr!uniform_HubertForSequenceClassificationr"r)r?rr4rCrCrD _init_weightssB         z#HubertPreTrainedModel._init_weights input_lengthscCs4dd}t|jj|jjD] \}}||||}q |S)zH Computes the output length of the convolutional layers cSstj|||dddS)Nfloor) rounding_moder)rdiv) input_lengthrrSrCrCrD_conv_out_lengthszPHubertPreTrainedModel._get_feat_extract_output_lengths.._conv_out_length)zipr@rXrY)r?r.r3rrSrCrCrD _get_feat_extract_output_lengthssz6HubertPreTrainedModel._get_feat_extract_output_lengthsfeature_vector_lengthrcCs~||dtj}|jd}tj||f|j|jd}d|tj |jd|jd|df<| dg d dg }|S)Nrdr)r devicer)r7) r5sumtorlongrzerosr r7arangeflipcumsumr)r?r6routput_lengths batch_sizerCrCrD"_get_feature_vector_attention_masks "z8HubertPreTrainedModel._get_feature_vector_attention_maskN)rJrKrLr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnr-rr LongTensorrr5rArCrCrCrDrs !rr mask_prob mask_length min_masksrcs|\}dkr tdkrtdddtjdfdd}|dur:|d n fd d t|D}tj |ft d }g} |} | d krZ|S|D];} || } tjj t | d| dd} t | d kr}d}n| d }t| tj| | tjd |g} | | q\t| } t| dddddf|| f} | || } t ddddf}t||| f|| }| |} | dkrd| | dk<t|| dd |S)an Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. rz&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `csXt|}t|}|kr}|d|kr*t|dd}|S)z;Given input length, compute how many spans should be maskedrr)rmax)r2num_masked_spanepsilonrKrJrLsequence_lengthrCrDcompute_num_masked_spans  z6_compute_mask_indices..compute_num_masked_spanNrdcsg|]}qSrCrCr)rRrCrDrq&sz)_compute_mask_indices..r rF)replace)rvnprandomritemdetachr8tolistrtr;rchoicer<len concatenateonesint32appendarray broadcast_torrNput_along_axis)rrJrKrrLr@rSr. spec_aug_maskspec_aug_mask_idxsmax_num_masked_spanr2rOspec_aug_mask_idxdummy_mask_idxoffsetsrCrPrD_compute_mask_indicess\    rjcseZdZdeffdd Z  ddejdeejdeejfdd Z e     dd eej deej deejd ee d ee d ee de eeffddZZS)r*r@cszt|||_t||_t||_|jdks|jdkr)t t |j |_|jr2t||_nt||_|dS)Nr)r'r(r@rjfeature_extractorrfeature_projectionmask_time_probmask_feature_probr) Parameterrrr+r+r!do_stable_layer_normrencoderr post_initrrArCrDr(fs      zHubertModel.__init__NrGmask_time_indicesrcCst|jdds |S|\}}}|dur|j|j||<n-|jjdkrK|jrKt||f|jj|jj ||jj d}t j ||j t jd}|j|j||<|jjdkr|jrt||f|jj|jj|jjd}t j ||j t jd}|dddfd|d}d||<|S) z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentTNr)rJrKrrL)r7r )rJrKrLrd)rr@rr!r9r rmrrjmask_time_lengthmask_time_min_masksrrr7rrnmask_feature_lengthmask_feature_min_masksexpand)r?rGrsrr@rRr+mask_feature_indicesrCrCrD_mask_hidden_statesxs4 zHubertModel._mask_hidden_statesrrrrrc Cs|dur|n|jj}|dur|n|jj}|dur|n|jj}||}|dd}|dur6||jd|}||}|j ||d}|j |||||d} | d}|s[|f| ddSt || j | j dS)a1 mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```Nrr)rsrrrrrr)r@rruse_return_dictrkrErArrlr{rqrrGr) r?rrrsrrrextract_featuresrGencoder_outputsrCrCrDrHs2$   zHubertModel.forwardrNNNNN)rJrKrLrr(rrrrIr{rrrrrrrHrMrCrCrArDr*ds@ . r*zn Hubert Model with a `language modeling` head on top for Connectionist Temporal Classification (CTC). ) custom_introcseZdZddeeffdd ZddZddZd d Zd d Z e     dd ee j dee j dee dee dee dee j deeeffddZZS) HubertForCTCN target_langcs~t|t||_t|j|_||_|j dur#t d|j dt |dr.|j r.|jn|j}t||j |_|dS)a0 target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`HubertForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `HubertForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration. add_adapter)r'r(r*rr)r final_dropoutrr vocab_sizervrBr3routput_hidden_sizer+rlm_headrr)r?r@rrrArCrDr(s     zHubertForCTC.__init__cCsv|j}|durt|jdddurtd|d|dur,t|jdddur,tddS|dur9|j|dddSdS)a' This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. NrzCannot pass `target_lang`: z- if `config.adapter_attn_dim` is not defined.z)By default `target_lang` is set to 'eng'.T) force_load)rrr@rvloggerinfo load_adapter)r?rrCrCrD tie_weightss zHubertForCTC.tie_weightscCtdt|dS) Calling this function will disable the gradient computation for the feature encoder so that its parameter will not be updated during training. The method `freeze_feature_extractor` is deprecated and will be removed in Transformers v5. Please use the equivalent `freeze_feature_encoder` method instead.Nwarningswarn FutureWarningfreeze_feature_encoderr?rCrCrDfreeze_feature_extractor)  z%HubertForCTC.freeze_feature_extractorcC|jjdSrNrrkrrrCrCrDr5z#HubertForCTC.freeze_feature_encodercC|jD]}d|_qdSz Calling this function will disable the gradient computation for the base model so that its parameters will not be updated during training. Only the classification head will be updated. FNrr|r}r~rCrCrDfreeze_base_model<zHubertForCTC.freeze_base_modelrrrrrlabelsrc Cs||dur|n|jj}|dur||jjkrtd|jj|j|||||d}|d}||}||} d} |dur|durC|ntj |tj d}| | d tj } |dk} | d} || }tjj| dtjddd}tjjjd d tjj||| | |jj|jj|jjd } Wdn1swY|s| f|td}| dur| f|S|St| | |j|jd S) a labels (`torch.LongTensor` of shape `(batch_size, target_length)`, *optional*): Labels for connectionist temporal classification. Note that `target_length` has to be smaller or equal to the sequence length of the output logits. Indices are selected in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size - 1]`. Nz$Label values must be <= vocab_size: r|rrTrd)r%r rF)enabled)blank reduction zero_infinitylosslogitsrGr)r@r}rNrrvrrrr ones_liker:r5r8r9 masked_selectr)r log_softmaxfloat32rEbackendscudnnflagsctc_loss pad_token_idctc_loss_reductionctc_zero_infinity_HIDDEN_STATES_START_POSITIONrrGr)r?rrrrrrrrGrrr. labels_masktarget_lengthsflattened_targets log_probsoutputrCrCrDrHDsN      zHubertForCTC.forwardr]r)rJrKrLrrr(rrrrrrrrrrrrHrMrCrCrArDrs6  rz Hubert Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. cseZdZfddZddZddZddZe dd ee j d ee j d ee d ee dee dee j de e effddZZS)r,cst|t|dr|jrtdt||_|jd}|jr*t t |||_ t |j|j|_t |j|j|_|dS)Nrz]Sequence classification does not support the use of Hubert adapters (config.add_adapter=True)r)r'r(r3rrvr*rruse_weighted_layer_sumr)rorr^r"rr+classifier_proj_size projector num_labels classifierrr)r?r@ num_layersrArCrDr(s    z(HubertForSequenceClassification.__init__cCr)z Calling this function will disable the gradient computation for the feature encoder so that its parameters will not be updated during training. rNrrrCrCrDrrz8HubertForSequenceClassification.freeze_feature_extractorcCrrrrrCrCrDrrz6HubertForSequenceClassification.freeze_feature_encodercCrrrr~rCrCrDrrz1HubertForSequenceClassification.freeze_base_modelNrrrrrrrcCsz|dur|n|jj}|jjrdn|}|j|||||d}|jjrB|t}tj|dd}tjj |j dd} || dddj dd}n|d}| |}|durV|jdd} n+||jd|} | ddd|jd} d || <|j dd| j dd dd} || } d}|durt}|| d|jj| d}|s| f|td}|dur|f|S|St|| |j|jd S) a  input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`HubertProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NTr|rrrdrrrr)r@r}rrrrstackr)rrr"rr8rrrArrrrrrrrGr)r?rrrrrrrrG norm_weights pooled_output padding_maskexpand_padding_maskrrloss_fctrrCrCrDrHsH    z'HubertForSequenceClassification.forwardr)rJrKrLr(rrrrrrrrrrrrHrMrCrCrArDr,s4   r,)rr,r*r)NrNrO)IrtypingrrrnumpyrVrtorch.nnr)r activationsrintegrations.deepspeedrintegrations.fsdpr modeling_attn_mask_utilsr r modeling_flash_attention_utilsr modeling_layersr modeling_outputsrrrmodeling_utilsrrprocessing_utilsrr2rrrconfiguration_hubertrintegrations.flex_attentionr get_loggerrJrModulerr<rQr_rerjrrrrrrrrrrrrrrrIndarrayrjr*rrr,__all__rCrCrCrDs           2 & X$].aJ  w  s