o ei@sddlmZddlZddlZddlmZddlmZddlm Z ddl m Z ddl mZddlmZdd lmZdd lmZdd lmZdd lmZmZmZdd lmZmZmZddlm Z ddl!m"Z"m#Z#m$Z$ddl%m&Z&e$'e(Z)Gdddej*Z+Gdddej*Z,GdddeZ-GdddeZ.GdddeZ/Gdddej*Z0Gdddej*Z1  dLd!ej*d"ej2d#ej2d$ej2d%ej2dBd&e3dBd'e3d(e e"fd)d*Z4Gd+d,d,ej*Z5Gd-d.d.ej*Z6Gd/d0d0eZ7Gd1d2d2ej*Z8Gd3d4d4ej*Z9Gd5d6d6eZ:Gd7d8d8ej*Z;e#Gd9d:d:eZ<  dMd;e=e>e>fdd%ej?dBd>e>d?ej@f d@dAZAe#GdBdCdCe<ZBdZCe#dDdEGdFdGdGe<ZDe#dHdEGdIdJdJe<ZEgdKZFdS)N)CallableN)CrossEntropyLoss)initialization)ACT2FN)is_deepspeed_zero3_enabled)is_fsdp_managed_module)create_bidirectional_mask)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputCausalLMOutputSequenceClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel*get_torch_context_manager_or_global_device)Unpack)TransformersKwargsauto_docstringlogging) HubertConfigc$eZdZfddZddZZS)HubertPositionalConvEmbeddingcs@ttj|j|j|j|jd|jd|_d|_|j r%t |j|_nmtj j }t tj jdr5tj jj }trddl}|jj|jjdd||jddd|_Wdn1sZwYt |jdrr|jjjj}|jjjj}n|jj}|jj}|j|||j||n ||jddd|_t|j|_t|j|_dS) N) kernel_sizepaddinggroups weight_normr modifier_rankweight)namedimparametrizations)super__init__nnConv1d hidden_sizenum_conv_pos_embeddingsnum_conv_pos_embedding_groupsconv batch_normconv_pos_batch_norm BatchNorm1dutilsrhasattrr$r deepspeedzeroGatheredParametersr! original0 original1weight_gweight_vregister_external_parameterHubertSamePadLayerrrfeat_extract_activation activation)selfconfigrr2r7r8 __class__h/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/hubert/modeling_hubert.pyr&.s:     z&HubertPositionalConvEmbedding.__init__cCsN|dd}|jdur||}||}||}||}|dd}|S)Nrr) transposer-r,rr<r= hidden_statesrArArBforwardSs       z%HubertPositionalConvEmbedding.forward__name__ __module__ __qualname__r&rF __classcell__rArAr?rBr-s %rcr)r:cs*t|ddkrd|_dSd|_dS)Nrrr)r%r&num_pad_remove)r=r*r?rArBr&`s  zHubertSamePadLayer.__init__cCs,|jdkr|ddddd|j f}|SNr)rLrDrArArBrFds zHubertSamePadLayer.forwardrGrArAr?rBr:_s r:c&eZdZdfdd ZddZZS)HubertNoLayerNormConvLayerrcsjt|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ t |j |_ dS)Nrrrstridebias)r%r&conv_dim in_conv_dim out_conv_dimr'r( conv_kernel conv_stride conv_biasr,rr;r<r=r>layer_idr?rArBr&ks  z#HubertNoLayerNormConvLayer.__init__cCs||}||}|SN)r,r<rDrArArBrFys  z"HubertNoLayerNormConvLayer.forwardrrGrArAr?rBrOjsrOcrN)HubertLayerNormConvLayerrcs|t|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ tj |jdd|_ t |j|_dS)NrrrPT)elementwise_affine)r%r&rSrTrUr'r(rVrWrXr, LayerNorm layer_normrr;r<rYr?rArBr&s  z!HubertLayerNormConvLayer.__init__cCs:||}|dd}||}|dd}||}|S)N)r,rCr`r<rDrArArBrFs     z HubertLayerNormConvLayer.forwardr\rGrArAr?rBr]sr]crN)HubertGroupNormConvLayerrcst|dkr|j|dnd|_|j||_tj|j|j|j||j||j d|_ t |j |_ tj|j|jdd|_dS)NrrrPT) num_groups num_channelsaffine)r%r&rSrTrUr'r(rVrWrXr,rr;r< GroupNormr`rYr?rArBr&s   z!HubertGroupNormConvLayer.__init__cCs"||}||}||}|Sr[)r,r`r<rDrArArBrFs   z HubertGroupNormConvLayer.forwardr\rGrArAr?rBrcsrccs0eZdZdZfddZddZddZZS)HubertFeatureEncoderz.Construct the features from raw audio waveformcstjdkr tddgfddtjdD}njdkr2fddtjD}n td jd t||_ d |_ d |_ dS) NgrouprrZcsg|] }t|ddqS)rrj)rO.0ir>rArB sz1HubertFeatureEncoder.__init__..rlayercsg|]}t|dqS)rj)r]rkrnrArBrosz`config.feat_extract_norm` is z), but has to be one of ['group', 'layer']FT) r%r&feat_extract_normrcrangenum_feat_extract_layers ValueErrorr' ModuleList conv_layersgradient_checkpointing_requires_grad)r=r>rvr?rnrBr&s       zHubertFeatureEncoder.__init__cCs|D]}d|_qd|_dSNF) parameters requires_gradrxr=paramrArArB_freeze_parameterss  z'HubertFeatureEncoder._freeze_parameterscCs:|dddf}|jr|jrd|_|jD]}||}q|S)NT)rxtrainingr{rv)r= input_valuesrE conv_layerrArArBrFs    zHubertFeatureEncoder.forward)rHrIrJ__doc__r&r~rFrKrArAr?rBrhs  rhcr)HubertFeatureProjectioncsXt|j|_|jrtj|jd|jd|_t|jd|j |_ t |j |_ dS)Nrbeps)r%r&feat_proj_layer_normr'r_rSlayer_norm_epsr`Linearr) projectionDropoutfeat_proj_dropoutdropoutr=r>r?rArBr&s z HubertFeatureProjection.__init__cCs(|jr||}||}||}|Sr[)rr`rrrDrArArBrFs    zHubertFeatureProjection.forwardrGrArAr?rBrs rmodulequerykeyvalueattention_maskscalingrkwargsc Ks|dur |dd}t||dd|}|dur||}tjj|dd}tjj|||jd}t||} | dd } | |fS)Nrbrrr#)prr) sizetorchmatmulrCr' functionalsoftmaxrr contiguous) rrrrrrrr attn_weights attn_outputrArArBeager_attention_forwards  rcseZdZdZ     ddededed ed ed ed edBffd d Z   dde j de j dBde j dBdedBde e de e j e j dBe e j dBff ddZZS)HubertAttentionz=Multi-headed attention from 'Attention Is All You Need' paperrFTN embed_dim num_headsr is_decoderrR is_causalr>cst||_||_||_|||_||_|j||jkr*td|jd|d|jd|_||_ ||_ t j |||d|_ t j |||d|_t j |||d|_t j |||d|_dS)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).r)rR)r%r&rrrhead_dimr>rtrrrr'rk_projv_projq_projout_proj)r=rrrrrRrr>r?rArBr& s&    zHubertAttention.__init__rEkey_value_statesroutput_attentionsrreturncKs|du}|jdd\}}|r|jdn|} ||d|jf} || d|jf} ||j| dd} |r4|n|} || j| dd}|| j| dd}t|j j t }||| |||f|j sbdn|j |j|d|\}}|||d}||}||dfS)z#Input shape: Batch x Time x ChannelNrbrrr)rrr)shaperrviewrCrrr get_interfacer>_attn_implementationrrrrreshaperr)r=rErrrris_cross_attentionbsztgt_lensrc_len q_input_shapekv_input_shape query_statescurrent_states key_states value_statesattention_interfacerrrArArBrF(s8    zHubertAttention.forward)rFTFN)NNF)rHrIrJrintfloatboolrr&rTensorrr tuplerFrKrArAr?rBrsL" rcr)HubertFeedForwardcsptt|j|_t|j|j|_ t |j t r"t |j |_n|j |_t|j|j|_t|j|_dSr[)r%r&r'ractivation_dropoutintermediate_dropoutrr)intermediate_sizeintermediate_dense isinstance hidden_actstrrintermediate_act_fn output_densehidden_dropoutoutput_dropoutrr?rArBr&]s  zHubertFeedForward.__init__cCs6||}||}||}||}||}|Sr[)rrrrrrDrArArBrFjs     zHubertFeedForward.forwardrGrArAr?rBr\s  rcs&eZdZfddZdddZZS)HubertEncoderLayercshtt|j|j|jd|d|_t|j |_ tj |j|j d|_ t||_tj |j|j d|_dS)NFrrrrr>r)r%r&rr)num_attention_headsattention_dropout attentionr'rrrr_rr`r feed_forwardfinal_layer_normrr?rArBr&us  zHubertEncoderLayer.__init__NFcCsf|}|j|||d\}}}||}||}||}|||}||}|f}|r1||f7}|SNrr)rrr`rrr=rErr attn_residualr_outputsrArArBrFs    zHubertEncoderLayer.forwardryrGrArAr?rBrts rc sLeZdZfddZ    d dejdejdBded ed ef d d ZZ S) HubertEncodercft|_t|_tjjjd|_ t j |_ t fddtjD|_d|_dS)Nrcg|]}tqSrA)rrlrrnrArBroz*HubertEncoder.__init__..Fr%r&r>rpos_conv_embedr'r_r)rr`rrrrurrnum_hidden_layerslayersrwrr?rnrBr&s    zHubertEncoder.__init__NFTrErroutput_hidden_states return_dictcCs.|rdnd}|r dnd}|dur"|ddd|jd}d||<t|j||d}||} || }||}||}tpCt |} |j D]3} |rP||f}t g} |j o]| |jjk} | rb| rm| |||d}|d}| rqd}|rz||df}qG|r||f}|std d |||fDSt|||d S) NrArbrrrr> inputs_embedsrrNNcs|] }|dur|VqdSr[rArlvrArArB z(HubertEncoder.forward..last_hidden_staterE attentions) unsqueezerepeatrr r>rr`rrrrrrandr layerdroprr r=rErrrrall_hidden_statesall_self_attentionsexpand_attention_maskposition_embeddings synced_gpusrpdropout_probabilityskip_the_layer layer_outputsrArArBrFsN          zHubertEncoder.forwardNFFT) rHrIrJr&rtensorrrrFrKrArAr?rBrs"  rcs,eZdZfddZdejfddZZS)HubertAttnAdapterLayercsZt|j|_|j|_t|j|_t |j|j|_ t |_ t |j|j|_ dS)z Implements adapter modules directly with 3D tensor weight as parameters and without using ModuleList to speed up training throughput. N)r%r&adapter_attn_dim input_dimr) hidden_dimr'r_normrlinear_1ReLUact_fnlinear_2rr?rArBr&s  zHubertAttnAdapterLayer.__init__rEcCs,||}||}||}||}|Sr[)rrr r rDrArArBrFs    zHubertAttnAdapterLayer.forward)rHrIrJr&r FloatTensorrFrKrArAr?rBrs rcs@eZdZfddZ  d dejdejdBdefdd ZZS) !HubertEncoderLayerStableLayerNormcstt|j|j|jd|d|_t|j |_ tj |j|j d|_ t||_tj |j|j d|_t|dddurAt||_dSd|_dS)NFrrr)r%r&rr)rrrr'rrrr_rr`rrrgetattrr adapter_layerrr?rArBr&s   z*HubertEncoderLayerStableLayerNorm.__init__NFrErrcCsz|}||}|j|||d\}}}||}||}||||}|jdur1|||}|f}|r;||f7}|Sr)r`rrrrrrrArArBrF s    z)HubertEncoderLayerStableLayerNorm.forwardry) rHrIrJr&rrrrFrKrArAr?rBr s r cs.eZdZfddZ    dddZZS) HubertEncoderStableLayerNormcr)NrcrrA)r rrnrArBro/rz9HubertEncoderStableLayerNorm.__init__..Frrr?rnrBr&(s   z%HubertEncoderStableLayerNorm.__init__NFTcCs.|rdnd}|r dnd}|dur"|ddd|jd}d||<t|j||d}||} || }||}tp>t|} |j D]3} |rK||f}t g} |j oX| |jj k} | r]| rh| |||d}|d}| rld}|ru||df}qB||}|r||f}|std d |||fDSt|||d S) NrArbrrrrrrcsrr[rArrArArBrkrz7HubertEncoderStableLayerNorm.forward..r)rrrr r>rrrrrrrrrr`rr rrArArBrF3sN          z$HubertEncoderStableLayerNorm.forwardrrGrArAr?rBr's rc@sjeZdZUeed<dZdZdZdZdZ dZ dZ e ddZde jeBfd d Zd ed e jfd dZdS)HubertPreTrainedModelr>hubertraudioTcCst|tjr tj|jd|jjd|jdurt |jdSdSt|tj tj tj frUt |jt |jt|dddurSt |jt |jt |jdSdSt|tjrtrddl}t|drt|dr|jj|j|jgddt|jWdn1swYn&|jj|jddt|jWdn1swYnt|j|jdurt |jdSdSt|trt|d rt|jdSdSt|trt|d rt|j d |jj!d dSdSdS) zInitialize the weightsr)meanstdN running_meanrr8r7rmasked_spec_embed layer_weightsg?r)"rr'rinitnormal_r!r>initializer_rangerRzeros_r_rgr/ones_r r running_varnum_batches_trackedr(rr2r1r3r4r8r7kaiming_normal_ HubertModeluniform_rHubertForSequenceClassification constant_rr)r=rr2rArArB _init_weights~sL             z#HubertPreTrainedModel._init_weights input_lengthscCs4dd}t|jj|jjD] \}}||||}q |S)zH Computes the output length of the convolutional layers cSstj|||dddS)Nfloor) rounding_moder)rdiv) input_lengthrrQrArArB_conv_out_lengthszPHubertPreTrainedModel._get_feat_extract_output_lengths.._conv_out_length)zipr>rVrW)r=r%r*rrQrArArB _get_feat_extract_output_lengthssz6HubertPreTrainedModel._get_feat_extract_output_lengthsfeature_vector_lengthrcCs~||dtj}|jd}tj||f|j|jd}d|tj |jd|jd|df<| dg d dg }|S)Nrbr)dtypedevicer)r/) r,sumtorlongrzerosr.r/arangeflipcumsumr)r=r-routput_lengths batch_sizerArArB"_get_feature_vector_attention_masks "z8HubertPreTrainedModel._get_feature_vector_attention_maskN)rHrIrJr__annotations__base_model_prefixmain_input_nameinput_modalitiessupports_gradient_checkpointing_supports_flash_attn_supports_sdpa_supports_flex_attnrno_gradr$ LongTensorrr,r9rArArArBrss  #rr mask_prob mask_length min_masksrcs|\}dkr tdkrtdddtjdfdd}|dur:|d n fd d t|D}tj |ft d }g} |} | d krZ|S|D];} || } tjj t | d| dd} t | d kr}d}n| d }t| tj| | tjd |g} | | q\t| } t| dddddf|| f} | || } t ddddf}t||| f|| }| |} | dkrd| | dk<t|| dd |S)an Computes random mask spans for a given shape. Used to implement [SpecAugment: A Simple Data Augmentation Method for ASR](https://huggingface.co/papers/1904.08779). Note that this method is not optimized to run on TPU and should be run on CPU as part of the preprocessing during training. Args: shape: The shape for which to compute masks. This should be of a tuple of size 2 where the first element is the batch size and the second element is the length of the axis to span. mask_prob: The percentage of the whole axis (between 0 and 1) which will be masked. The number of independently generated mask spans of length `mask_length` is computed by `mask_prob*shape[1]/mask_length`. Note that due to overlaps, `mask_prob` is an upper bound and the actual percentage will be smaller. mask_length: size of the mask min_masks: minimum number of masked spans attention_mask: A (right-padded) attention mask which independently shortens the feature axis of each batch dimension. rz&`mask_length` has to be bigger than 0.zO`mask_length` has to be smaller than `sequence_length`, but got `mask_length`: z and `sequence_length`: `csXt|}t|}|kr}|d|kr*t|dd}|S)z;Given input length, compute how many spans should be maskedrr)rmax)r)num_masked_spanepsilonrErDrFsequence_lengthrArBcompute_num_masked_spans  z6_compute_mask_indices..compute_num_masked_spanNrbcsg|]}qSrArAr)rLrArBrosz)_compute_mask_indices..r.rF)replace)rtnprandomritemdetachr0tolistrrr3rchoicer4len concatenateonesint32appendarray broadcast_torrHput_along_axis)rrDrErrFr8rMr% spec_aug_maskspec_aug_mask_idxsmax_num_masked_spanr)rIspec_aug_mask_idxdummy_mask_idxoffsetsrArJrB_compute_mask_indicess\    rdcseZdZdeffdd Z  ddejdejdBdejdBfdd Ze     dd ej dBdej dBdejdBd e dBd e dBd e dBde e BfddZZS)r r>cszt|||_t||_t||_|jdks|jdkr)t t |j |_|jr2t||_nt||_|dSNr)r%r&r>rhfeature_extractorrfeature_projectionmask_time_probmask_feature_probr' Parameterrrr)r!rdo_stable_layer_normrencoderr post_initrr?rArBr&7s      zHubertModel.__init__NrEmask_time_indicesrcCst|jdds |S|\}}}|dur|j|j||<n-|jjdkrK|jrKt||f|jj|jj ||jj d}t j ||j t jd}|j|j||<|jjdkr|jrt||f|jj|jj|jjd}t j ||j t jd}|dddfd|d}d||<|S) z Masks extracted features along time axis and/or along feature axis according to [SpecAugment](https://huggingface.co/papers/1904.08779). apply_spec_augmentTNr)rDrErrF)r/r.)rDrErFrb)r r>rrr1r.rhrrdmask_time_lengthmask_time_min_masksrrr/rrimask_feature_lengthmask_feature_min_masksexpand)r=rErnrr8rLr)mask_feature_indicesrArArB_mask_hidden_statesIs4 zHubertModel._mask_hidden_statesrrrrrc Ks|dur|n|jj}|dur|n|jj}|dur|n|jj}||}|dd}|dur6||jd|}||} |j | |d} |j | ||||d} | d} |s[| f| ddSt | | j | j dS)a1 mask_time_indices (`torch.BoolTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices to mask extracted features for contrastive loss. When in training mode, model learns to predict masked extracted features in *config.proj_codevector_dim* space. Example: ```python >>> from transformers import AutoProcessor, HubertModel >>> from datasets import load_dataset >>> processor = AutoProcessor.from_pretrained("facebook/hubert-large-ls960-ft") >>> model = HubertModel.from_pretrained("facebook/hubert-large-ls960-ft") >>> def map_to_array(example): ... example["speech"] = example["audio"]["array"] ... return example >>> ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation") >>> ds = ds.map(map_to_array) >>> input_values = processor(ds["speech"][0], return_tensors="pt").input_values # Batch size 1 >>> hidden_states = model(input_values).last_hidden_state ```Nrr)rnrrrrrr)r>rruse_return_dictrfrCr9rrgrvrlr rEr) r=rrrnrrrrextract_featuresrEencoder_outputsrArArBrFws2%   zHubertModel.forwardrNNNNN)rHrIrJrr&rr rCrvrrrrr rFrKrArAr?rBr 5s@ . r zn Hubert Model with a `language modeling` head on top for Connectionist Temporal Classification (CTC). ) custom_introcseZdZddedBffdd ZddZddZd d Ze     dd e j dBd e j dBd e dBde dBde dBde j dBde e BfddZZS) HubertForCTCN target_langcs~t|t||_t|j|_||_|j dur#t d|j dt |dr.|j r.|jn|j}t||j |_|dS)a0 target_lang (`str`, *optional*): Language id of adapter weights. Adapter weights are stored in the format adapter..safetensors or adapter..bin. Only relevant when using an instance of [`HubertForCTC`] with adapters. Uses 'eng' by default. NzYou are trying to instantiate z with a configuration that does not define the vocabulary size of the language model head. Please instantiate the model as follows: `HubertForCTC.from_pretrained(..., vocab_size=vocab_size)`. or define `vocab_size` of your model's configuration. add_adapter)r%r&r rr'r final_dropoutrr~ vocab_sizertr@r1routput_hidden_sizer)rlm_headrm)r=r>r~rr?rArBr&s     zHubertForCTC.__init__cKsttdkr dS|j}|dur"t|jdddur"td|d|dur6t|jdddur6tddS|durC|j |dddSdS) a' This method overwrites [`~PreTrainedModel.tie_weights`] so that adapter weights can be correctly loaded when passing `target_lang=...` to `from_pretrained(...)`. This method is **not** supposed to be called by the user and is prone to be changed in the future. metaNrzCannot pass `target_lang`: z- if `config.adapter_attn_dim` is not defined.z)By default `target_lang` is set to 'eng'.T) force_load) rrr/r~r r>rtloggerinfo load_adapter)r=rr~rArArB tie_weightsszHubertForCTC.tie_weightscC|jjdSz Calling this function will disable the gradient computation for the feature encoder so that its parameter will not be updated during training. Nrrfr~r=rArArBfreeze_feature_encoderz#HubertForCTC.freeze_feature_encodercC|jD]}d|_qdSz Calling this function will disable the gradient computation for the base model so that its parameters will not be updated during training. Only the classification head will be updated. FNrrzr{r|rArArBfreeze_base_modelzHubertForCTC.freeze_base_modelrrrrrlabelsrc Ks||dur|n|jj}|dur||jjkrtd|jj|j|||||d}|d} || } || } d} |dur|durC|ntj |tj d}| | d tj } |dk} | d}|| }tjj| dtjddd}tjjjd d tjj||| ||jj|jj|jjd } Wdn1swY|s| f|td}| dur| f|S|St| | |j|jd S) a labels (`torch.LongTensor` of shape `(batch_size, target_length)`, *optional*): Labels for connectionist temporal classification. Note that `target_length` has to be smaller or equal to the sequence length of the output logits. Indices are selected in `[-100, 0, ..., config.vocab_size - 1]`. All labels set to `-100` are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size - 1]`. Nz$Label values must be <= vocab_size: rwrrNrb)r#r.rF)enabled)blank reduction zero_infinitylosslogitsrEr)r>rxrHrrtrrrr ones_liker2r,r0r1 masked_selectr'r log_softmaxfloat32rCbackendscudnnflagsctc_loss pad_token_idctc_loss_reductionctc_zero_infinity_HIDDEN_STATES_START_POSITIONr rEr)r=rrrrrrrrrErrr% labels_masktarget_lengthsflattened_targets log_probsoutputrArArBrF sN      zHubertForCTC.forwardr[r{)rHrIrJrr&rrrrrrrrr rFrKrArAr?rBr}s4 r}z Hubert Model with a sequence classification head on top (a linear layer over the pooled output) for tasks like SUPERB Keyword Spotting. cseZdZfddZddZddZe     ddejdBd ejdBd e dBd e dBd e dBd ejdBde e BfddZ Z S)r"cst|t|dr|jrtdt||_|jd}|jr*t t |||_ t |j|j|_t |j|j|_|dS)Nrz]Sequence classification does not support the use of Hubert adapters (config.add_adapter=True)r)r%r&r1rrtr rruse_weighted_layer_sumr'rjrrXrrr)classifier_proj_size projector num_labels classifierrm)r=r> num_layersr?rArBr&]s    z(HubertForSequenceClassification.__init__cCrrrrrArArBrnrz6HubertForSequenceClassification.freeze_feature_encodercCrrrr|rArArBrurz1HubertForSequenceClassification.freeze_base_modelNrrrrrrrcKsz|dur|n|jj}|jjrdn|}|j|||||d}|jjrB|t} tj| dd} tjj |j dd} | | dddj dd} n|d} | | } |durV| jdd} n+|| jd|} | ddd| jd} d | | <| j dd| j dd dd} || }d}|durt}|| d|jj| d}|s|f|td}|dur|f|S|St|||j|jd S) a  input_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`): Float values of input raw speech waveform. Values can be obtained by loading a `.flac` or `.wav` audio file into an array of type `list[float]`, a `numpy.ndarray` or a `torch.Tensor`, *e.g.* via the torchcodec library (`pip install torchcodec`) or the soundfile library (`pip install soundfile`). To prepare the array into `input_values`, the [`AutoProcessor`] should be used for padding and conversion into a tensor of type `torch.FloatTensor`. See [`HubertProcessor.__call__`] for details. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). NTrwrrrbrrrr)r>rxrrrrstackr'rrrrr0rrr9rrrrrrrrEr)r=rrrrrrrrrE norm_weights pooled_output padding_maskexpand_padding_maskrrloss_fctrrArArBrF}sH    z'HubertForSequenceClassification.forwardr{)rHrIrJr&rrrrrrrrrFrKrArAr?rBr"Vs2  r")r}r"r rrerM)Gcollections.abcrnumpyrPrtorch.nnr'rrr activationsrintegrations.deepspeedrintegrations.fsdpr masking_utilsr modeling_flash_attention_utilsr modeling_layersr modeling_outputsr r rmodeling_utilsrrrprocessing_utilsrr0rrrconfiguration_hubertr get_loggerrHrModulerr:rOr]rcrhrrrrrrrrrr rrrrrCndarrayrdr rr}r"__all__rArArArBs             2 & V$H.LN  w  h