o i @sddlmZddlZddlmZeeZdejde dejfdd Z  dd ej j d ejd ejdejdeejde dee deedeejdffddZdS))OptionalN)logging hidden_statesn_repreturncCs^|j\}}}}|dkr |S|dddddddddf|||||}||||||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) N)shapeexpandreshape)rrbatchnum_key_value_headsslenhead_dimrk/home/ubuntu/maya3_transcribe/venv/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py repeat_kv s 0rmodulequerykeyvalueattention_maskdropoutscaling is_causalc Ks|dds|dddurtdt|dr$t||j}t||j}|dur@|jdkr@|ddddddd|jdf}|}|}|}|dura|jdd ko`|duo`t |d d }t j rpt |t jrp|}t jjj|||||||d } | d d} | dfS) Noutput_attentionsF head_maskz`sdpa` attention does not support `output_attentions=True` or `head_mask`. Please set your attention to `eager` if you want any of these features.num_key_value_groupsrrrT) attn_mask dropout_pscaler)getlogger warning_oncehasattrrrndimr contiguousgetattrtorchjit is_tracing isinstanceTensoritemnn functionalscaled_dot_product_attention transpose) rrrrrrrrkwargs attn_outputrrrsdpa_attention_forwards6    &" r7)rNN)typingrr+utilsr get_logger__name__r%r/intrr1Modulefloatbooltupler7rrrrs6