o if@s<dZddlmZmZmZddlZddlmZddlmZddl m Z m Z ddl m Z mZmZmZmZmZddlmZmZmZmZd d lmZd d lmZmZd d lmZd d lm Z d dl!m"Z"m#Z#m$Z$d dl%m&Z&d dl'm(Z(m)Z)ddl*m+Z+e)rddl,m-Z-ddl.m/Z/m0Z0ndZ-e(rddl1m2Z2m3Z3nd\Z3Z2e4e-e2e3fZ5e$6e7Z8GdddeddZ9Gddde Z GdddeZ:d6dd Z;Gd!d"d"e ZGd'd(d(ej?Z@Gd)d*d*eZAGd+d,d,eZBGd-d.d.e ZCe"Gd/d0d0eZDe"Gd1d2d2eDZEGd3d4d4eZFgd5ZGdS)7zPyTorch Bamba model.)Optional TypedDictUnionN)nn)ACT2FN) HybridMambaAttentionDynamicCacheJambaAttentionDecoderLayer)LlamaAttentionLlamaForCausalLMLlamaMLP LlamaRMSNormLlamaRotaryEmbedding rotate_half)MambaRMSNormGatedpad_tensor_by_sizereshape_into_chunks segment_sum)AttentionMaskConverter)BaseModelOutputWithPastCausalLMOutputWithPast)PreTrainedModel)Unpack)auto_docstringcan_return_tuplelogging)deprecate_kwarg)is_causal_conv1d_availableis_mamba_2_ssm_available) BambaConfig)selective_state_update)mamba_chunk_scan_combined mamba_split_conv1d_scan_combined)causal_conv1d_fncausal_conv1d_update)NNc@s@eZdZUdZejed<ejed<eed<eed<ejed<dS)BambaFlashAttentionKwargsa Keyword arguments for advanced Flash Attention, causal-conv1d, and mamba_ssm kernel usage. Use cases include padding-free training and fewer `torch.compile` graph breaks. Attributes: cu_seq_lens_q (`torch.LongTensor`) Gets cumulative sequence length for query state. cu_seq_lens_k (`torch.LongTensor`) Gets cumulative sequence length for key state. max_length_q (`int`): Maximum sequence length for query state. max_length_k (`int`): Maximum sequence length for key state. seq_idx (`torch.IntTensor): Index of each packed sequence. cu_seq_lens_q cu_seq_lens_k max_length_q max_length_kseq_idxN) __name__ __module__ __qualname____doc__torch LongTensor__annotations__int IntTensorr5r5d/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/transformers/models/bamba/modular_bamba.pyr&Ks   r&F)totalc@s&eZdZdZejdfdefddZdS)ra A dynamic cache that can handle both the attention cache (which has a seq_len dimension) and the mamba cache (which has a constant shape regardless of seq_len). This cache has two sets of lists of tensors: `key_cache` and `value_cache` for attention cache and `conv_states` and `ssm_states` for mamba cache. Each of these lists has `num_layers` tensors. The expected shape for each tensor For attention layers, `key_cache` and `value_cache` have a shape of `(batch_size, num_heads, seq_len, head_dim)`, while `conv_states` and `ssm_states` have a shape of `(batch_size, 0)` (empty tensors). For mamba layers, `key_cache` and `value_cache` have a shape of `(batch_size, 0)` (empty tensors), while `conv_states` represents the convolution state and has a shape of `(batch_size, d_inner, d_conv)`, and `ssm_states` represents the ssm state and has a shape of `(batch_size, d_inner, d_state)`. Nconfigc s0|j|_d|_|j}|j}g|_g|_g|_t|jD]^}|j|dkrS|jt j |j |j d|j |||dg7_|jt j |j|j||dg7_q|jt jggdg7_|jt jggdg7_|j|qfddt|jD|_fddt|jD|_dS) NFmambadevicedtyper<c g|] }tjggdqSr>r0tensor.0_ batch_sizer<r5r6  z=HybridMambaAttentionDynamicCache.__init__..cr?r@rArCrFr5r6rHrI)layers_block_typehas_previous_state mamba_d_conv mamba_d_state conv_states ssm_statestransformer_layersrangenum_hidden_layersr0zeros mamba_expand hidden_sizemamba_n_groups mamba_n_heads mamba_d_headrBappend key_cache value_cache)selfr8rGr=r<conv_kernel_sizessm_state_sizeir5rFr6__init__ssB   z)HybridMambaAttentionDynamicCache.__init__)r,r-r.r/r0float16r r`r5r5r5r6res rc@ eZdZdS)BambaRotaryEmbeddingNr,r-r.r5r5r5r6rcrcc Cs||}||}|jd}|dd|f|d|df}}|dd|f|d|df} } ||t||} | |t| |} tj| |gdd} tj| | gdd} | | fS)aApplies Rotary Position Embedding to the query and key tensors. Removes the interleaving of cos and sin from GLM Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. .Ndim) unsqueezeshaperr0cat) qkcossin position_ids unsqueeze_dim rotary_dimq_rotq_passk_rotk_passq_embedk_embedr5r5r6apply_rotary_pos_embs   ""ryc@rb)BambaAttentionNrdr5r5r5r6rzrerzc@rb)BambaRMSNormGatedNrdr5r5r5r6r{rer{cCsN|dur%|jddkr%|jddkr%|j}||dddddf|}|S)zm Tunes out the hidden states for padding tokens, see https://github.com/state-spaces/mamba/issues/66 Nrr)rjr=to) hidden_statesattention_maskr=r5r5r6apply_mask_to_padding_statess$ rc seZdZdZdedeffdd Z    ddejde e d e ej d e ejd e ej f d d Z   dde e d e ej d e ejfddZ    dde e d e ej d e ejd e ej fddZZS) BambaMixeruP Compute ∆, A, B, C, and D the state space parameters and compute the `contextualized_states`. A, D are input independent (see Mamba paper [1] Section 3.5.2 "Interpretation of A" for why A isn't selective) ∆, B, C are input-dependent (this is a key difference between Mamba and the linear time invariant S4, and is why Mamba is called **selective** state spaces) The are a few differences between this and Mamba2Mixer: - The variable use_precomputed_states is slightly different due to the hybrid cache structure - There's a few non-obvious bugs fixed with batching in the slow path that exist in main - Some extra variables that our layer doesn't need have been removed - We ported most of the refactors in https://github.com/huggingface/transformers/pull/35154, which is (as of Dec 18, 2024) unmerged r8 layer_idxcst|j|_|j|_|j|_|j|_t |j |j|_ ||_ |j |_|j|_t|j|_|j|_|j|_|j|_|j|_|j|_dtdf|_d|_d|_ |j d|j|j|_!t"j#|j!|j!|j |j|j!|jdd|_$|j |j!|j}t"j%|j||jd|_&t"'t()|j|_*t(+d|jd}t"'t(,||_-t.|j |jd |_/t"'t()|j|_0t"j%|j |j|jd|_1t2st34d dSt34d dS) NinfgMbP?g?r:r) in_channels out_channelsbias kernel_sizegroupspadding)repsaThe fast path is not available because one of `(selective_state_update, causal_conv1d_fn, causal_conv1d_update)` is None. Falling back to the naive implementation. To install follow https://github.com/state-spaces/mamba/#installation and https://github.com/Dao-AILab/causal-conv1dzDThe fast path for Bamba will be used when running the model on a GPU)5superr`rW num_headsrUrMr^rLr]r3rTintermediate_sizermamba_conv_bias use_conv_bias hidden_act activationractmamba_proj_biasuse_bias rms_norm_epslayer_norm_epsilonrVn_groupsrXhead_dimmamba_chunk_size chunk_sizefloattime_step_limit time_step_min time_step_maxconv_dimrConv1dconv1dLinearin_proj Parameterr0onesdt_biasarangelogA_logr{normDout_projis_fast_path_availablelogger warning_once)r\r8rprojection_sizeA __class__r5r6r`sX    zBambaMixer.__init__Nr} cache_paramscache_positionr~r+cCst||}||}|j\}}} |j|j} |duoD|joD|dkoD|j|jjd|j|jjdko8|knoD|duoD|ddk} | r)| dj |j |j |j gdd\} } }t| |j|j|jj d|jj|j} tj | |j | | gdd\}}}t|j }|ddddfdddddfd|j|jjtjd}|dddddfdd|j}|jddddfd|j}|jddddfd|j}|||j|jd|j}|||j|jd|j}|||j |j}t|j|j||||||d|dd }|||j |j}||| }| |ddddf}|St|j }|j!d td fkr>ind |j!i}|j"r||dur|t#||jj d|jj|j|f|j|j$||j|jj|jj%|j j|j j|j|jd d d |}|S|j |j |j |j gdd\} } }|dur| &dd}t'j()||j*|jddf}|j|j+||jdvr|,|| &dddd|f&dd} nt-| &dd|jj d|jj|j|d&dd} t| |} tj | |j | | gdd\}}}t.|||d|j||||||jd||||jdf|j$|jd|d|jdd|\}}|dur:|dur:|j|j+||||d}||| }| |}|S)Nrrrfrg.r=T)zr dt_softplusrrdt_limitF) rrr+rrmsnorm_weight rmsnorm_epsoutproj_weight outproj_biasheaddimngroupsnorm_before_gatereturn_final_statesr:)siluswish)xweightrrr+)rrrr+rrr)/rrrjrr^rKrNrrOsqueezesplitrrrr%rrrrr0exprrexpandrr|float32rrviewr!rrrtrainingr#rvariance_epsilon transposer functionalpadr]copy_rr$r")r\r}rrr~r+projected_statesrGseq_lenrEgroups_time_state_sizeuse_precomputed_statesgatehidden_states_B_CdtBCrrrhidden_states_reshapedoutdt_limit_kwargshidden_states_B_C_transposedrN scan_output ssm_stater5r5r6cuda_kernels_forward*s          <"  ^" V   $        zBambaMixer.cuda_kernels_forwardc3 s|j\}}}|j}t||}|} | jjjjgdd\} } } |duoQ|joQ|dkoQ|j j jd|j j jdkoE|knoQ|duoQ|ddk} | r|j j j ddd|j j <| dddddf |j j j|j j dddddf<|j j j jjjd}tj|jjddd} jr| jj} | } n8|dur| dd}tj|j|jddf}|j j || dddd|fdd} t| |} tj| jjjjjgdd\}}}tj ! }| r[|j j j}| dddddfddddf} | dd"|| jdj#} j$d "j$jdj#}tjj%| | | j} t&| j'dj'd} |d "jj#jj tj(d }t| d |j |d}|)|jdddddf}|"|jjj|jd*}|)|d|jd}| d |ddddf}|)|dj#}||d j |d}|j j |j j |||)|jdddddf}|"|jjj|jd*}|)|d|jd}|j j j |j|jd }|+|jj#j}|+|jjd}t,||}|+|jj#}j-d "j-jdj#}||| |j}|)|dddddf}ntj%| j$} t&| j'dj'd} |)||dj#!}|)||dj!}|)||dj!}|j.jjdjd }|j.jjdjd }j/|j/j/j-d t0|}|| d }| |j| }fdd||||fD\}}}}|1dddd}tj2|dd}tt3|} |dddddddddddf|dddddddddddf}!|!jdd}"|"d | 1dddddd }#|#jdd}$|$d |dddddfjdd}%t|ddddddddf|}&||&1ddddd }'|'ddddf|d jdd}(| r|j j ddddfj |(jd})n t4|(ddddf})tj5|)|(gdd}(tt3tj|dddddddfd}*|*dd}*|*d |(ddddddfjdd}+|+ddddf|+dddf}(},t|}-|ddddf|(ddddddf}.|-1dddd}/|.d|/d }0|%|0}|)|djj#}||}dkrB|ddd|ddddf}|)||d}|,dur\|dur\|j j |,6|| }17|1 |}2|2S)Nrfrgrr)shiftsdimsr>r:.).N).NNrr;)rh output_sizecsg|] }t|jqSr5)rr)rDtpad_sizer\r5r6rH]sz,BambaMixer.torch_forward..r)rr)8rjr=rrrrrrrKrNrrOrollr|r<rrr0sumrrrrrrrrr]rrr^rrrrrrsoftplusclamprrreshape contiguousrbmmrrepeat_interleaverrpermutecumsumr zeros_likerkrr)3r\ input_statesrrr~rGrrEr=rrrrrrNrr}rrr cache_devicerdAdBdBxrOssm_states_reshaped C_reshapedyr D_residualA_cumsumLG_intermediateGM_intermediateMY_diag decay_statesB_decaystatesprevious_states decay_chunk new_statesrstate_decay_outC_times_statesstate_decay_out_permutedY_offrcontextualized_statesr5rr6 torch_forwards      @   ,  $"$ $  $P  &*"&0 (& *   zBambaMixer.torch_forwardcKstrd|jjjjvr||||||S|durtd|j}|dur@|jddkr@|jddkr@||dddddf |}| ||||S)Ncudaz\`seq_idx` support requires fast path support. Please install `mamba_ssm` and `causal_conv1d`rr) rrrr<typerNotImplementedErrorr=rjr|r)r\r}rrr~r+kwargsr=r5r5r6forwards $ zBambaMixer.forward)NNNN)NNN)r,r-r.r/r r3r`r0Tensorrrr1r4rrr __classcell__r5r5rr6rsV D . Src@rb)BambaMLPNrdr5r5r5r6rrerc@rb) BambaRMSNormNrdr5r5r5r6rrercseZdZddededeffdd Zeddd d  dd ej de ej de ej de e de e de e de ej de eej ej fdeedeeje eejejfffddZZS)BambaDecoderLayerr9r8r layer_typecspt|||`d}|dkrtnd}|||_||_|dkr(t||d|_dS|dkr4t|||_dSt d)Nrr9)r8r attentionzInvalid layer_type) rr` self_attnr feed_forwardrrr9rz ValueError)r\r8rr num_expertsffn_layer_classrr5r6r`s zBambaDecoderLayer.__init__past_key_valuepast_key_valuesz4.58)new_nameversionNFr}r~rpoutput_attentions use_cacherposition_embeddingsrreturnc Ks|} ||}|jdkr|jd||||d| }d} n|jdkr4|jd||||||||d| \}} | |}|} ||}||}| |}|f} |rR| | f7} | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, sequence_length)` where padding elements are indicated by 0. past_key_values (`HybridMambaAttentionDynamicCache`, *optional*): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*): Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`, with `head_dim` being the embedding dimension of each attention head. kwargs (`dict`, *optional*): Arbitrary kwargs. Can be used to provide `BambaFlashAttentionKwargs` for padding-free training and/or improve torch.compile performance. r9)r}rrr~Nr)r}r~rpr$r'r(rr)r5)input_layernormrr9rpre_ff_layernormr) r\r}r~rpr$r'r(rr)rresidualself_attn_weightsoutputsr5r5r6rsD#        zBambaDecoderLayer.forward)r9)NNNFFNN)r,r-r.r r3strr`rr0rrr1rbooltuplerr& FloatTensorrrr5r5rr6rs>    rcsDeZdZUeed<dZdZdgZdZdZ dZ dZ fddZ Z S)BambaPreTrainedModelr8modelTrr$csVt|t|tr)|jjdtt d|j d|j _|j jddSdS)Ng?r) r _init_weights isinstancerrdatafill_r0rrrrr)r\modulerr5r6r6/s  z"BambaPreTrainedModel._init_weights)r,r-r.r r2base_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_supports_sdpa _is_statefulr6rr5r5rr6r4#s r4cseZdZdeffdd Zee         ddeej deej deej dee d eej d ee d ee d ee d eej deedefddZdej dej d ej de d e f ddZedej dededejd ej def ddZddZZS) BambaModelr8cst||j|_|j|_t|j|j|j|_g}t |j D]}| t |||j |dq t||_|j|_t|j|jd|_t|d|_d|_|dS)N)rrr)r8F)rr` pad_token_id padding_idx vocab_sizer EmbeddingrU embed_tokensrQrRrYrrJ ModuleListlayers_attn_implementationrrfinal_layernormrc rotary_embgradient_checkpointing post_init)r\r8decoder_layersr_rr5r6r`9s    zBambaModel.__init__N input_idsr~rpr$ inputs_embedsr(r'output_hidden_statesrrr*c  Ks|dur|n|jj}|dur|n|jj}|dur|n|jj}|du|duAr*td|jr9|jr9|r9tdd}|durB| |}|} |rO|durOtd| dur^t j | j d| j d} |durg| d}|||| ||} ||| } || |}|rdnd}|rdnd}|jD]5}|jd kr| n| }|r|| f7}|| f|||||| |d | }|d} |r|ddur||df7}q|| } |r|| f7}|r|jsd |_|sdn|}t| |||d S) Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FzBamba requires an initialized `HybridMambaAttentionDynamicCache` to return a cache. None was provided, so no cache will be returned.rr>rr5r9)r~rpr$r'r(rr)T)last_hidden_stater$r} attentions)r8r'rRr(r rMrrrrGr0rrjr<ri_update_causal_mask_update_mamba_maskrLrIrrKrKr)r\rPr~rpr$rQr(r'rRrrr} causal_mask mamba_maskr)all_hidden_statesall_self_attns decoder_layer layer_mask layer_outputs next_cacher5r5r6rLs~                 zBambaModel.forward input_tensorc Cs|jjdkr|durd|vr|SdS|dur|nd}|jjdkr0|s0tj||||jdr0dS|j}|jd}t|t j rC|jdn||d} |j ||| |||jdd} |jjdkru|duru|j j d vru|sut |j} t| | } | S) Nflash_attention_2rrsdpa)rQpast_key_values_length is_trainingrrf)sequence_length target_lengthr=rrG)rxpunpu)r8rJget_seq_lengthr_ignore_causal_mask_sdparr=rjr7r0r5_prepare_4d_causal_attention_mask_with_cache_positionr<rfinfomin_unmask_unattended) r\r~r_rr$r'past_seen_tokensr=rdrerW min_dtyper5r5r6rUsF        zBambaModel._update_causal_maskrdrer=rGc Ks||dur|dkr|}|St|j}tj||f|||jd}|dkr+tj|dd}|tj||jd|ddk9}|ddddddf |ddd}|dur| }|j d} |ddddddf|ddddddfkdddd| dddf |} |ddddddd| f| } | dk} |ddddddd| f | ||ddddddd| f<|S) aM Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. Nr) fill_valuer=r<r)diagonalr>rfr)rhr0rkrlfullr<triurrrclonerjr| masked_fill) r~rdrer=rrGrrWro mask_lengthpadding_attention_mask padding_maskr5r5r6rjs2  $ .$  z@BambaModel._prepare_4d_causal_attention_mask_with_cache_positioncCs.|}|ddks|durt|dkrd}|S)zv No need for zeroing states when 1. Cached forward 2. Attending to all inputs rNr)r0all)r\r~rrXr5r5r6rV$s"zBambaModel._update_mamba_mask) NNNNNNNNN)r,r-r.r r`rrrr0r1rrr3r1rr&rrrU staticmethodr3r=rjrVrr5r5rr6rB7s|    b < 7rBcseZdZfddZ           ddeejdeejdeejdeed eej d eejd ee d ee d ee deejde e ejfde fddZ      dddZZS)BambaForCausalLMcs t||j|_|dS)N)rr`z_loss_coefficientrN)r\r8rr5r6r`1s  zBambaForCausalLM.__init__NrrPr~rpr$rQlabelsr(r'rRrlogits_to_keepr*c  Ks|dur|n|jj}| dur| n|jj} |jd |||||||| | d | } | j}t| tr4t| dn| }||dd|ddf}d}|durt|j d |||jj d| }|j dkrt|j ddj |jdd}||j |}t||| j| j| jd S) aJ labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, BambaForCausalLM >>> model = BambaForCausalLM.from_pretrained("...") >>> tokenizer = AutoTokenizer.from_pretrained("...") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) rPr~rpr$rQr(r'rRr)logitsr}rErrfrgrr:)lossrr$r}rTr5)r8r'rRr5rSr7r3slicelm_head loss_functionrEr| logsumexpr|r=powmeanrr$r}rT)r\rPr~rpr$rQr}r(r'rRrr~rr/r} slice_indicesrrz_lossr5r5r6r8s@%    zBambaForCausalLM.forwardTc Ks<|du} | s5|dus|d|jdkr"|dd|jd df}n!|jd|jdkr4|dd|f}nt|j|jd|j|jd}|durl|durl|dd}||dkd| sl|dd|jd df}|durw| rwd|i} nd|i} | |||||jj |d| D] \} } | | vr| | | <q| S)Nrfrrr>rQrP)rpr$r(r~r~r) rjrr8r=r<longr masked_fill_rupdatenum_logits_to_keepitems) r\rPr$r~rQrrpr(r empty_past_kv model_inputskeyvaluer5r5r6prepare_inputs_for_generationsB     z.BambaForCausalLM.prepare_inputs_for_generation) NNNNNNNNNNr)NNNNNT)r,r-r.r`rr0r1rrr3r1rr3rrrrr5r5rr6r{0sZ       Pr{)rBr{r4)Nr)Hr/typingrrrr0rtransformers.activationsr(transformers.models.jamba.modeling_jambarr(transformers.models.llama.modeling_llamar r r r r r*transformers.models.mamba2.modeling_mamba2rrrrmodeling_attn_mask_utilsrmodeling_outputsrrmodeling_utilsrprocessing_utilsrutilsrrrutils.deprecationrutils.import_utilsrrconfiguration_bambar +mamba_ssm.ops.triton.selective_state_updater!!mamba_ssm.ops.triton.ssd_combinedr"r# causal_conv1dr$r%ryr get_loggerr,rr&rcryrzr{rModulerrrrr4rBr{__all__r5r5r5r6s^          5 ( bay