o .i% @s"UdZddlmZddlmZmZddlmZmZddl m Z ddl m Z er-ddl mZe eZGdd d eZGd d d eed ZGd ddeed ZejjejjejjejjejjejjdZiZeeefed<iZ eeefed<  ddeeBdedBde!dee"ge"ffddZ#dS)zAttention backend registry)Callable)EnumEnumMeta) TYPE_CHECKINGcast) init_logger)resolve_obj_by_qualname)AttentionBackendcs&eZdZdZdeffdd ZZS)_AttentionBackendEnumMetazDMetaclass for AttentionBackendEnum to provide better error messages.namecsNzt|WSty&td|j}d|}td|d|dw)z0Get backend by name with helpful error messages.zdict[str, Enum]z, zUnknown attention backend: 'z'. Valid options are: N)super __getitem__KeyErrorr __members__keysjoin ValueError)clsr membersvalid_backends __class__`/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/v1/attention/backends/registry.pyr s  z%_AttentionBackendEnumMeta.__getitem__)__name__ __module__ __qualname____doc__strr __classcell__rrrrr sr c@seZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdZdZd%dedefddZd&dd Zdefd!d"Zd'd#d$Z dS)(AttentionBackendEnumaEnumeration of all supported attention backends. The enum value is the default class path, but this can be overridden at runtime using register_backend(). To get the actual backend class (respecting overrides), use: backend.get_class() z;vllm.v1.attention.backends.flash_attn.FlashAttentionBackendzHvllm.v1.attention.backends.flash_attn_diffkv.FlashAttentionDiffKVBackendz=vllm.v1.attention.backends.triton_attn.TritonAttentionBackendz9vllm.v1.attention.backends.rocm_attn.RocmAttentionBackendz=vllm.v1.attention.backends.mla.rocm_aiter_mla.AiterMLABackendzEvllm.v1.attention.backends.mla.aiter_triton_mla.AiterTritonMLABackendzCvllm.v1.attention.backends.rocm_aiter_fa.AiterFlashAttentionBackendzNvllm.v1.attention.backends.mla.rocm_aiter_mla_sparse.ROCMAiterMLASparseBackendz7vllm.v1.attention.backends.flashinfer.FlashInferBackendzBvllm.v1.attention.backends.mla.flashinfer_mla.FlashInferMLABackendz:vllm.v1.attention.backends.mla.triton_mla.TritonMLABackendzvllm.v1.attention.backends.flex_attention.FlexAttentionBackendz9vllm.v1.attention.backends.tree_attn.TreeAttentionBackendzSvllm.v1.attention.backends.rocm_aiter_unified_attn.RocmAiterUnifiedAttentionBackendz7vllm.v1.attention.backends.cpu_attn.CPUAttentionBackendNTinclude_classnamereturncCDt||j}|std|jd|jd|s |ddd}|SzGet the class path for this backend (respects overrides). Returns: The fully qualified class path string Raises: ValueError: If Backend.CUSTOM is used without being registered zBackend z= must be registered before use. Use register_backend(Backend.z, 'your.module.YourClass').r)_ATTN_OVERRIDESgetvaluerr rsplitselfr"pathrrrget_pathU  zAttentionBackendEnum.get_pathtype[AttentionBackend]cC t|SaGet the backend class (respects overrides). Returns: The backend class Raises: ImportError: If the backend class cannot be imported ValueError: If Backend.CUSTOM is used without being registered rr/r-rrr get_classh zAttentionBackendEnum.get_classcC|tvSzCheck if this backend has been overridden. Returns: True if the backend has a registered override )r(r5rrr is_overriddentz"AttentionBackendEnum.is_overriddencCt|ddSz>Clear any override for this backend, reverting to the default.N)r(popr5rrrclear_override|z#AttentionBackendEnum.clear_overrideTr#r1r#N)!rrrr FLASH_ATTNFLASH_ATTN_DIFFKV TRITON_ATTN ROCM_ATTNROCM_AITER_MLAROCM_AITER_TRITON_MLA ROCM_AITER_FAROCM_AITER_MLA_SPARSE TORCH_SDPA FLASHINFERFLASHINFER_MLA TRITON_MLA CUTLASS_MLAFLASHMLAFLASHMLA_SPARSEFLASH_ATTN_MLAIPEX NO_ATTENTIONFLEX_ATTENTION TREE_ATTNROCM_AITER_UNIFIED_ATTNCPU_ATTNCUSTOMboolrr/r6r:r?rrrrr "sH   r ) metaclassc@s^eZdZdZdZdZdZdZdZdZ dd e d e fd d Z dddZ d e fddZdddZdS)MambaAttentionBackendEnuma Enumeration of all supported mamba attention backends. The enum value is the default class path, but this can be overridden at runtime using register_backend(). To get the actual backend class (respecting overrides), use: backend.get_class() z=vllm.v1.attention.backends.mamba1_attn.Mamba1AttentionBackendz=vllm.v1.attention.backends.mamba2_attn.Mamba2AttentionBackendzDvllm.v1.attention.backends.short_conv_attn.ShortConvAttentionBackendz=vllm.v1.attention.backends.linear_attn.LinearAttentionBackendz7vllm.v1.attention.backends.gdn_attn.GDNAttentionBackendNTr"r#cCr$r%)_MAMBA_ATTN_OVERRIDESr)r*rr r+r,rrrr/r0z"MambaAttentionBackendEnum.get_pathr1cCr2r3r4r5rrrr6r7z#MambaAttentionBackendEnum.get_classcCr8r9)r^r5rrrr:r;z'MambaAttentionBackendEnum.is_overriddencCr<r=)r^r>r5rrrr?r@z(MambaAttentionBackendEnum.clear_overriderArBrC)rrrrMAMBA1MAMBA2 SHORT_CONVLINEARGDN_ATTNrZr[rr/r6r:r?rrrrr]s   r])mamba1mamba2 short_convlinear_attention gdn_attentioncustomr(r^NFbackend class_pathis_mambar#csHdtdtffdd }|dur"r|t<ddS|t<ddS|S)aRegister or override a backend implementation. Args: backend: The AttentionBackendEnum member to register class_path: Optional class path. If not provided and used as decorator, will be auto-generated from the class. Returns: Decorator function if class_path is None, otherwise a no-op Examples: # Override an existing attention backend @register_backend(AttentionBackendEnum.FLASH_ATTN) class MyCustomFlashAttn: ... # Override an existing mamba attention backend @register_backend(MambaAttentionBackendEnum.LINEAR, is_mamba=True) class MyCustomMambaAttn: ... # Register a custom third-party attention backend @register_backend(AttentionBackendEnum.CUSTOM) class MyCustomBackend: ... # Direct registration register_backend( AttentionBackendEnum.CUSTOM, "my.module.MyCustomBackend" ) rr#cs8r|jd|jt<|S|jd|jt<|S)Nr&)rrr^r()rrjrlrr decorators z#register_backend..decoratorNcSs|S)Nr)xrrrsz"register_backend..)typer^r()rjrkrlrnrrmrregister_backends&rr)NF)$rcollections.abcrenumrrtypingrr vllm.loggerrvllm.utils.import_utilsrvllm.v1.attention.backendr rloggerr r r]r_r r`rarbrcrZMAMBA_TYPE_TO_BACKEND_MAPr(dictr__annotations__r^r[rqrrrrrrs@    _@