o i @s<UddlZddlmZddlmZmZmZmZmZddl Z ddl m Z m Z m Z ddlmZddlmZddlmZmZed ZiZeeefed <eZeeed <d Zd ZdeeeefeffddZ e eeeeZeeegeegeffed<defddZ ddZ!ddl"m#Z#gdZ$e eee%j&ddS)N)Path)AnyCallableDictSetUnion)generate_matching_configget_registry_decoratorimport_all_modules)SparseCS) AttentionMask) AttentionAttentionConfigxformersATTENTION_REGISTRYATTENTION_CLASS_NAMESg333333?Tconfigc Csvt|ts0z t|t|dj}Wn ty/}z|d}t|dt|d}~ww|}t|j j |S)aQBuilds an attention from a config. This assumes a 'name' key in the config which is used to determine what attention class to instantiate. For instance, a config `{"name": "my_attention", "foo": "bar"}` will find a class that was registered as "my_attention" (see :func:`register_attention`) and call .from_config on it.namez not available among N) isinstancerrrrKeyErrorloggerwarningkeysr constructor from_config)rconfig_instanceerrZ/home/ubuntu/.local/lib/python3.10/site-packages/xformers/components/attention/__init__.pybuild_attention$s  r register_attentionreturncCs,t||tkrt|St|SN)torch count_nonzeroitemnumel_DENSITY_THRESHOLDr from_boolsparsifymatrixrrrmaybe_sparsifyTs r-cCstrt|S|Sr#) _USE_SPUTNIKr to_sparser+rrrr*]sr*)ScaledDotProduct)r0rr r r!zxformers.components.attention)'loggingpathlibrtypingrrrrrr$xformers.utilsrr r _sputnik_sparser attention_maskr baserr getLoggerrrstr__annotations__setrr(r.r r!r-r*scaled_dot_productr0__all____file__parentrrrrs.     "