o i1 @svddlZddlmZddlmZmZddlZddlmZddlm Z e dZ eGdddZ Gd d d ej ZdS) N) dataclass)OptionalTuple)nn)deprecated_functionxformersc@s&eZdZUeed<eed<eed<dS)InputProjectionConfig in_features out_featuresbiasN)__name__ __module__ __qualname__int__annotations__boolrrX/home/ubuntu/.local/lib/python3.10/site-packages/xformers/components/input_projection.pyrs  rc sneZdZdZ ddedeedeedeffdd Zd ej d ej d ej d e ej ej ej ffd dZ Z S)InputProjectionz] Handle all the input projections in one go, opportunistically fuse some operations. Tquery_proj_paramskey_proj_paramsvalue_proj_paramsuse_separate_proj_weightcstt||j|_t|j|j|j|_|dur(t|j|j|j|_ n t d|j|_ |durAt|j|j|j|_ n t d|j|_ |sot |jj|j _|jj|j _WddS1shwYdSdS)NzhNo Key projection parameters were passed, assuming that the weights are shared with the query projectionzjNo Value projection parameters were passed, assuming that the weights are shared with the query projection)super__init__rr rLinearr r q_projk_projloggerinfov_projtorchno_gradweight)selfrrrr __class__rrr"sD   "zInputProjection.__init__querykeyvaluereturncCs2tdd|j|j|jg|||g\}}}|||fS)NcSs||S)Nr)fnxrrrasz)InputProjection.forward..)maprrr )r$r'r(r)qkvrrrforwardVs   zInputProjection.forward)T) r r r__doc__rrrrr!Tensorrr2 __classcell__rrr%rrs* 4r)logging dataclassesrtypingrrr!rxformers._deprecation_warningr getLoggerrrModulerrrrrs