o پi>@sddlmZddlmZddlmZddlZddlmZddl m Z ddl m Z ddl mZGdd d eZeGd d d ZGd d d ejZGdddejZdS)) dataclass)IntEnum)OptionalN)PretrainedConfig)%get_cross_encoder_activation_function) ForwardBatchc@seZdZdZdZdS) PoolingTyperN)__name__ __module__ __qualname__LASTCLSrrL/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/pooler.pyrsrc@s"eZdZUejeejBed<dS)EmbeddingPoolerOutput embeddingsN)r r r torchTensorlist__annotations__rrrrrs rcsBeZdZdZdedeffdd Zdejde de fd d Z Z S) PooleraA layer that pools specific information from hidden states. This layer does the following: 1. Extracts specific tokens or aggregates data based on pooling method. 2. Normalizes output if specified. 3. Returns structured results as `PoolerOutput`. Attributes: pooling_type: The type of pooling to use (LAST, AVERAGE, MAX). normalize: Whether to normalize the pooled data. pooling_type normalizecst||_||_dSN)super__init__rr)selfrr __class__rrr's  zPooler.__init__ hidden_states forward_batchreturncCs|jtjkrtj|jddd}||}n.|jtjkr;|j}t|}|ddtj|dddd7<||}ntd|j|j durjt t |j dk}|r_|dd|j df}n ddt ||j D}|j rt|trzd d|D}n tjj |d dd }t|d S) Nr)dimr zInvalid pooling type: .cSs g|] \}}|dd|fqS).Nr).0tensorr#rrr @sz"Pooler.forward..cSsg|] }tjj|dddqS)r$pr#)nn functionalr)r%r&rrrr'Gsr(r)r)rrr rcumsumextend_seq_lensr zeros_like ValueError dimensionslensetzipr isinstancerr+r,r)rr r!last_token_indices pooled_data prompt_lensfirst_token_flat_indicesall_same_dimensionsrrrforward,s.    &     zPooler.forward) r r r __doc__rboolrrrrrr< __classcell__rrrrrs rcsReZdZdZ d dedejdeejffdd Zde j d e d e fd d Z ZS)CrossEncodingPooleraA layer that pools specific information from hidden states. This layer does the following: 1. Extracts specific tokens or aggregates data based on pooling method. 2. Normalizes output if specified. 3. Returns structured results as `EmbeddingPoolerOutput`. Nconfig classifierpoolercs$t||_||_t||_dSr)rrrBrCrdefault_activation_function)rrArBrCrrrrZs zCrossEncodingPooler.__init__r r!r"c Cs|j}d}g}|D]$}||||}|jdur|||}n||}||||7}q t|} |jdur=|| } || d} t| dS)z2Pools sentence pair scores from the hidden_states.rNr$r-) r/rCrBappendrstackrDsqueezer) rr r!r9offsetpooled_data_lst prompt_len pooled_data_ifinal_shape_tensor pooled_outputscoresrrrr<es        zCrossEncodingPooler.forwardr)r r r r=rr+Modulerrrrrrr<r?rrrrr@Qs"  r@) dataclassesrenumrtypingrrtorch.nnr+ transformersrsglang.srt.layers.activationr,sglang.srt.model_executor.forward_batch_inforrrrOrr@rrrrs       5