o پi` @srddlmZddlZddlmZddlmmZddlm Z ddl m Z eGdddZ Gdddej ZdS) ) dataclassN)PretrainedConfig) ForwardBatchc@seZdZUejed<dS)SparseEmbeddingOutput embeddingsN)__name__ __module__ __qualname__torchTensor__annotations__r r S/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/sparse_pooler.pyr s rcsLeZdZdZdeffdd Zdejdede fdd Z d e fd d Z Z S) SparsePooleraJA layer that pools hidden states into sparse vocabulary-space embeddings. This layer does the following: 1. Applies a linear transformation + ReLU to get token-level weights 2. Maps these weights to vocabulary positions using token IDs 3. Aggregates weights for repeated tokens using max pooling 4. Returns sparse embeddings in vocabulary space Attributes: config: Model configuration containing vocab_size and hidden_size sparse_linear: Linear layer for computing token weights vocab_size: Size of vocabulary for output embeddings configcshtt|dstdt|dt|ds#tdt|d|j|_t|jd|_ d|_ dS)N vocab_sizezConfig z( missing required 'vocab_size' attribute hidden_sizez) missing required 'hidden_size' attributeF) super__init__hasattrAttributeErrortypernnLinearr sparse_linear_weights_loaded)selfr __class__r rrs    zSparsePooler.__init__ hidden_states forward_batchreturncCs|jstdt||d}ttjt |j |j d|j }tj t |j |j |j|j d}||j |j}|djd||ddt|dS) aM Forward pass for sparse pooling. Args: hidden_states: Packed sequence hidden states [total_tokens, hidden_size] forward_batch: Batch information with sequence lengths and input_ids Returns: SparseEmbeddingOutput with embeddings of shape [batch_size, vocab_size] zs