o ۾i @sddlmZmZddlmZddlmZddlmZm Z ddl m Z ddl Z ddl mZe r4ddlmZGd d d eZeZeeeeedBeefZeeeefZed d Gd ddZGdddeZdS))ABCabstractmethod)Sequence) dataclass)Enumauto) TYPE_CHECKINGN)SamplingParams) VllmConfigc@seZdZeZeZdS)MoveDirectionalityN)__name__ __module__ __qualname__rUNIDIRECTIONALSWAPrr]/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/sample/logits_processor/interface.pyr s r T)frozenc@s>eZdZUdZeed<eeed<eeed<ee ed<dS) BatchUpdatez2Persistent batch state change info for logitsprocs batch_sizeremovedaddedmovedN) r r r__doc__int__annotations__rRemovedRequest AddedRequest MovedRequestrrrrr$s   rc @seZdZedefddZedddejde dd fd d Z ed ej dej fd dZ ede fddZ e    dddZd S)LogitsProcessorsampling_paramscCsdS)zhValidate sampling params for this logits processor. Raise ValueError for invalid ones. Nr)clsr rrrvalidate_params=szLogitsProcessor.validate_params vllm_configr device is_pin_memoryreturnNcCt)NNotImplementedError)selfr#r$r%rrr__init__EszLogitsProcessor.__init__logitscCr')zApply LogitsProcessor to batch logits tensor. The updated tensor must be returned but may be modified in-place. r()r*r,rrrapplyKszLogitsProcessor.applycCr')aTrue if logits processor has no impact on the argmax computation in greedy sampling. NOTE: may or may not have the same value for all instances of a given LogitsProcessor subclass, depending on subclass implementation. r()r*rrris_argmax_invariantTsz#LogitsProcessor.is_argmax_invariant batch_updateBatchUpdate | NonecCr')zCalled when there are new output tokens, prior to each forward pass. Args: batch_update: Non-None iff there have been changes to the batch makeup. r()r*r/rrr update_state^s zLogitsProcessor.update_state)r/r0r&N)r r r classmethodr r"rtorchr$boolr+Tensorr-r.r1rrrrr<s.  r)abcrrcollections.abcr dataclassesrenumrrtypingrr3vllmr vllm.configr r rrtuplelistrrrrrrrrs