o ۾i"@slddlmZddlmZddlmZddlmZmZm Z m Z er&ddlm Z GdddZ Gdd d Z d S) )Iterator)chain) TYPE_CHECKING) AddedRequest BatchUpdate MovedRequestRemovedRequest)LogitsProcessorc @seZdZUdZeeed<eed<eeed<ee ed<   ddeedBdeedBdee dBddfd d Z dd d Z e deefd dZ deddfddZdefddZdedBfddZdedBfddZdefddZdededBfddZdS)BatchUpdateBuilderaHelps track persistent batch state changes and build a batch update data structure for logitsprocs Assumptions: * All information about requests removed from persistent batch during a step is aggregated in self._removed through calls to self.removed_append() at the beginning of a step. This must happen before the first time that self.removed, self.pop_removed() or self.peek_removed() are invoked in a given step * After the first time that self.removed, self.pop_removed() or self.peek_removed() are read in a step, no new removals are registered using self.removed_append() * Elements of self._removed are never directly modified, added or removed (i.e. modification is only via self.removed_append() and self.pop_removed()) Guarantees under above assumptions: * self.removed is always sorted in descending order * self.pop_removed() and self.peek_removed() both return the lowest removed request index in the current step _removed_is_removed_sortedaddedmovedNremovedreturncCs.|pg|_|pg|_|p g|_d|_d|_dS)NF)r r rr batch_changed)selfrr rrY/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/sample/logits_processor/state.py__init__,s    zBatchUpdateBuilder.__init__cCs"|js|jjddd|_dSdS)zSort removed request indices in descending order. Idempotent after first call in a given step, until reset. T)reverseN)r r sortrrrr_ensure_removed_sorted;s z)BatchUpdateBuilder._ensure_removed_sortedcCs||jS)z:Removed request indices sorted in descending order)rr rrrrrEszBatchUpdateBuilder.removedindexcCs$|jrtd|j|d|_dS)zRegister the removal of a request from the persistent batch. Must not be called after the first time self.removed, self.pop_removed() or self.peek_removed() are invoked. Args: index: request index zECannot register new removed request after self.removed has been read.TN)r RuntimeErrorr appendr)rrrrrremoved_appendLs    z!BatchUpdateBuilder.removed_appendcCs t|jSN)boolr rrrr has_removed\s zBatchUpdateBuilder.has_removedcCs|r ||jdSdS)z#Return lowest removed request indexN)r rr rrrr peek_removed_ zBatchUpdateBuilder.peek_removedcCs|r ||jSdS)z Pop lowest removed request indexN)r rr poprrrr pop_removedfr#zBatchUpdateBuilder.pop_removedcCs4d|_|j|j|j|j}d|_|S)z4Returns True if there were any changes to the batch.F)r r clearr rr)rrrrrresetms   zBatchUpdateBuilder.reset batch_sizecCsPd|_d|_t|j|j|jfsdSt||j|j|jd}g|_g|_g|_|S)aGenerate a logitsprocs batch update data structure and reset internal batch update builder state. Args: batch_size: current persistent batch size Returns: Frozen logitsprocs batch update instance; `None` if no updates FN)r(rrr )r ranyr rr r)rr( batch_updaterrr get_and_resetws z BatchUpdateBuilder.get_and_reset)NNN)rN)__name__ __module__ __qualname____doc__listr__annotations__rrrrrpropertyrintrr r"r%r'rr+rrrrr s6          r c@sBeZdZdZd deddBddfddZededfdd ZdS) LogitsProcessorsz,Encapsulates initialized logitsproc objects.N logitsprocsr rcCs<g|_g|_|r|D]}|r|jn|j|q dSdSr)argmax_invariantnon_argmax_invariantis_argmax_invariantr)rr5 logitprocrrrrs zLogitsProcessors.__init__cCst|j|jS)z$Iterator over all logits processors.)rr6r7rrrrallszLogitsProcessors.allr)r,r-r.r/rrr2r:rrrrr4s  r4N)collections.abcr itertoolsrtypingr)vllm.v1.sample.logits_processor.interfacerrrrr r r4rrrrs