o .iR @sbddlZddlmZmZddlmZddlmZddlm Z Gdddej Z Gdd d eZ dS) N)ABCabstractmethod)Iterable) BlockHash) LoadStoreSpeccs>eZdZdZdejfgZfddZede fddZ Z S) BlockStatusaW Offloading status for a single block of KV data. Holds the following information: ref_cnt - the current number of transfers using this block as a source. A value of -1 indicates the block is not yet ready to be read. load_store_spec - backend-specific information on how to actually read/write the block. ref_cntcstd|_dS)N)super__init__rself __class__W/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/v1/kv_offload/backend.pyr s  zBlockStatus.__init__returncCs |jdkS)z@ Returns whether the block is ready to be read. r)rr rrris_readys zBlockStatus.is_ready) __name__ __module__ __qualname____doc__ctypesc_int32_fields_r propertyboolr __classcell__rrrrr s  rc@s|eZdZdZdedefddZeddZede e d e e fd d Z ed e fd dZ dee dee d efddZdS)Backendzi An abstract class for allocating and returning specs for writing KV blocks to some backend. block_sizemediumcCs||_||_dS)N)rr )r rr rrrr +s zBackend.__init__cCdS)zW Returns the number of current number of blocks that can be allocated. Nrr rrrget_num_free_blocks/szBackend.get_num_free_blocks block_hashesrcCr!)a Allocate space for writing blocks. This method assumes there is enough space for allocation. It is unsafe to use without checking get_num_free_blocks beforehand. Args: block_hashes: the hashes identifying the blocks to be written. Returns: A list of BlockStatus for the allocated blocks. The ref_cnt of each returned item will be -1, meaning the block is not yet ready to be read. Nr)r r#rrrallocate_blocks6szBackend.allocate_blocksblockcCr!)z Free a previously allocated block. You should only call this function with blocks returned by allocate_blocks, and only once per each block. Args: block: The block to be freed. Nr)r r%rrrfreeGs z Backend.freeblockscCst)aB Get backend-specific information on how to read/write blocks. Args: block_hashes: the list of block hashes identifying the blocks. blocks: the list of blocks. Returns: A LoadStoreSpec that can be used by a worker to read/write the blocks. )NotImplementedError)r r#r'rrrget_load_store_specSszBackend.get_load_store_specN)rrrrintstrr rr"listrrr$r&rrr)rrrrr%s   r) rabcrrcollections.abcrvllm.v1.core.kv_cache_utilsrvllm.v1.kv_offload.abstractr Structurerrrrrrs