o Û¾iR ã@sbddlZddlmZmZddlmZddlmZddlm Z Gdd„dej ƒZ Gdd „d eƒZ dS) éN)ÚABCÚabstractmethod)ÚIterable)Ú BlockHash)Ú LoadStoreSpeccs>eZdZdZdejfgZ‡fdd„Zede fdd„ƒZ ‡Z S)Ú BlockStatusaW Offloading status for a single block of KV data. Holds the following information: ref_cnt - the current number of transfers using this block as a source. A value of -1 indicates the block is not yet ready to be read. load_store_spec - backend-specific information on how to actually read/write the block. Úref_cntcstƒ ¡d|_dS)Néÿÿÿÿ)ÚsuperÚ__init__r©Úself©Ú __class__©úN/home/ubuntu/.local/lib/python3.10/site-packages/vllm/v1/kv_offload/backend.pyr s  zBlockStatus.__init__ÚreturncCs |jdkS)z@ Returns whether the block is ready to be read. r)rr rrrÚis_readys zBlockStatus.is_ready) Ú__name__Ú __module__Ú __qualname__Ú__doc__ÚctypesÚc_int32Ú_fields_r ÚpropertyÚboolrÚ __classcell__rrrrr s  rc@s|eZdZdZdedefdd„Zedd„ƒZede e d e e fd d „ƒZ ed e fd d„ƒZ dee dee d efdd„ZdS)ÚBackendzi An abstract class for allocating and returning specs for writing KV blocks to some backend. Ú block_sizeÚmediumcCs||_||_dS)N)rr )r rr rrrr +s zBackend.__init__cCódS)zW Returns the number of current number of blocks that can be allocated. Nrr rrrÚget_num_free_blocks/szBackend.get_num_free_blocksÚ block_hashesrcCr!)aà Allocate space for writing blocks. This method assumes there is enough space for allocation. It is unsafe to use without checking get_num_free_blocks beforehand. Args: block_hashes: the hashes identifying the blocks to be written. Returns: A list of BlockStatus for the allocated blocks. The ref_cnt of each returned item will be -1, meaning the block is not yet ready to be read. Nr)r r#rrrÚallocate_blocks6szBackend.allocate_blocksÚblockcCr!)zç Free a previously allocated block. You should only call this function with blocks returned by allocate_blocks, and only once per each block. Args: block: The block to be freed. Nr)r r%rrrÚfreeGs z Backend.freeÚblockscCst‚)aB Get backend-specific information on how to read/write blocks. Args: block_hashes: the list of block hashes identifying the blocks. blocks: the list of blocks. Returns: A LoadStoreSpec that can be used by a worker to read/write the blocks. )ÚNotImplementedError)r r#r'rrrÚget_load_store_specSszBackend.get_load_store_specN)rrrrÚintÚstrr rr"Úlistrrr$r&rrr)rrrrr%s   ÿÿþr) rÚabcrrÚcollections.abcrÚvllm.v1.core.kv_cache_utilsrÚvllm.v1.kv_offload.abstractrÚ StructurerrrrrrÚs