o .i#@s UddlZddlZddlmZddlmZddlmZddlZddl m Z ddl m Z ddl mZddlmZe eZdeed fd ejd efd d ZdZdZdaeded<GdddZd efddZdddZ d dejdedBd dfddZ d!ddZ!d!ddZ"dS)"N) accumulate)prod)Optional) init_loggerround_up)dbo_current_ubatch_idshape.dtypereturncCst||jSN)ritemsize)r r rU/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/vllm/v1/worker/workspace.py_compute_bytessrii@WorkspaceManager_managerc@seZdZdZddejdedBfddZedej dBdefd d Z dd d Z de fd dZ deeedfejfdeej fddZdedej fddZdS)rzManager for workspace allocation. Manages workspace buffers for DBO (Dual Batch Overlap) execution. Can be locked to prevent further growth during execution. Ndevice num_ubatchescCs,||_|dur |nd|_ddg|_d|_dS)NF)_device _num_ubatches_current_workspaces_locked)selfrrrrr__init__'s  zWorkspaceManager.__init__ workspacer cCs|durdS||S)zGet size of workspace in bytes.Nr)numel element_size)rrrr_workspace_size_bytes.sz&WorkspaceManager._workspace_size_bytescs0d_tjrtdfddjDdSdS)zLock the workspace to prevent further growth. After locking, any attempt to allocate a larger workspace will raise an assertion error. This ensures workspace size is fixed during execution. Tz5[WORKSPACE DEBUG] Workspace locked. Current sizes: %scs"g|] }|dur|tqSr )r_MB).0wsrrr ?s  z)WorkspaceManager.lock..N)renvsVLLM_DEBUG_WORKSPACEloggerinforr#rr#rlock5s zWorkspaceManager.lockcCs|jS)zCheck if workspace is locked.)rr#rrr is_lockedFszWorkspaceManager.is_lockedshapes_and_dtypes.cshddDddD}t|}ttdg|dd||fddttDS)a Get multiple workspace tensors simultaneously from a single allocation. Args: *shapes_and_dtypes: One or more (shape, dtype) tuples. Returns: List of tensor views into the workspace buffer, one per shape/dtype pair. cSsg|] \}}t||qSr)r)r!sdrrrr$Usz5WorkspaceManager.get_simultaneous..cSsg|]}t|dqS)r)r!actualrrrr$VsrNcsDg|]}||||d|dqS)rr)viewreshape)r!i actual_bytescurrent_workspaceoffsetsr+rrr$^s )sumlistr_ensure_workspace_sizerangelen)rr+ aligned_bytes total_bytesrr4rget_simultaneousJs   z!WorkspaceManager.get_simultaneousrequired_bytesc Cst}|j|}||}||krzdtfdd}|jr0td|d|tdd|tddt|jD]&}|j|}|d usG|||kr[d |j|<~t j |ft j |j d |j|<q5t jrttd ||t|t|j||jt|jt}|S) zEnsure workspace is allocated and large enough, return current workspace. Args: required_bytes: The number of bytes required. Returns: The current workspace tensor. r cSsjt}|dur dS|j}|dur3t|jdtr|j}q tj |j j }|d|j d|j j SdS)z*Find first frame outside WorkspaceManager.Nunknownr:)inspect currentframef_back isinstancef_localsgetrospathbasenamef_code co_filenamef_linenoco_name) curr_framefilenamerrrget_caller_infotsz@WorkspaceManager._ensure_workspace_size..get_caller_infoz)Workspace is locked but allocation from 'z ' requires z.2fz MB, current size is z3 MB. Workspace growth is not allowed after locking.N)r rze[WORKSPACE DEBUG] Resized workspace from '%s': %.2f MB -> %.2f MB (%d ubatches, total memory %.2f MB))rrrstrrAssertionErrorr r;rtorchemptyuint8rr%r&r'r()rr@ ubatch_idr6 current_sizerRrrrr:esD         z'WorkspaceManager._ensure_workspace_sizer r N)__name__ __module__ __qualname____doc__rUrintr staticmethodTensorrr)boolr*tupler r9r?r:rrrrr s  cCstduS)zCheck if workspace manager has been initialized. Returns: True if workspace manager is initialized, False otherwise. Nrrrrr is_workspace_manager_initializedsrecCstdusJdtS)zGet the current workspace manager instance. Raises: AssertionError: If workspace manager has not been initialized. NzoWorkspaceManager not initialized. Call init_workspace_manager() with a device before using workspace functions.rdrrrrcurrent_workspace_managers rfrrcCs&tdur tdtj|t||adS)aInitialize the workspace manager with a device. Must be called before using any workspace functions. Typically called from GPUModelRunner.__init__. Args: device: The device to allocate workspace on. num_ubatches: Number of micro-batches. Defaults to 1. NzNWorkspaceManager already initialized on device %s, reinitializing on device %s)rr'warningrr)rrrrrinit_workspace_managers rhcCstdS)aeLock the workspace to prevent further growth. After calling this function, any attempt to allocate a workspace larger than the current size will raise an AssertionError. This ensures that workspace size is fixed during execution and prevents unexpected memory allocations in the hot path. Example: # During initialization init_workspace_manager(device) reserve_workspace(shape1, dtype1) reserve_workspace(shape2, dtype2) # Lock after warmup/profiling lock_workspace() # Now all get_workspace calls must fit in pre-allocated size N)rfr)rrrrlock_workspacesricCsdadS)zReset the workspace manager to uninitialized state. This is primarily intended for testing purposes to allow tests to reinitialize the workspace manager cleanly. Nrdrrrrreset_workspace_managersrj)r rr rZ)#rCrI itertoolsrmathrtypingrrU vllm.envsr% vllm.loggerrvllm.utils.math_utilsrvllm.v1.worker.ubatchingrr[r'rcr_r rr _GiBr__annotations__rrbrerfrrhrirjrrrrs: