o ۷i@sdZddlmZddlmZddlmZddlmZddl m Z GdddZ Gd d d Z Gd d d Z eGd ddZGdddZdS)zBase hook classes for model forward interception. This module provides the foundational hook mechanism that allows intercepting and modifying model forward passes without invasive changes to model code. ) annotations)Callable) dataclass)AnyNc@seZdZdZdddZdS) BaseStatez%Base class for hook state containers.returnNonecCsdSNselfr r T/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm_omni/diffusion/hooks/base.pyresetszBaseState.resetNrr)__name__ __module__ __qualname____doc__rr r r r rsrc@s8eZdZdZdddZdd d Zdd dZdddZdS) StateManagerz(Manage per-context hook state instances. state_clsCallable[[], BaseState]cCs||_i|_d|_dSNdefault) _state_cls_states_context)r rr r r __init__s zStateManager.__init__namestrrrcCs|pd|_dSr)rr rr r r set_context!zStateManager.set_contextrcCs(|j|jvr||j|j<|j|jSr )rrrr r r r get_state$s  zStateManager.get_statecCs|jdSr )rclearr r r r r)r!zStateManager.resetN)rrrrrr)rrr)rrrrrr r"rr r r r rs    rc@sBeZdZdZdddZdd d ZdddZdddZdddZdS) ModelHookaBase class for model hooks that can override a module's forward. Hooks can intercept the forward pass at two points: - pre_forward: Called before the original forward, can modify args/kwargs - post_forward: Called after the original forward, can modify output Subclasses can override either or both methods. The default implementations pass through args/kwargs/output unchanged. For more complex behavior, override new_forward to completely replace the forward logic. module nn.ModulercC|S)zInitialize the hook when it's registered to a module. Args: module: The module this hook is being attached to. Returns: The module (possibly modified). r r r&r r r initialize_hook; zModelHook.initialize_hookargsrkwargstuple[tuple, dict]cOs||fS)a*Called before the module's forward pass. Args: module: The module being called. *args: Positional arguments to forward. **kwargs: Keyword arguments to forward. Returns: Tuple of (args, kwargs) to pass to the forward method. r )r r&r,r-r r r pre_forwardFs zModelHook.pre_forwardoutputcCs|S)zCalled after the module's forward pass. Args: module: The module that was called. output: The output from the forward method. Returns: The (possibly modified) output. r )r r&r0r r r post_forwardSs zModelHook.post_forwardcOs8|j|g|Ri|\}}|j|i|}|||S)aOverride the module's forward pass completely. The default implementation calls pre_forward, then the original forward, then post_forward. Override this method for more complex behavior. Args: module: The module being called. *args: Positional arguments to forward. **kwargs: Keyword arguments to forward. Returns: The output of the forward pass. )r/_original_forwardr1)r r&r,r-r0r r r new_forward_s zModelHook.new_forwardcCr()zReset any state associated with this hook. Args: module: The module this hook is attached to. Returns: The module. r r)r r r reset_stateqr+zModelHook.reset_stateN)r&r'rr')r&r'r,rr-rrr.)r&r'r0rrr)r&r'r,rr-rrr) rrrrr*r/r1r3r4r r r r r%-s r%c@s$eZdZUdZded<d ddZd S) _WrappedForwardz>Wrapper that intercepts forward calls and dispatches to hooks.r'r&r,rr-cOs>t|jdd}|dus|js|jj|i|S|j|i|S)N_hook_registry)getattrr&_hooksr2dispatch)r r,r-registryr r r __call__sz_WrappedForward.__call__N)r,rr-r)rrrr__annotations__r;r r r r r5}s r5c@sZeZdZdZdddZedddZdddZd ddZd!ddZ d"ddZ d ddZ dS)# HookRegistryzRegistry of hooks attached to a module. Manages multiple hooks that can intercept a module's forward pass. Hooks are called in sorted order by name for determinism. r&r'cCs||_i|_dSr )r&r8r)r r r rs zHookRegistry.__init__rcCsHt|dd}|dur"||}t|d|t|ds"|j|_t||_|S)zGet existing registry or create a new one for the module. Args: module: The module to get/create a registry for. Returns: The HookRegistry for this module. r6Nr2)r7setattrhasattrforwardr2r5)clsr&r:r r r get_or_creates    zHookRegistry.get_or_createrrhookr%rcCs||j||j|<dS)zRegister a hook with the given name. Args: name: Unique name for this hook. hook: The hook instance to register. N)r*r&r8r rrCr r r register_hooks zHookRegistry.register_hookcCs||jvr |j|=dSdS)z`Remove a hook by name. Args: name: The name of the hook to remove. N)r8rr r r remove_hooks  zHookRegistry.remove_hookModelHook | NonecCs |j|S)zGet a hook by name. Args: name: The name of the hook. Returns: The hook if found, None otherwise. )r8getrr r r get_hooks zHookRegistry.get_hookr,rr-cOs|js |jj|i|St|jdkr)tt|j}|j|jg|Ri|St|j ddd}|D]\}}|j |jg|Ri|\}}q6|jj|i|}t |D] \}}| |j|}qW|S)aDispatch a forward call through registered hooks. Currently supports a single active hook. Multiple hooks are called in sorted order by name, with each hook's output passed to the next. Args: *args: Positional arguments to forward. **kwargs: Keyword arguments to forward. Returns: The output of the forward pass. cSs|dS)Nrr )xr r r sz'HookRegistry.dispatch..)key) r8r&r2lennextitervaluesr3sorteditemsr/reversedr1)r r,r-rC sorted_hooks_r0r r r r9s   zHookRegistry.dispatchcCs(|j|}|dur||jdSdS)zfReset a hook's state by name. Args: name: The name of the hook to reset. N)r8rHr4r&rDr r r reset_hooks zHookRegistry.reset_hookN)r&r')r&r'rr=)rrrCr%rrr$)rrrrG)r,rr-rrr) rrrrr classmethodrBrErFrIr9rWr r r r r=s    &r=)r __future__rcollections.abcr dataclassesrtypingrtorch.nnnnrrr%r5r=r r r r s     P