o ٷi*@sdZddlmZddlmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZmZdd lmZmZmZGd d d eZdddZdS)a; Hook-based TeaCache implementation for vLLM-Omni. This module implements a diffusers-style hook system that completely intercepts the transformer forward pass, eliminating the need for any TeaCache-specific code in model definitions. Model developers only need to add an extractor function to support new models. ) annotations)AnyN)TeaCacheConfig) get_extractor) TeaCacheState)!get_classifier_free_guidance_rank'get_classifier_free_guidance_world_size) HookRegistry ModelHook StateManagercsNeZdZdZdZdfdd Zdd d ZdddZdddZdddZ Z S) TeaCacheHookab ModelHook implementing TeaCache for transformer models. This hook completely intercepts the transformer's forward pass and implements adaptive caching based on timestep embedding similarity. It's model-agnostic and supports multiple model types through extractor functions. Key features: - Zero changes to model code - CFG-aware with separate states for positive/negative branches - CFG-parallel compatible: properly detects branch identity across ranks - Model-specific polynomial rescaling - Auto-detection of model types Attributes: config: TeaCache configuration with thresholds and callbacks rescale_func: Polynomial function for rescaling L1 distances state_manager: Manages TeaCacheState across forward passes extractor_fn: Model-specific function to extract modulated input teacacheconfigrcs8t||_t|j|_tt|_ d|_ d|_ dS)zl Initialize TeaCacheHook. Args: config: TeaCache configuration object. Nr) super__init__rnppoly1d coefficients rescale_funcr r state_manager extractor_fn _forward_cnt)selfr __class__[/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/diffusion/cache/teacache/hook.pyr6s   zTeaCacheHook.__init__moduletorch.nn.ModulereturncCst|jj|_|jd|S)z Initialize hook with extractor from config transformer model type. Args: module: The module to initialize the hook for. Returns: The initialized module. r )rrtransformer_typerr set_contextrrrrrinitialize_hookDs zTeaCacheHook.initialize_hookargsrkwargscOs|j|g|Ri|}t|ddr1t}|dkr%t}|dkr"dnd}n|jddkr.dnd}nd}d|}|j||j} || |j } | sm| j d urm|j | j |_ | j d uri|j d uri|j | j |_ |j } nA|j } |j d ur||j nd } |}|d|_ t|dkr|j d ur|d|_ |j | | _ | d ur|j | | _ |j } |j | _| jd7_|jd7_|| S) aJ Generic forward handler that works for ANY model. This method is completely model-agnostic. All model-specific logic is encapsulated in the extractor function that returns a CacheContext. The extractor does: - Model-specific preprocessing - Extraction of modulated input for cache decision - Providing transformer execution callable - Providing postprocessing callable This hook does: - CFG-aware state management - Cache decision logic (generic) - Residual caching and reuse Args: module: Transformer module (any architecture) *args: Positional arguments for model forward **kwargs: Keyword arguments for model forward Returns: Model output (format depends on model) do_true_cfgFrnegativepositive teacache_N)rgetattrrrrrr! get_state _should_compute_full_transformermodulated_inputprevious_residual hidden_statesprevious_residual_encoderencoder_hidden_statesclonerun_transformer_blockslendetachprevious_modulated_inputcnt postprocess)rrr$r%ctxcfg_parallel_sizecfg_rank cache_branch context_namestateshould_computeoutputori_hidden_statesori_encoder_hidden_statesoutputsrrr new_forwardWs@         zTeaCacheHook.new_forwardr@r modulated_inp torch.TensorboolcCs|jdkr d|_dS|jdurdS||j|jd}t||}|jt|7_|j|j j kr@dSd|_dS)a Determine whether to compute full transformer or reuse cached residual. This implements the core TeaCache algorithm: 1. Always compute first timestep 2. For intermediate steps: - Compute relative L1 distance between current and previous modulated inputs - Apply polynomial rescaling with model-specific coefficients - Accumulate rescaled distances - Compare to threshold: below = cache, above = compute Args: state: Current TeaCacheState containing counters and cached values modulated_inp: Modulated input extracted from first transformer block Returns: True to compute full transformer, False to reuse cached residual rgTNg:0yE>F) r9accumulated_rel_l1_distancer8absmeancpuitemfloatrr rel_l1_thresh)rr@rG rel_distancerescaled_distancerrrr.s"   z-TeaCacheHook._should_compute_full_transformercCs|jd|_|S)z Reset all cached states for a new inference run. Args: module: The module to reset state for. Returns: The module with reset state. r)rresetrr"rrr reset_states zTeaCacheHook.reset_state)rr)rrrr)rrr$rr%rrr)r@rrGrHrrI) __name__ __module__ __qualname____doc__ _HOOK_NAMErr#rFr.rT __classcell__rrrrr s   b1r rrrrrNonecCs$t|}t|}|tj|dS)a Apply TeaCache optimization to a transformer module. This function registers a TeaCacheHook that completely intercepts the module's forward pass, implementing adaptive caching without any changes to the model code. Args: module: Transformer model to optimize (e.g., QwenImageTransformer2DModel) config: TeaCacheConfig specifying caching parameters Example: >>> config = TeaCacheConfig( ... rel_l1_thresh=0.2, ... transformer_type="QwenImageTransformer2DModel" ... ) >>> apply_teacache_hook(transformer, config) >>> # Transformer bound to the pipeline now uses TeaCache automatically, ... # no code changes needed! N)r get_or_creater register_hookrY)rrregistryhookrrrapply_teacache_hooks r`)rrrrrr[)rX __future__rtypingrnumpyrtorch)vllm_omni.diffusion.cache.teacache.configr-vllm_omni.diffusion.cache.teacache.extractorsr(vllm_omni.diffusion.cache.teacache.stater.vllm_omni.diffusion.distributed.parallel_staterrvllm_omni.diffusion.hooksr r r r r`rrrrs    \