o [۷i @s^ddlZddlZddlmZddlmZddlmZddlm Z m Z ddl Z GdddZ dS)N)abstractmethod)contextmanager)CodeType)CallableListc@s\eZdZdZddedefddZddZed d Z d e d e fd dZ e de fddZdS)'TorchCompileWrapperWithCustomDispatchera A wrapper class for torch.compile, with a custom dispatch logic. Subclasses should: 1. Implement the forward method 2. Implement the dispatch logic in the __call__ method It can use `self.compiled_codes` to access the compiled bytecode, and `with self.dispatch_to_code(index):` to dispatch to the compiled code. 3. Implement the `__init__` method to determine how to call `torch.compile` over the forward method. Tcompiled_callableuse_custom_dispatchercCs2||_|jjj|_g|_tjj |j ||_ dSN) r __class__forward__code__original_code_objectcompiled_codestorch_dynamo convert_frameregister_bytecode_hook bytecode_hookr )selfrr rH/home/ubuntu/vllm_env/lib/python3.10/site-packages/depyf/optimization.py__init__s   z0TorchCompileWrapperWithCustomDispatcher.__init__cOs|j|i|S)zImplement the dispatch logic here, beyond the torch.compile level. NOTE: this function can have additional arguments beyond the forward method, for directly dispatching to the compiled code. )rrargskwargsrrr__call__ sz0TorchCompileWrapperWithCustomDispatcher.__call__cOsdSr rrrrrr 'sz/TorchCompileWrapperWithCustomDispatcher.forwardold_codenew_codecCs||jurdSt} |j}|jj}|jjtj j d}|dkr'|dkr'nq |j d}|j|ks4J|j d|ur=dS|j |dS)z8Hook to save the compiled bytecode for direct execution.NT_compilezconvert_frame.pyframer)rsys _getframef_backf_codeco_name co_filenamesplitospathsepf_localsrappend)rrrr! code_name file_namerrrr+s  z5TorchCompileWrapperWithCustomDispatcher.bytecode_hookindexccs(|j||jj_dV|j|jj_dS)a Context manager to dispatch to the compiled code. Why does this work? Because Dynamo guarantees that the compiled bytecode has exactly the same arguments, cell variables, and free variables as the original code. Therefore we can directly switch the code object in the function and call it. See https://dev-discuss.pytorch.org/t/what-is-the-relationship-requirement-among-original-bytecode-transformed-bytecode-and-bytecode-returned-by-hooks-in-dynamo/1693/7 for more details. N)rr r r r)rr0rrrdispatch_to_code>s z8TorchCompileWrapperWithCustomDispatcher.dispatch_to_codeN)T)__name__ __module__ __qualname____doc__rboolrrrr rrrintr1rrrrr s  r) r)r"abcr contextlibrtypesrtypingrrrrrrrrs