o ۷ia#@s ddlZddlZddlZddlmZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZmZddl m!Z!ddl"m#Z#ddl$m%Z%erw e e&Z'GdddeZ(dS)N) TYPE_CHECKING) VllmConfig) init_logger)MULTIMODAL_REGISTRYMultiModalRegistry)cached_tokenizer_from_config) init_tracer)(maybe_register_config_serialize_by_value) UsageContext)deprecate_kwargs)AsyncLLM)EngineCoreClient)Executor)StatLoggerFactoryStatLoggerManager)AsyncOmniEngineArgs)OmniInputProcessor)MultimodalOutputProcessorc@seZdZdZejedddddddf deded e e d e d ed e d e de de de edBdeeefdBdededdfddZeeddddejdddddddf dedede d ede edBde de deeefdBdedede ddfddZdS) AsyncOmniLLMaAsync single-stage LLM engine for use within a stage worker process. This class extends the base vLLM AsyncLLM class with omni-specific processors for handling multimodal inputs and outputs. It is used internally by AsyncOmniStage workers and should not be instantiated directly by users. Args: engine_args: AsyncOmniEngineArgs containing engine configuration vllm_config: Global vLLM configuration executor_class: Executor implementation class, e.g. MultiprocExecutor log_stats: Whether to log statistics usage_context: Usage context of the LLM (default: ENGINE_CONTEXT) mm_registry: Multi-modal registry for processing multimodal inputs use_cached_outputs: Whether to use cached outputs log_requests: Whether to log requests start_engine_loop: Whether to start the engine loop automatically stat_loggers: Customized stat loggers for the engine. If not provided, default stat loggers will be used. Note: Stat logger interface may change in V1. client_addresses: Optional dictionary mapping client names to addresses client_count: Total number of clients (default: 1) client_index: Index of this client (default: 0) FTNr engine_args vllm_configexecutor_class log_stats usage_context mm_registryuse_cached_outputs log_requestsstart_engine_loop stat_loggersclient_addresses client_count client_indexreturncCst|j|_||_|j|_||_|p| du|_|s#| dur#td|jjr*d}nt |jd}t ||d|_ t ||j|j d|_|jjdurRtd|jj}||j_t|_d|_tj|||j| | | d|_d|_|jr~t||jj| || d |_|jd|_z t|Wn t yYnw|j!j"d kr|j!j#s|j!j$}td |t%&d t'(d }t)j"j*t)j"j+j,g|j!j-t)j"j.|||j!j/dd|_"dSd|_"dS)a Create an AsyncOmniLLM. Args: vllm_config: global configuration. executor_class: an Executor impl, e.g. MultiprocExecutor. log_stats: Whether to log stats. usage_context: Usage context of the LLM. mm_registry: Multi-modal registry. use_cached_outputs: Whether to use cached outputs. log_requests: Whether to log requests. start_engine_loop: Whether to start the engine loop. stat_loggers: customized stat loggers for the engine. If not provided, default stat loggers will be used. PLEASE BE AWARE THAT STAT LOGGER IS NOT STABLE IN V1, AND ITS BASE CLASS INTERFACE MIGHT CHANGE. Returns: None NzuAsyncLLM created with log_stats=False and non-empty custom logger list; enabling logging without default stat loggers) model_config)rr) tokenizerrengine_core_output_typezvllm.llm_engineF)rrrr r!r")r engine_idxscustom_stat_loggersenable_default_loggersr!torchzJTorch profiler enabled. AsyncOmniLLM CPU traces will be collected under %s_z.async_omni_llm) worker_nameuse_gzip) activities with_stackon_trace_ready)0r r$robservability_configrrloggerinfoskip_tokenizer_initrrinput_processorrengine_output_typeoutput_processorotlp_traces_endpointrtracerasyncio Condition _pause_cond_pausedr make_async_mp_client engine_corelogger_managerrengine_ranks_managedlog_engine_initializedoutput_handlerget_running_loop_run_output_handler RuntimeErrorprofiler_configprofilerignore_frontendtorch_profiler_dirsocket gethostnameosgetpidr*profileProfilerActivityCPUtorch_profiler_with_stacktensorboard_trace_handlertorch_profiler_use_gzip)selfrrrrrrrrrrr r!r"r%r9 profiler_dirr,rWZ/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm_omni/entrypoints/async_omni_llm.py__init__:s%         zAsyncOmniLLM.__init__disable_log_requestszEThis argument will have no effect. Use `enable_log_requests` instead.)additional_messageenable_log_requestsdisable_log_statsr c Cs&||t||||| ||| | |d S)N) rrrrrrrr r!r"r)r get_class) clsrrrrrr\r]r r!r"rZrWrWrXfrom_vllm_configszAsyncOmniLLM.from_vllm_config)__name__ __module__ __qualname____doc__r ENGINE_CONTEXTrrrtyperboolrlistrdictstrintrY classmethodr r`rWrWrWrXr s            r))r:rMrKtypingrr* vllm.configr vllm.loggerrvllm.multimodalrrvllm.tokenizersr vllm.tracingrvllm.transformers_utils.configr vllm.usage.usage_libr vllm.utils.func_utilsr vllm.v1.engine.async_llmr vllm.v1.engine.core_clientr vllm.v1.executor.abstractrvllm.v1.metrics.loggersrrvllm_omni.engine.arg_utilsr vllm_omni.engine.input_processorr!vllm_omni.engine.output_processorrrar2rrWrWrWrXs0