o ٷi;@sddlZddlZddlmZddlmZddlZddlm Z ddl m Z ddl m Z ddlmZmZmZddlmZdd lmZmZdd lmZe eZd ed efd dZd ed efddZd ed efddZGdddZ dS)N)Iterable)Any) init_logger)OmniDiffusionConfig)DiffusionExecutor)DiffusionModelRegistryget_diffusion_post_process_funcget_diffusion_pre_process_func)OmniDiffusionRequest)OmniDiffusionSamplingParamsOmniTextPrompt)OmniRequestOutputmodel_class_namereturncC&t|}|dur dStt|ddS)NFsupport_image_inputr_try_load_model_clsboolgetattrr model_clsrX/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/diffusion/diffusion_engine.pysupports_image_input rcCst|}t|ddS)N color_formatRGB)rrrrrrrimage_color_format!s  rcCr)NFsupport_audio_outputrrrrrsupports_audio_output&rr c @seZdZdZdefddZdedeefddZ e d eddfd d Z defd d Z d#de dBddfddZdefddZddZ    d$de dedBdededBdedBdef ddZd%ddZd e ee Bddfd!d"ZdS)&DiffusionEnginez4The diffusion engine for vLLM-Omni diffusion models. od_configc Csr||_t||_t||_t|}|||_z|WdSt y8}zt d|| |d}~ww)zxInitialize the diffusion engine. Args: config: The configuration for the diffusion engine. zDummy run failed: N) r"rpost_process_funcr pre_process_funcr get_classexecutor _dummy_run Exceptionloggererrorclose)selfr"executor_classerrr__init__0s    zDiffusionEngine.__init__requestrc s|jdurt}|t|}td|dd|}|jr,t|jtd|jdurGtdfddt j DSt}|j durV| |jn|j}t|}td|ddt |t sw|duru|gng}tjjtjj|d d }|jdur|d |d <tj d krֈj d } jrjd nd} t|jjrt|d kr|d n|} tj| g| ||jd| iddgStj| || ||jdgSg} d } t j D]`\}} |tjkrj|nd} jj}| t|kr|| | |ng}| |7} t|jjr1t|d kr|d n|} | tj| g| ||jd| iddq| tj| || ||jdq| S)NzPre-processing completed in z.4fz secondsz"Generation completed successfully.z1Output is None, returning empty OmniRequestOutputc s<g|]\}}tj|tjkrj|ndg|iddqS)N request_idimagespromptmetricslatents)r from_diffusionlen request_ids).0ir5r0rr Tsz(DiffusionEngine.step..zPost-processing completed in i) image_num resolutionpostprocess_time_mspreprocessing_time_msrr1audio)r3r4r5r6r7multimodal_outputfinal_output_typer2)r$timer)infoadd_req_and_wait_for_responser*r(outputwarning enumeratepromptsr# isinstancelistintsampling_paramsnum_outputs_per_promptr@r9r:r r"rr r8trajectory_latentsappend)r,r0preprocess_start_timepreprocess_timerJpostprocess_start_timeoutputspostprocess_timer6r5r3 audio_payloadresults output_idxr< num_outputsrequest_outputsrr=rstepEs                    "  zDiffusionEngine.stepconfigcCst|S)zFactory method to create a DiffusionEngine instance. Args: config: The configuration for the diffusion engine. Returns: An instance of DiffusionEngine. )r!)r`rrr make_engines zDiffusionEngine.make_enginecCs |j|SN)r&add_req)r,r0rrrrIs z-DiffusionEngine.add_req_and_wait_for_responseNtrace_filenamec Cs |durdttd}tjdd}tj|}tj|}z tj|ddWnt yB}z t d|d |d}~wwtj ||}|d }t d |t d |z |jd |fdWdSty}zt j dddtd||d}~ww)a Start torch profiling on all diffusion workers. Creates a directory (if needed) and sets up a base filename template for per-rank profiler traces (typically saved as