o ٷi/ @sddlZddlmZddlmZddlmZmZddlm Z ddl m Z m Z ddl mZeeZiddd d d d d ddddddddddddddddddd d!d"d#d$d%d&d'd(d)d*d+Zed,d-eDZhd.Zd'hZd/e d0ejfd1d2Zd/e d0dfd3d4Zidd5d d6d d7dd8dd9dd:dd;dd<dd=dd>dd?dd>d!d@d%dAd'dBdCdDdEdFZdGdHdIdJdKdLdMdNdOZd/e dPefdQdRZd/e fdSdTZd/e fdUdVZdS)WN) init_logger)_LazyRegisteredModel_ModelRegistry)OmniDiffusionConfig)SequenceParallelConfigget_sp_plan_from_model)apply_sequence_parallelQwenImagePipeline) qwen_imagepipeline_qwen_imager QwenImageEditPipeline)r pipeline_qwen_image_editr QwenImageEditPlusPipeline)r pipeline_qwen_image_edit_plusrQwenImageLayeredPipeline)r pipeline_qwen_image_layeredrGlmImagePipeline) glm_imagepipeline_glm_imagerZImagePipeline)z_imagepipeline_z_imagerOvisImagePipeline) ovis_imagepipeline_ovis_imager WanPipeline)wan2_2pipeline_wan2_2 Wan22PipelineStableAudioPipeline) stable_audiopipeline_stable_audiorWanImageToVideoPipeline)rpipeline_wan2_2_i2vWan22I2VPipelineLongCatImagePipeline) longcat_imagepipeline_longcat_imager% BagelPipeline)bagelpipeline_bagelr(LongCatImageEditPipeline)r&pipeline_longcat_image_editr+StableDiffusion3Pipeline)sd3 pipeline_sd3r-HunyuanImage3ForCausalMM)hunyuan_image_3pipeline_hunyuan_image_3HunyuanImage3PipelineFlux2KleinPipeline) flux2_kleinpipeline_flux2_kleinr4NextStep11Pipeline) nextstep_1_1pipeline_nextstep_1_1r7)flux pipeline_flux FluxPipeline)omnigen2pipeline_omnigen2OmniGen2Pipeline)r<r?cCs0i|]\}\}}}|td|d||dqS)vllm_omni.diffusion.models..) module_name class_name)r).0 model_arch mod_folder mod_relnamecls_namerIP/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/diffusion/registry.py us rK>rr7r- od_configreturncCst|j}|dur{||d}|jj}|dkr%|jtvr%td|tt|dkr:|jtvr:|j s:t d|d|_ t |j drE|j |j _t |j drP|j |j _|dkrtt |d rt|jtvrt|j rtd d lm}d d lm}||||d t|||Std|jd)aInitialize a diffusion model from the registry. This function: 1. Loads the model class from the registry 2. Instantiates the model with the config 3. Configures VAE optimization settings 4. Applies sequence parallelism if enabled (similar to diffusers' enable_parallelism) Args: od_config: The OmniDiffusion configuration. Returns: The initialized pipeline model. Raises: ValueError: If the model class is not found in the registry. N)rLz]vae_patch_parallel_size=%d is set but VAE patch parallelism is only enabled for %s; ignoring.zNvae_patch_parallel_size=%d requires vae_use_tiling; automatically enabling it.T use_slicing use_tilingvaer) get_dit_group),maybe_wrap_vae_decode_with_patch_parallelism)vae_patch_parallel_size group_getterz Model class z' not found in diffusion model registry.)DiffusionModelRegistry_try_load_model_clsmodel_class_nameparallel_configrT_VAE_PATCH_PARALLEL_ALLOWLISTloggerwarningsortedvae_use_tilinginfohasattrrQvae_use_slicingrOrP.vllm_omni.diffusion.distributed.parallel_staterR2vllm_omni.diffusion.distributed.vae_patch_parallelrS#_apply_sequence_parallel_if_enabled ValueError)rL model_classmodel vae_pp_sizerRrSrIrIrJinitialize_modelsN           ric CsHz|jj}|dkr WdSgd}d}|D]`}t||sqt||}|dur&qt|}|dur/qt|jj|jjd}|jdkrE|jdkrEdn|jdkrLdnd} t d |j j d |d |d | d |jd|jd t ||||d7}q|dkrt d|dWdSWdSty} zt d| dWYd} ~ dSd} ~ ww)aApply sequence parallelism hooks if SP is enabled. This is the centralized location for enabling SP, similar to diffusers' ModelMixin.enable_parallelism() method. It applies _sp_plan hooks to transformer models that define them. Note: Our "Sequence Parallelism" (SP) corresponds to "Context Parallelism" (CP) in diffusers. We use _sp_plan instead of diffusers' _cp_plan. Args: model: The pipeline model (e.g., ZImagePipeline). od_config: The OmniDiffusion configuration. rNN) transformer transformer_2ditunetr)ulysses_degree ring_degreehybridulyssesringz!Applying sequence parallelism to z (z ) (sp_size=z, mode=z , ulysses=z, ring=)z)Sequence parallelism is enabled (sp_size=zs) but no transformer with _sp_plan found. SP hooks not applied. Consider adding _sp_plan to your transformer model.z&Failed to apply sequence parallelism: z. Continuing without SP hooks.)rYsequence_parallel_sizer`getattrrrrnror[r_ __class____name__rr\ Exception) rgrLsp_sizetransformer_attrs applied_countattrrjplan sp_configmodeerIrIrJrdsZ       rd get_qwen_image_post_process_func%get_qwen_image_edit_post_process_func*get_qwen_image_edit_plus_post_process_funcget_glm_image_post_process_funcget_post_process_func get_ovis_image_post_process_funcget_wan22_post_process_func"get_stable_audio_post_process_funcget_wan22_i2v_post_process_func#get_longcat_image_post_process_funcget_bagel_post_process_funcget_sd3_image_post_process_func!get_flux2_klein_post_process_func get_nextstep11_post_process_funcr<get_flux_post_process_funcr?get_omnigen2_post_process_funcget_glm_image_pre_process_func$get_qwen_image_edit_pre_process_func)get_qwen_image_edit_plus_pre_process_func'get_longcat_image_edit_pre_process_func'get_qwen_image_layered_pre_process_funcget_wan22_pre_process_funcget_wan22_i2v_pre_process_funcget_omnigen2_pre_process_func)rr rr+rrr"r? func_namecCs<t|j\}}}d|d|}t|}t||}||S)z?Load and return a process function from the appropriate module.r@rA)_DIFFUSION_MODELSrX importlib import_moduleru)rLrrFrG_rBmodulefuncrIrIrJ_load_process_func<s   rcC"|jtvrdSt|j}t||SN)rX_DIFFUSION_POST_PROCESS_FUNCSrrLrrIrIrJget_diffusion_post_process_funcE   rcCrr)rX_DIFFUSION_PRE_PROCESS_FUNCSrrrIrIrJget_diffusion_pre_process_funcLrr)rtorch.nnnn vllm.loggerr#vllm.model_executor.models.registryrrvllm_omni.diffusion.datar'vllm_omni.diffusion.distributed.sp_planrr+vllm_omni.diffusion.hooks.sequence_parallelrrwr[ritemsrVrZ_NO_CACHE_ACCELERATIONModulerirdrrstrrrrrIrIrIrJs      %*/49>CHMRWd   IB