o ٷil @srddlmZddlmZddlmZmZddlmZddl m Z ddl m Z m Z ddlmZeeZdddZdS)) annotations) init_logger)NoParallelAttentionParallelAttentionStrategy)RingParallelAttention)UlyssesParallelAttention) get_sequence_parallel_world_size get_sp_group)get_forward_context scatter_idxint gather_idxuse_syncboolreturnrc Cs0z tj}|j}Wnty&}ztd|tWYd}~Sd}~wwt|dd}t|dd}zt}t dkr@tWSWn,tym}z |dksQ|dkr`t d|d|d|d tWYd}~Sd}~ww|dkrtd |d t ||||d S|dkrtd |d t |dStS)a&Select a parallel attention strategy based on current diffusion config. Design principle: - Attention kernel backend selection remains in `attention/selector.py`. - Parallel attention selection is handled here, based on distributed config and initialized process groups. z>No forward context available for parallel attention strategy: Nulysses_degree ring_degreezSP configured (ulysses=z, ring=z) but SP group not available: zH. Falling back to NoParallelAttention. This may cause incorrect results.z/Using UlyssesParallelAttention (ulysses_degree=))sp_groupr r rz)Using RingParallelAttention (ring_degree=)r) r omni_diffusion_configparallel_config Exceptionloggerdebugrgetattrr rwarningrr) r r rcfgperrrr b/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/diffusion/attention/parallel/factory.py!build_parallel_attention_strategysJ      r"N)r r r r rrrr) __future__r vllm.loggerr+vllm_omni.diffusion.attention.parallel.baserr+vllm_omni.diffusion.attention.parallel.ringr.vllm_omni.diffusion.attention.parallel.ulyssesr.vllm_omni.diffusion.distributed.parallel_staterr #vllm_omni.diffusion.forward_contextr __name__rr"r r r r!s