o پi @sRddlZGdddZGdddeZeZ d dedeeed efd d ZdS) Ncs eZdZdZfddZZS) SingletonNcs,|jstt|j|g|Ri||_|jSN) _instancesuperr__new__)clsargskwargs __class__m/home/ubuntu/.local/lib/python3.10/site-packages/sglang/multimodal_gen/runtime/distributed/parallel_groups.pyr s zSingleton.__new__)__name__ __module__ __qualname__rr __classcell__r r r r rsrc@seZdZddZdS)ProcessGroupSingletoncCsd|_d|_dSr) ULYSSES_PGRING_PG)selfr r r __init__s zProcessGroupSingleton.__init__N)rrrrr r r r rs rTrank sp_groupsuse_ulysses_lowcs||dks Jtfdd|Ds$Jdddd|Dd}d}|}|}d ttd ttd ttfd d } |D]} |rt|D] } tt| || d|} | | | } tj| }|| vrg|}qGt|D]} tt| |} | | | }tj|}||vr|}qlq?t|D] } tt| || d|} | | | }tj|}||vr|}qt|D]} tt| |} | | | } tj| }|| vr|}qq?|t_|t_ dS)aCreate Ulysses/Ring process groups inside each SP group. This is required when TP>1, because SP groups are not necessarily made of consecutive global ranks (e.g., tp-sp order makes SP ranks strided). Args: sp_ulysses_degree: ulysses degree inside SP. sp_ring_degree: ring degree inside SP. rank: global rank of current process. sp_groups: list of global-rank lists for each SP group. use_ulysses_low: keep the same semantics as the original function. rc3s|] }t|kVqdSrlen.0g sp_degreer r .s  z3set_seq_parallel_pg_by_sp_groups..zEach SP group must have size z , got sizes cSsg|]}t|qSr rrr r r 0z4set_seq_parallel_pg_by_sp_groups..Nranksindicesreturncsfdd|DS)Ncsg|]}|qSr r )rir$r r r"9r#zSset_seq_parallel_pg_by_sp_groups.._map_indices_to_ranks..r )r$r%r r(r _map_indices_to_ranks8sz?set_seq_parallel_pg_by_sp_groups.._map_indices_to_ranks) alllistintrangetorch distributed new_group PROCESS_GROUPrr)sp_ulysses_degreesp_ring_degreerrr ulyssess_pgring_pgnum_ulysses_pgs num_ring_pgsr)sp_ranksr'idx ulysses_ranksgroup ring_ranksr rr set_seq_parallel_pg_by_sp_groupss\   "             r>)T)r/rrr2r-r,boolr>r r r r s