o پie @sddlmZddlZdejdedeejejffddZdejdedeejejejffd d Zdejd ed ed edef ddZdejded ed edededeejejejffddZ dgZ dS))TupleNweight num_packsreturnc sL|j\}}||dks J||dkr2tj|dtj|jd|j}tj|tjd}||fS|j dddj }tj |dtjdd }tj |dd }t |D]N}dg|}dg|||D]=} tfd d t |D|jd } | ks~J| ||| f<| ||| f<|| ||| f7<| d7<qcqS||fS)a Pack n weighted objects to m packs, such that each bin contains exactly n/m objects and the weights of all packs are as balanced as possible. Parameters: weight: [X, n], the weight of each item num_packs: number of packs Returns: pack_index: [X, n], the pack index of each item rank_in_pack: [X, n], the rank of the item in the pack rdtypedevice)r T) descendingcpu) fill_valuer r )r c3s |] }|kr|VqdS)N).0igroups_per_pack pack_itemsr\/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/eplb/eplb_algorithms/deepseek.py )sz#balanced_packing..)key)shapetorcharangesizeint64r expand zeros_likefloatsortindicesr full_likerangemin __getitem__) rr num_layers num_groups pack_index rank_in_packr r pack_weightsgrouppackrrrbalanced_packings:       r,num_phyc Cs|j\}}||}|dksJ|j}tj|tj|d|d}tj||tj|d}tj||tj|d}tj|tj|d} t||D])} ||j ddj } | |dd| f<|| | f|dd| f<|| | fd7<qA|||fS)a Replicate `num_log` experts to `num_phy` replicas, such that the maximum load of all replicas is minimized. Parameters: weight: [X, num_log] num_phy: total number of experts after replication Returns: phy2log: [X, num_phy], logical expert id of each physical expert rank: [X, num_phy], the replica rank logcnt: [X, num_log], number of replicas for each logical expert rrrr)dimN) rr rrrrepeatzerosonesr"maxr ) rr-nnum_log num_redundantr phy2logranklogcntarangenrredundant_indicesrrrreplicate_experts4s   r;num_physical_expertsr& num_nodesnum_gpuscCs|j\}}||dks J||}||dksJ||}||dks%J||dks-J||} dtjdtjfdd} |d||fd} t| |\} } | || |dtj|tj| j d d}| |}| d| d||}t |||\}}}|| d|}t|||\}}|| |}| |}| d|}| ||dtjd|||| j d  d dd  d}| d|}| d| |d}| |d d|}|||fS) aC Parameters: weight: [num_moe_layers, num_logical_experts] num_physical_experts: number of physical experts after replication num_groups: number of expert groups num_nodes: number of server nodes, where the intra-node network (e.g, NVLink) is faster num_gpus: number of GPUs, must be a multiple of `num_nodes` Returns: physical_to_logical_map: [num_moe_layers, num_physical_experts] logical_to_physical_map: [num_moe_layers, num_logical_experts, X] logical_count: [num_moe_layers, num_logical_experts] rpermrc Ss:t|}|d|tj|dtj|jd|j|S)Nrr) r empty_likescatter_rrrr rr)r?invrrrinverseps z/rebalance_experts_hierarchical..inverserr)r r)rrTensor unflattensumr, unsqueezerrr flattengatherviewr;)rr<r&r=r>r%num_logical_experts group_sizegroups_per_nodephy_experts_per_gpurCtokens_per_groupgroup_pack_indexgroup_rank_in_packlog2mlogmlog2logtokens_per_mlogphy2mlogphyrankmlogcnttokens_per_phyr'r(phy2pphypphy2phy pphy2mlogpphy2logpphyrankr8rrrrebalance_experts_hierarchicalSs\         r_ num_replicasenable_hierarchicalc Cs|j\}}|}|rt|||||\}} } n t||dd|\}} } | } tj||| fdtj| j d} | |d d|| | tj |tj| j d |d|| | fS)a Entry point for expert-parallelism load balancer. Parameters: weight: [layers, num_logical_experts], the load statistics for all logical experts num_replicas: number of physical experts, must be a multiple of `num_gpus` num_groups: number of expert groups num_nodes: number of server nodes, where the intra-node network (e.g, NVLink) is faster num_gpus: number of GPUs, must be a multiple of `num_nodes` Returns: physical_to_logical_map: [layers, num_replicas], the expert index of each replica logical_to_physical_map: [layers, num_logical_experts, X], the replica indices for each expert expert_count: [layers, num_logical_experts], number of physical replicas for each logical expert rrr)rrr r_r2itemrfullrr rKrArr) rr`r&r=r>rar%rLr6rWr8 maxlogcntlog2phyrrrrebalance_expertss0         rf) typingrrrEintr,r;r_boolrf__all__rrrrsV  -  U 5