o ÔÙ¾ijã @s¾ddlmZmZmZmZddlZddlZddlmZdej dej fdd„Z ddej d e dej fd d „Z ddej d e d e deej fdd„Z ddeeeeej effde fdd„ZdS)é)ÚAnyÚDictÚOptionalÚUnionNé)Ú get_tp_groupÚinput_ÚreturncCs tƒ |¡S)z8All-reduce the input tensor across model parallel group.)rÚ all_reduce)r©r ú[/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/distributed/communication_op.pyÚ tensor_model_parallel_all_reduce s r éÿÿÿÿÚdimcCstƒ ||¡S)z8All-gather the input tensor across model parallel group.)rÚ all_gather)rrr r r Ú tensor_model_parallel_all_gathersrÚdstcCstƒ |||¡S)z4Gather the input tensor across model parallel group.)rÚgather)rrrr r r Útensor_model_parallel_gathersrÚ tensor_dictÚsrccCstj ¡s|Stƒ ||¡S)N)ÚtorchÚ distributedÚis_initializedrÚbroadcast_tensor_dict)rrr r r rs r)r)rr)Nr)ÚtypingrrrrrÚtorch.distributedÚparallel_staterÚTensorr Úintrrrr r r r Ús: ÿÿÿ þÿÿÿÿ þÿÿÿ