o پi @sUddlZddlZddlmZddlmZmZmZddlZddl Z ddl m Z GdddZ da ee ed<dd Zd e fd d ZdS) N)Path)AnyListOptional)MultiprocessingSerializerc@sreZdZdededefddZddZdd Z dd ej d e ej d efddZ de de e fddZ ddZdS)NaiveDistributedrank world_size rendezvouscCsT||_||_d|_t||_|jjdddd|kr!|ks$JJ|dS)NrT)parentsexist_ok)_rank _world_size_operation_indexr _directorymkdirbarrier)selfrr r r\/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/distributed/naive_distributed.py__init__ s  zNaiveDistributed.__init__cC|jSN)r rrrrget_rankzNaiveDistributed.get_rankcCrr)rrrrrget_world_sizerzNaiveDistributed.get_world_sizertensor scatter_listsrccs|jkrt||jksJn|dusJ||jkr*tfddt|Ddnt}|d|j}|jkrH|dusBJ||j}nt|}||| dS)Ncs&g|]\}}|kr dnt|qSr)r serialize).0 item_rankitemrrr (s z,NaiveDistributed.scatter..)serialized_scatter_listr&) r lenrall_gather_objectdict enumerater deserializecopy_r)rrrrgathered_objectsremote_serialized_tensor remote_tensorrr$rscatters.           zNaiveDistributed.scatterobjreturncsvjd7_ddtffdd jtt|ddtffdd fd d t j DS) N interesting_rankcsjd|djdS)Nr_opz.txt)rr)r5rrr _get_pathIsz5NaiveDistributed.all_gather_object.._get_pathzutf-8csR|} |r#|}r#ttj|dt ddSt dq)NT)validategMbP?) exists read_textendswithpickleloadspybase64 b64decoder'timesleep)r5ptext)r7 text_postfixrr _read_oneSs z5NaiveDistributed.all_gather_object.._read_onecsg|]}|qSrr)r!r5)rErrr%\sz6NaiveDistributed.all_gather_object..) rintr write_textr> b64encoder<dumpsdecoderanger)rr1r)r7rErrDrr(Ds  z"NaiveDistributed.all_gather_objectcCs0||j}|tt|jksJd|dS)Nz actual_objs=)r(r listrKr)r actual_objsrrrr`s $zNaiveDistributed.barrierN)r)__name__ __module__ __qualname__rFstrrrrtorchTensorrr0rr(rrrrrr s  & r _instancecCstdusJtSrrTrrrrget_naive_distributedis rVinstancecCstdusJ|adSrrU)rWrrrset_naive_distributedns rX)r<r@pathlibrtypingrrrr>rRsglang.srt.utilsrrrT__annotations__rVrXrrrrs   Z