o پi/ @sddlZddlZddlmZddlZddlmZddlmZz ddl m Z dZ Wne y<Z ze Z WYdZ [ ndZ [ wweeZGdddeZdS)N)List)DisaggregationMode)MooncakeTransferEngine)TransferEnginec@sReZdZdededefddZddd Zd eed eefd d Z e ddZ dS)AscendTransferEnginehostnamenpu_iddisaggregation_modecCstdur tdtt|_||_||_td|_ |t j kr$d|_ n|t j kr-d|_ ntd|td||jd|j|_|dS)NzSPlease install memfabric_hybrid, for details, see docs/backend/pd_disaggregation.mdASCEND_MF_STORE_URLPrefillDecodez Unsupported DisaggregationMode: :) import_errorloggerwarningrenginerrosgetenv store_urlrPREFILLroleDECODEerror ValueError get_rpc_port session_id initialize)selfrrr rd/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/disaggregation/ascend/transfer_engine.py__init__s"    zAscendTransferEngine.__init__returnNcsddlm}m}|}|dus|dkrtjj}n#tjj} tj dddfddt |D}tj j ||j d  |j|j|j|j|j|}|dkrXtd td dS) Nr)$get_tensor_model_parallel_world_size get_tp_groupsdmanpu)devicecsg|]}tqSr)torch empty_like).0_ tmp_tensorrr Bsz3AscendTransferEngine.initialize..)groupz-Ascend Transfer Engine initialization failed.)sglang.srt.layers.dp_attentionr"r#_get_transfer_protocolrTransDataOpTypeSDMA DEVICE_RDMAr(zerosrange distributed all_gather device_grouprrrrrrrr RuntimeError)rr"r#transfer_protocol trans_op_typeoutput_tensor_list ret_valuerr,rr5s*    zAscendTransferEngine.initializeptrslengthscCsLz |j||}Wn tyd}Ynw|dkr$td|ddSdS)Nrz#Ascend memory registration for ptr z failed.)rbatch_register_memory Exceptionrdebug)rr?r@r>rrrbatch_registerRs z#AscendTransferEngine.batch_registercCs8td}ddh}|r||vr|StddS)NASCEND_MF_TRANSFER_PROTOCOL device_rdmar$zBInvalid or no transfer protocol specified, using default protocol.)rrlowerrr)protocolallowed_protocolsrrrr1[s z+AscendTransferEngine._get_transfer_protocol)r!N) __name__ __module__ __qualname__strintrr rrrE staticmethodr1rrrrrs  r)loggingrtypingrr(sglang.srt.disaggregation.utilsrDsglang.srt.distributed.device_communicators.mooncake_transfer_enginermemfabric_hybridrr ImportErrore getLoggerrKrrrrrrs