o پi @sddlZddlZddlmZddlZddlmZddlm Z ej ddZ dej fd d Z defd d Zdej fd dZdej dededefddZdej dej ddfddZd'ddZdefddZdefddZdeedejd ejdej fd!d"Zd'd#d$Zd'd%d&ZdS)(N)Sequence)env)gen_nvshmem_modulecCst}d}ddg}|D]}|D]}||}|r|}nq|dur%nq |dur1td|tj|tjdt}|S)Nzlibnvshmem_host.sozlibnvshmem_host.so.3z=Could not find libnvshmem_host.so or libnvshmem_host.so.3 in )mode) jit_envget_nvshmem_lib_dirsexistsFileNotFoundErrorctypesCDLL RTLD_GLOBALrbuild_and_load)lib_dirslib_path lib_nameslib_dirlib_namecandidate_pathmodulerK/home/ubuntu/.local/lib/python3.10/site-packages/flashinfer/comm/nvshmem.pyget_nvshmem_module s( rreturncCst}t||SN)alloc_empty_unique_idrnvshmem_get_unique_id)uidrrr get_unique_id&s rcC tSr)rnvshmem_unique_id_sizerrrrunique_id_size, r!cCstjttjddS)Ncpu)dtypedevice)torchzerosr!uint8rrrrr0srrrank world_sizecCst|||}tj|Sr)r nvshmem_initr&cuda synchronize)rr)r*statusrrrinit4s r/destsourcecCst||Sr)rnvshmem_alltoall)r0r1rrralltoall:r3cCstjtdSr)r&r,r-rnvshmem_finalizerrrrfinalize>s r6cCrr)r nvshmem_my_perrrrmy_peCr"r8cCrr)r nvshmem_n_pesrrrrn_pesGr"r:shaper$r%cCst|||}t|S)aAllocates memory using NVSHMEM collective malloc operation. This is a collective operation that requires participation by all PEs (Processing Elements). All participants must call this function with the same parameters. Note: This tensor should be explicitly deleted (del tensor) to ensure proper ordering of nvshmem_free operations rather than relying on garbage collection. Args: shape: The shape of the tensor to allocate. dtype: The data type of the tensor. device: The device to allocate the tensor on. Returns: A tensor allocated using NVSHMEM collective malloc. Reference: https://docs.nvidia.com/nvshmem/api/gen/api/memory.html#nvshmem-malloc-nvshmem-free-nvshmem-align )rnvshmem_mallocr& from_dlpack)r;r$r%outputrrrmallocKs r?cCtdSr)rnvshmem_barrier_allrrrr barrier_allhr4rBcCr@r)r%nvshmem_barrier_all_on_current_streamrrrrbarrier_all_on_current_streamlr4rD)rN)r functoolstypingrr&jitrrjit.commrcacherTensorrintr!rr/r3r6r8r:r$r%r?rBrDrrrrs6