o i%@sJUddlZddlZddlmZddlmZddlmZddlm Z m Z ddl m Z ddl mZdd lmZmZmZeeZd ed efd d Zd edBfddZd edBfddZd edBfddZd edBfddZd edBfddZeeeeedZd efddZda da!ee"d<eree"d<defdd Z#defd!d"Z$gd#Z%dS)$N)chain) TYPE_CHECKING)envs)PLATFORM_PLUGINS_GROUPload_plugins_by_group)resolve_obj_by_qualname) supports_xccl) CpuArchEnumPlatform PlatformEnumsubstrreturnc CsJddlm}m}z |d}W||vS|y$}ztd|d}~ww)z? Check to see if the vLLM version matches a substring. r)PackageNotFoundErrorversionvllmzqThe vLLM package was not found, so its version could not be inspected. This may cause platform detection to fail.N)importlib.metadatarrloggerwarning)r rr vllm_versionerM/home/ubuntu/vllm_env/lib/python3.10/site-packages/vllm/platforms/__init__.pyvllm_version_matches_substrs rc CsntdtjrtddSz ddl}tdWdSty6}ztdt|WYd}~dSd}~ww)Nz&Checking if TPU platform is available.z7Confirmed TPU platform is available via Pathways proxy.z0tpu_inference.platforms.tpu_platform.TpuPlatformrz$Confirmed TPU platform is available.zvllm.platforms.tpu.TpuPlatformz)TPU platform is not available because: %s)rdebugrVLLM_TPU_USING_PATHWAYSlibtpu Exceptionstr)rrrrrtpu_platform_plugin#s   rc s&d}tdzAddlm}|}|z,|dko td }|dkr,tdtdr5td|rCUDA platform is not available because vLLM is built with CPU.z%Confirmed CUDA platform is available.z1Exception happens when checking CUDA platform: %snvmlrcsjdp jdS)Nz/etc/nv_tegra_releasez/sys/class/tegra-firmware)pathisfileexistsrosrrcuda_is_jetson`sz,cuda_platform_plugin..cuda_is_jetsonz/Confirmed CUDA platform is available on Jetson.Tz*CUDA platform is not available because: %sz vllm.platforms.cuda.CudaPlatform)rrvllm.utils.import_utilsr nvmlInitnvmlDeviceGetCountr nvmlShutdownrr __class____name__lowerr'bool)is_cudar pynvmlrr(rr&rcuda_platform_plugin;sB        r3c Csd}tdz+ddl}|zt|dkr!d}tdntdW|n|wWntyL}ztdt|WYd}~nd}~ww|rQdSdS) NFz'Checking if ROCm platform is available.rTz%Confirmed ROCm platform is available.z7ROCm platform is not available because no GPU is found.z*ROCm platform is not available because: %sz vllm.platforms.rocm.RocmPlatform) rramdsmi amdsmi_initlenamdsmi_get_processor_handlesamdsmi_shut_downrr)is_rocmr4rrrrrocm_platform_pluginns"    r:c Csd}tdz,ddl}tr!d}ddlm}||_td|jt|dr2|j r2d}td Wnt yM}ztd t |WYd}~nd}~ww|rRd SdS) NFz&Checking if XPU platform is available.rxccl) XPUPlatformz"Confirmed %s backend is available.xpuTz$Confirmed XPU platform is available.z)XPU platform is not available because: %szvllm.platforms.xpu.XPUPlatform) rrtorchrvllm.platforms.xpur< dist_backendhasattrr= is_availablerr)is_xpur>r@r<rrrrxpu_platform_plugins$    rDc Csd}tdz td}|rtd|s&ddl}|jd}|r&tdWntyA}ztdt|WYd}~nd}~ww|rFd SdS) NFz&Checking if CPU platform is available.r!zCConfirmed CPU platform is available because vLLM is built with CPU.rdarwinzAConfirmed CPU platform is available because the machine is MacOS.z)CPU platform is not available because: %szvllm.platforms.cpu.CpuPlatform)rrrsysplatform startswithrr)is_cpurFrrrrcpu_platform_plugins*   rJ)tpucudarocmr=r!c Cs4tt}g}tt|D]!\}}zt|sJ|}|dur&||Wqty0Yqwtt |t t @}tt |t | @}t |dkrVt d|t |dkrm||d}t d|d|St |dkrzt d|t |dkrt|d}t d|d|Sd}t d|S) Nz4Only one platform plugin can be activated, but got: r rzPlatform plugin %s is activatedz#Automatically detected platform %s.z,vllm.platforms.interface.UnspecifiedPlatformz|dkr|adS|tvr|t|<dStd|dtd)Nrbrcrdre)rfrjrkr.)rZvaluerrr __setattr__s  rn)r r rbr ra)&loggingrh itertoolsrtypingrrr vllm.pluginsrrr)rvllm.utils.torch_utilsr interfacer r r getLoggerr.rrr0rrr3r:rDrJrOr_rfra__annotations__rlrn__all__rrrrs<       3 +