o پi@sdZddlZddlmZmZmZddlZddlmZe e Z gdZ de deeeeee ffdd Zdeeee ffd d ZdS) z>Mooncake-specific utilities for custom memory pool management.N)AnyOptionalTuple)envs)NVLINKBAREX INTRA_NVLINKdevicereturnc Cs6t\}}d}|rzC|dkrddlm}||}n|dkr*ddlm}||}n|dkr1WdStd |tj| }t d |d |WnIt yp}zt d |d |dd}d}d}WYd}~n*d}~wty}zt d|d |dd}d}d}WYd}~nd}~wwdS|||fS)z Initialize custom memory pool based on environment variable. Args: device: The device to allocate memory on Returns: Tuple of (enable_custom_mem_pool, custom_mem_pool, custom_mem_pool_type) Nrr)NVLinkAllocatorr)BarexAllocatorINTRA_NODE_NVLINK)FNNz"Unsupported custom mem pool type: z Initialized custom memory pool: z on device z(Failed to import mooncake allocator for z: z&. Falling back to default memory pool.Fz(Failed to initialize custom memory pool )&check_mooncake_custom_mem_pool_enabledmooncake.allocatorr get_allocatorr ValueErrortorchcudaMemPool allocatorloggerdebug ImportErrorwarning Exceptionerror)r enable_custom_mem_poolcustom_mem_pool_typecustom_mem_poolr rr er \/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/disaggregation/mooncake/utils.pyinit_mooncake_custom_mem_poolsN      r"cCsBtj}|dur|dkrd}|tv}||fSd}d}||fS)z Check if custom memory pool is enabled without importing allocators. Returns: Tuple of (enable_custom_mem_pool, custom_mem_pool_type) NtruerF)rSGLANG_MOONCAKE_CUSTOM_MEM_POOLgetlower(SUPPORTED_MOONCAKE_CUSTOM_MEM_POOL_TYPES)rrr r r!r\s  r)__doc__loggingtypingrrrrsglang.srt.environr getLogger__name__rr'strboolr"rr r r r!s    ?