o پiG @sUddlmZddlZddlmZddlmZddlmZer%ddl m Z e e Z GdddeZdad ed <dddZdddZdS)) annotationsN)Enum) TYPE_CHECKING)envs) ServerArgsc@sReZdZdZdZdZdZdZddd Zdd d Z dd d Z dddZ dddZ dS)Fp4GemmRunnerBackendz+Enum for FP4 GEMM runner backend selection.autoflashinfer_cudnnflashinfer_cutlassflashinfer_trtllmreturnboolcC |tjkSN)rAUTOselfr\/home/ubuntu/.local/lib/python3.10/site-packages/sglang/srt/layers/quantization/fp4_utils.pyis_auto zFp4GemmRunnerBackend.is_autocCrr)rFLASHINFER_CUDNNrrrris_flashinfer_cudnnrz(Fp4GemmRunnerBackend.is_flashinfer_cudnncCrr)rFLASHINFER_CUTLASSrrrris_flashinfer_cutlassrz*Fp4GemmRunnerBackend.is_flashinfer_cutlasscCrr)rFLASHINFER_TRTLLMrrrris_flashinfer_trtllm rz)Fp4GemmRunnerBackend.is_flashinfer_trtllmstrcCs|jdr |jdS|jS)a,Get the backend string to pass to FlashInfer's mm_fp4 API. This remaps SGLang's user-facing backend names to FlashInfer's API names. Examples: 'flashinfer_trtllm' -> 'trtllm' 'flashinfer_cutlass' -> 'cutlass' 'flashinfer_cudnn' -> 'cudnn' flashinfer_)value startswith removeprefixrrrrget_flashinfer_backend#s z+Fp4GemmRunnerBackend.get_flashinfer_backendN)r r )r r) __name__ __module__ __qualname____doc__rrrrrrrrr"rrrrrs    rzFp4GemmRunnerBackend | NoneFP4_GEMM_RUNNER_BACKEND server_argsrr NonecCsj|j}tjr/tj}|dkr&td|d|ds#d|}|}n td|dt|a dS)z3Initialize FP4 GEMM configuration from server args.rzQSGLANG_FLASHINFER_FP4_GEMM_BACKEND is deprecated. Please use '--fp4-gemm-backend=z ' instead.rzFP4 GEMM backend set to 'zx' via --fp4-gemm-backend overrides environment variable SGLANG_FLASHINFER_FP4_GEMM_BACKEND. Using server argument value.N) fp4_gemm_runner_backendr"SGLANG_FLASHINFER_FP4_GEMM_BACKENDis_setgetloggerwarningr rr')r(backend env_backendrrrinitialize_fp4_gemm_config5s      r2cCstdurtjatS)z(Get the current FP4 GEMM runner backend.N)r'rrrrrrget_fp4_gemm_runner_backendQsr3)r(rr r))r r) __future__rloggingenumrtypingrsglang.srt.environrsglang.srt.server_argsr getLoggerr#r.rr'__annotations__r2r3rrrrs      #