o c۷il!@sdZddlZddlZddlZddlZddlZddlmZddlZej ddZ e r.dej d<ej dddkZ da dad ad efd d Zd ededBfddZd ededBfddZdeeefdBfddZd ededefddZddZddZddZdS)a System ptxas replacement for CUTLASS DSL. Usage:: CUTE_DSL_KEEP_PTX=1 CUTE_DSL_PTXAS_PATH=/usr/local/cuda/bin/ptxas pytest tests/ Environment variables: CUTE_DSL_PTXAS_PATH - Path to ptxas (e.g., /usr/local/cuda/bin/ptxas) CUTE_DSL_KEEP_PTX - Must be set to 1 before cutlass is imported CUTE_DSL_PTXAS_VERBOSE - Set to 1 for verbose output CUTE_DSL_DUMP_DIR - Directory for dumped PTX files (default: cwd) CUTE_DSL_KEEP_CUBIN - Set to 1 to save compiled cubin files N)PathCUTE_DSL_PTXAS_PATH1CUTE_DSL_KEEP_PTXCUTE_DSL_PTXAS_VERBOSE0FmsgcCs trtd|tjddSdS)Nz[ptxas] )file)VERBOSEprintsysstderr)rrJ/home/ubuntu/vllm_env/lib/python3.10/site-packages/quack/cute_dsl_ptxas.py_log%srptx_pathreturnc CsVz |jddddWSty*}ztd|d|WYd}~dSd}~ww)Nzutf-8ignore)errorszFailed to read : ) read_bytesdecoderstripOSErrorr)rexcrrr _read_ptx*srcCs&t|}|dus|dsdS|S)N})rrendswith)rcontentrrr_read_complete_ptx2sr cst|ddstddSttjdt}|jdddt| ddd dd }td d |td t |d|fdd|D}|rvtd t |d|D]}t |}|durfq[tdd|||fSt dt dt j}|D] }t |}|durq||rtdd|||fSqt |dkrt |d}|durtdd|d||dfStdd |dS)z*Find dumped PTX for the compiled function. function_nameNz*Compiled function is missing function_nameCUTE_DSL_DUMP_DIRT)parentsexist_okz*.ptxcSs |jS)N)stat st_mtime_ns)pathrrrDs z_get_ptx..)keyreversezSearching dumped PTX for z in zFound z PTX candidate files in csg|] }|jvr|qSr)name).0r func_namerr Jsz_get_ptx..z filename matches for zUsing PTX filename match for rz \.entry\s+z (?:\s|\()zFound PTX for rzUsing sole PTX candidate for zNo PTX found for function )getattrrrosenvirongetcwdmkdirsortedrgloblenr recompileescape MULTILINEsearch) compiled_funcdump_dir ptx_pathsfilename_matchesrr entry_patternrr-r_get_ptx9sH       rD ptx_contentc Cstd|}|r |dnd}||kr|||d}z`tdus&Jtjtd|ddt |t |gd d d }|j d krHt d |j | }td |jdt|d|dtjdddkrw|d}||td||W|jd dS|jd dw)z(Compile PTX to cubin using system ptxas.z\.target\s+(sm_\d+[a-z]?)r0sm_90az .cubin.tmpNz-arch=z-O3z-oT)capture_outputtextrzptxas failed: z Compiled z -> z bytes ()CUTE_DSL_KEEP_CUBINrrz.cubinzSaved:  missing_ok)r:r>group read_text write_text with_suffixr subprocessrunstr returncode RuntimeErrorr rrr+r9r2r3r4 write_bytesunlink)rrEmatcharch cubin_tmpresult cubin_data cubin_outrrr _compile_ptxis,      "  r^c Cst|}|stdt|S|\}}zt||}Wnty7}ztd|dt|WYd}~Sd}~wwddlmm}||dddddd\}}||j j kr`td|dt|S| \} } t t|} t | } t | } t d}t d}t j dt | t j t t |t j t t |t j }t|jD]}||_| ||jdkrtdt|Sqtd |jts|jd d || jgS) z:Replacement for _load_cuda_library that uses system ptxas.z-PTX not found, falling back to embedded ptxaszCompilation failed (z!), falling back to embedded ptxasNrzcudaLibraryLoadData failed (z:cuda_load_to_device failed, falling back to embedded ptxaszLoaded kernel from TrK)rDr_original_load_cuda_libraryr^ Exceptioncuda.bindings.runtimebindingsruntimecudaLibraryLoadData cudaError_t cudaSuccess_get_cuda_init_and_loadctypesc_void_pintpointerc_int32castrange num_devicesvaluer+_user_wanted_ptxrW cudaLibrary_t)selfr[rErcubine cuda_runtimeerrlibrary_cuda_load_to_device lib_handle ptr_to_libptr_to_ptr_to_libdev_iderr_valargsdevrrr_patched_load_cuda_librarysN          rcCs8t|dddur||_tdt|jdt|S)N_ptxas_cuda_libraryzLoaded z0 CUDA libraries before creating TVM FFI function)r1_load_cuda_libraryrrr9!_original_create_tvm_ffi_function)rtrrr _patched_create_tvm_ffi_functions  rcCstdusJtjtrtttjstdttjdddka tjdddks0Jdd}t j j j }|jturD|jat|_d}d d lm}|jturW|jat|_d}|rbtd tdStd dS) z9Install system ptxas hook. Call before importing cutlass.Nzptxas not found: rrrz1Require CUTE_DSL_KEEP_PTX=1 to use system's ptxasFTr)TVMFFIJitCompiledFunctionBasez"Installed system ptxas patch with z$System ptxas patch already installed)rr2r'isfileaccessX_OKrUr3r4rrcutlass cutlass_dslcuda_jit_executorCudaDialectJitCompiledFunctionrrr`$cutlass.cutlass_dsl.tvm_ffi_providerr_create_tvm_ffi_functionrrr)patchedcuda_jit_function_clsrrrrpatchs.     r)__doc__r2r r:rirQpathlibrrr3r4rr r`rrrrSrrr tuplerDbytesr^rrrrrrrs.  0$5