o oi@sddlZddlZddlmZddlmZmZmZddlZddl m Z ddl m Z ddl mZddlmZdZhd Zd ZeeZd ed eefd dZdeded efddZd eeeffddZd eefddZded dfddZdddZdS)N)Path)DictIterableIterator)get_cuda_bnb_library_path)NONPYTORCH_DOC_URL) CUDASpecs)print_dedented) CONDA_PREFIXLD_LIBRARY_PATH>_PWDHOMEMAILPATHTMUXSHELLOLDPWDSSH_TTYLESSOPEN LESSCLOSE SSH_AUTH_SOCK XDG_DATA_DIRSXDG_RUNTIME_DIRXDG_GREETER_DATA_DIRDBUS_SESSION_BUS_ADDRESSGOOGLE_VM_CONFIG_LOCK_FILE)z cudart64*.dllzlibcudart*.so*z nvcuda*.dllpaths_list_candidatereturnc cs|tjD]K}|s qtj|vrqz5t|}z|s'td|WWqWn ty1Ynwt D]}| |D] }| rD|Vq;q4Wqtt fyRYqwdS)Nz?The directory listed in your path is found to be non-existent: ) splitospathsepseprexistsloggerwarningOSErrorCUDA_RUNTIME_LIB_PATTERNSglobis_filePermissionError)r dir_stringdir lib_patternpthr/Q/home/ubuntu/.local/lib/python3.10/site-packages/bitsandbytes/diagnostics/cuda.py find_cuda_libraries_in_path_list,s6  r1env_varvaluecCs2|tvptj|vo|tvod|vod|vod|vS)NCONDA BASH_FUNC )CUDART_PATH_PREFERRED_ENVVARSr r"CUDART_PATH_IGNORED_ENVVARS)r2r3r/r/r0is_relevant_candidate_env_varBs r9cCsddtjDS)NcSs i|] \}}t||r||qSr/)r9).0r2r3r/r/r0 Ps z@get_potentially_lib_path_containing_env_vars..)r environitemsr/r/r/r0,get_potentially_lib_path_containing_env_varsOsr>ccs^t}tD]}||vr||}t|EdH||q|D] \}}t|EdHq!dS)a Searches for a cuda installations, in the following order of priority: 1. active conda env 2. LD_LIBRARY_PATH 3. any other env vars, while ignoring those that - are known to be unrelated - don't contain the path separator `/` If multiple libraries are found in part 3, we optimistically try one, while giving a warning message. N)r>r7r1popr=)candidate_env_varsenvvar directoryr2r3r/r/r0find_cudart_librariesSs  rC cuda_specscCsxtd|jd|jdt|}|std|d|j\}}|dkr*tdtdt|js:td dSdS) Nz%PyTorch settings found: CUDA_VERSION=z, Highest Compute Capability: .z Library not found: aZ. Maybe you need to compile it from source? If you compiled from source, try again with `make CUDA_VERSION=DETECTED_CUDA_VERSION`, for example, `make CUDA_VERSION=113`. The CUDA version for the compile might depend on your conda install, if using conda. Inspect CUDA version via `conda list | grep cuda`. z WARNING: CUDA versions lower than 11 are currently not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines! z:To manually override the PyTorch CUDA version please see: a WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! If you run into issues with 8-bit matmul, you can try 4-bit quantization: https://huggingface.co/blog/4bit-transformers-bitsandbytes ) printcuda_version_stringhighest_compute_capabilityrr#r cuda_version_tuplerhas_imma)rD binary_path cuda_major cuda_minorr/r/r0print_cuda_diagnosticsks.  rOcCsZtt}|s tddSt|dkr)tdtjjd|D] }td|qdSdS)NzLCUDA SETUP: WARNING! CUDA runtime files not found in any environmental path.z Found duplicate CUDA runtime files (see below). We select the PyTorch default CUDA runtime, which is a], but this might mismatch with the CUDA version that is needed for bitsandbytes. To override this behavior set the `BNB_CUDA_VERSION=` environmental variable. For example, if you want to use the CUDA version 122, BNB_CUDA_VERSION=122 python ... OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122 In the case of a manual override, make sure you set LD_LIBRARY_PATH, e.g. export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2, z* Found CUDA runtime at: )listrCrGlenr torchversioncuda) cudart_pathsr.r/r/r0print_cuda_runtime_diagnosticss   rW)rN)loggingr pathlibrtypingrrrrSbitsandbytes.cextensionrbitsandbytes.constsrbitsandbytes.cuda_specsrbitsandbytes.diagnostics.utilsr r7r8r' getLogger__name__r$strr1boolr9r>rCrOrWr/r/r/r0s&       -