o 3wi6@sddlmZmZddlZddlZddlmZddlZddlm Z ddl m Z ddl m Z dZhdZd ZeeZd ed eefd d Zdeded efddZd eeeffddZd eefddZde d dfddZdddZdS))IterableIteratorN)Path)get_cuda_bnb_library_path) CUDASpecs)print_dedented) CONDA_PREFIXLD_LIBRARY_PATH>_PWDHOMEMAILPATHTMUXSHELLOLDPWDSSH_TTYLESSOPEN LESSCLOSE SSH_AUTH_SOCK XDG_DATA_DIRSXDG_RUNTIME_DIRXDG_GREETER_DATA_DIRDBUS_SESSION_BUS_ADDRESSGOOGLE_VM_CONFIG_LOCK_FILE)z cudart64*.dllzlibcudart*.so*z nvcuda*.dllpaths_list_candidatereturnc cs|tjD]K}|s qtj|vrqz5t|}z|s'td|WWqWn ty1Ynwt D]}| |D] }| rD|Vq;q4Wqtt fyRYqwdS)Nz?The directory listed in your path is found to be non-existent: ) splitospathsepseprexistsloggerwarningOSErrorCUDA_RUNTIME_LIB_PATTERNSglobis_filePermissionError)r dir_stringdir lib_patternpthr-Z/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/bitsandbytes/diagnostics/cuda.py find_cuda_libraries_in_path_list+s6  r/env_varvaluecCs2|tvptj|vo|tvod|vod|vod|vS)NCONDA BASH_FUNC )CUDART_PATH_PREFERRED_ENVVARSrr CUDART_PATH_IGNORED_ENVVARS)r0r1r-r-r.is_relevant_candidate_env_varAs r7cCsddtjDS)NcSs i|] \}}t||r||qSr-)r7).0r0r1r-r-r. Os z@get_potentially_lib_path_containing_env_vars..)renvironitemsr-r-r-r.,get_potentially_lib_path_containing_env_varsNsr<ccs^t}tD]}||vr||}t|EdH||q|D] \}}t|EdHq!dS)a Searches for a cuda installations, in the following order of priority: 1. active conda env 2. LD_LIBRARY_PATH 3. any other env vars, while ignoring those that - are known to be unrelated - don't contain the path separator `/` If multiple libraries are found in part 3, we optimistically try one, while giving a warning message. N)r<r5r/popr;)candidate_env_varsenvvar directoryr0r1r-r-r.find_cudart_librariesRs  rA cuda_specscCsPtd|jd|jdt|}|std|d|js&tddSdS)Nz%PyTorch settings found: CUDA_VERSION=z, Highest Compute Capability: .z Library not found: z8. Maybe you need to compile it from source? a WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! If you run into issues with 8-bit matmul, you can try 4-bit quantization: https://huggingface.co/blog/4bit-transformers-bitsandbytes )printcuda_version_stringhighest_compute_capabilityrr!rhas_imma)rB binary_pathr-r-r.print_cuda_diagnosticsjs" rIcCsZtt}|s tddSt|dkr)tdtjjd|D] }td|qdSdS)NzLCUDA SETUP: WARNING! CUDA runtime files not found in any environmental path.z Found duplicate CUDA runtime files (see below). We select the PyTorch default CUDA runtime, which is a], but this might mismatch with the CUDA version that is needed for bitsandbytes. To override this behavior set the `BNB_CUDA_VERSION=` environmental variable. For example, if you want to use the CUDA version 122, BNB_CUDA_VERSION=122 python ... OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122 In the case of a manual override, make sure you set LD_LIBRARY_PATH, e.g. export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2, z* Found CUDA runtime at: )listrArDlenrtorchversioncuda) cudart_pathsr,r-r-r.print_cuda_runtime_diagnosticss   rQ)rN)collections.abcrrloggingrpathlibrrMbitsandbytes.cextensionrbitsandbytes.cuda_specsrbitsandbytes.diagnostics.utilsrr5r6r% getLogger__name__r"strr/boolr7dictr<rArIrQr-r-r-r.s$