o zi @sddlmZddlmZmZmZddlZddlmZddl m Z ddl m Z ddl mZGdd d e Zdd ed eefd dZd eefddZd efddZd efddZddeejd efddZeddejd dfddZdddZdS) ) lru_cache)ListOptionalUnionN)override) Accelerator)_AcceleratorRegistry)rank_zero_infoc@seZdZdZedejddfddZedddZe ed e e e e e fdee e fd d Ze ed e e de ejfd d Ze ede fddZe edefddZeededdfddZdS)CUDAAcceleratorz$Accelerator for NVIDIA CUDA devices.devicereturnNcCs2|jdkr td|dt|tj|dS)zm Raises: ValueError: If the selected device is not of type CUDA. cudazDevice should be CUDA, got z instead.N)type ValueError_check_cuda_matmul_precisiontorchr set_device)selfr rV/home/ubuntu/.local/lib/python3.10/site-packages/lightning_fabric/accelerators/cuda.py setup_devices zCUDAAccelerator.setup_devicecCs tdSN)_clear_cuda_memory)rrrrteardown( zCUDAAccelerator.teardowndevicescCsddlm}||ddS)z!Accelerator device parsing logic.r)_parse_gpu_idsT) include_cuda)(lightning_fabric.utilities.device_parserr)rrrrr parse_devices,s  zCUDAAccelerator.parse_devicescCsdd|DS)z*Gets parallel devices for the Accelerator.cSsg|]}td|qS)r )rr ).0irrr 8sz8CUDAAccelerator.get_parallel_devices..r)rrrrget_parallel_devices4sz$CUDAAccelerator.get_parallel_devicescCstS)z!Get the devices when set to auto.num_cuda_devicesrrrrauto_device_count:sz!CUDAAccelerator.auto_device_countcCs tdkS)Nrr$rrrr is_available@ zCUDAAccelerator.is_availableaccelerator_registrycCs|jd||jddS)Nr ) description)register__name__)clsr)rrrregister_acceleratorsEs  z%CUDAAccelerator.register_acceleratorsr N)r, __module__ __qualname____doc__rrr rr staticmethodrintstrrrrr#r&boolr' classmethodrr.rrrrr s* , r num_devicesr c Cs|dkrgSt}|std|d|t|kr&td|dt|dg}g}|D]+}z tjdtd|dWntyI||Yq,w||t||krWnq,|dkrrt||krrtd|d t|d |d |S) aReturns a list of all available and usable CUDA GPU devices. A GPU is considered usable if we can successfully move a tensor to the device, and this is what this function tests for each GPU on the system until the target number of usable devices is found. A subset of GPUs on the system might be used by other processes, and if the GPU is configured to operate in 'exclusive' mode (configurable by the admin), then only one process is allowed to occupy it. Args: num_devices: The number of devices you want to request. By default, this function will return as many as there are usable CUDA GPU devices available. Warning: If multiple processes call this function at the same time, there can be race conditions in the case where both processes determine that the device is unoccupied, leading into one of them crashing later on. rzYou requested to find z? devices but there are no visible CUDA devices on this machine.z# devices but this machine only has z GPUs.r r r8z devices but only z& are currently available. The devices zA are occupied by other processes and can't be used at the moment.)_get_all_visible_cuda_devicesrlenrtensorr RuntimeErrorappend)r9visible_devicesavailable_devicesunavailable_devicesgpu_idxrrrfind_usable_cuda_devicesOs>      rDcCstttS)aReturns a list of all visible CUDA GPU devices. Devices masked by the environment variabale ``CUDA_VISIBLE_DEVICES`` won't be returned here. For example, assume you have 8 physical GPUs. If ``CUDA_VISIBLE_DEVICES="1,3,6"``, then this function will return the list ``[0, 1, 2]`` because these are the three visible GPUs after applying the mask ``CUDA_VISIBLE_DEVICES``. )listranger%rrrrr;sr;cC tjS)z-Returns the number of available CUDA devices.)rr device_countrrrrr%rr%cCrG)z9Returns a bool indicating if CUDA is currently available.)rr r'rrrris_cuda_availabler(rIr cCstj|\}}|dkS)N)rr get_device_capability)r major_rrr_is_ampere_or_latersrNcCsBtjr t|s dStdkrtdtj|ddSdS)NhighestzYou are using a CUDA device (a5) that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision)rr r'rNget_float32_matmul_precisionr get_device_namer:rrrrs rcCs$ttjdr tjtjdS)N_cuda_clearCublasWorkspaces)hasattrr_CrSr empty_cacherrrrrs  r)r8rr/) functoolsrtypingrrrrtyping_extensionsr)lightning_fabric.accelerators.acceleratorr&lightning_fabric.accelerators.registryr$lightning_fabric.utilities.rank_zeror r r4rDr;r%r6rIr rNrrrrrrs     65