o 3wi-@sdZddlZddlZddlZddlZddlZddlZddlmZddl m Z m Z m Z m Z mZmZmZmZmZddlmZddd Zd d Zd ed efddZ ddededefddZdefddZdS)z A collection of utilities for ensuring that training can always occur. Heavily influenced by the [toma](https://github.com/BlackHC/toma) library. N)version) is_cuda_availableis_hpu_availableis_ipex_availableis_mlu_availableis_mps_availableis_musa_availableis_npu_availableis_sdaa_availableis_xpu_available)compare_versionsFcCs|rttrtjdStrtjdStr$tj dSt r.tj dSt r8tj dStddrDtjdStrNtjdStrT dSdS)z Clears the device cache by calling `torch.{backend}.empty_cache`. Can also run `gc.collect()`, but do note that this is a *considerable* slowdown and should be used sparingly. z2.0) min_versionN)gccollectr torchxpu empty_cachermlur sdaar musar npurmpsrcudargarbage_collectionrT/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/accelerate/utils/memory.pyclear_device_cache+s& rcGs:t|ts t|}tt|D]}d||<qtdd|S)aN Releases memory from `objects` by setting them to `None` and calls `gc.collect()` and `torch.cuda.empty_cache()`. Returned objects should be reassigned to the same variables. Args: objects (`Iterable`): An iterable of objects Returns: A list of `None` objects to replace `objects` Example: ```python >>> import torch >>> from accelerate.utils import release_memory >>> a = torch.ones(1000, 1000).cuda() >>> b = torch.ones(1000, 1000).cuda() >>> a, b = release_memory(a, b) ``` NTr) isinstancelistrangelenr)objectsirrrrelease_memoryFs   r% exceptionreturncs:gd}ttrtjdkrtfdd|DSdS)z Checks if `exception` relates to CUDA out-of-memory, XPU out-of-memory, CUDNN not supported, or CPU out-of-memory Args: exception (`Exception`): An exception )z out of memory.z(cuDNN error: CUDNN_STATUS_NOT_SUPPORTED.z*DefaultCPUAllocator: can't allocate memoryz1FATAL ERROR :: MODULE:PT_DEVMEM Allocation failedrc3s|] }|jdvVqdS)rN)args).0errr&rr ssz+should_reduce_batch_size..F)r RuntimeErrorr"r(any)r& _statementsrr+rshould_reduce_batch_sizedsr0functionstarting_batch_sizereduce_batch_size_fncsBdur tjt|dS|durfddfdd}|S)a A basic decorator that will try to execute `function`. If it fails from exceptions related to out-of-memory or CUDNN, the batch size is multiplied by 0.9 and passed to `function` `function` must take in a `batch_size` parameter as its first argument. Args: function (`callable`, *optional*): A function to wrap starting_batch_size (`int`, *optional*): The batch size to try and fit into memory Example: ```python >>> from accelerate.utils import find_executable_batch_size >>> @find_executable_batch_size(starting_batch_size=128) ... def train(batch_size, model, optimizer): ... ... >>> train(model, optimizer) ``` N)r3cstdS)Ng?)intr) batch_sizerrr4s z8find_executable_batch_size..reduce_batch_size_fnc stddttj}t|t|dkr>dddt|dd|ddD}t dj dj d |d  d krGt d z g|Ri|WSt ys}zt |rhtddnWYd}~nd}~wwq?) NTrrz, cSsg|] \}}|d|qS)=r)r)argvaluerrr szAfind_executable_batch_size..decorator..zBatch size was passed into `zS` as the first argument when called.Remove this as the decorator already does so: `(z)`rz-No executable batch size found, reached zero.)rr inspect signature parameterskeysr"joinzip TypeError__name__r- Exceptionr0)r(kwargsparamsarg_strer6r2r4rr decorators2 *   z-find_executable_batch_size..decorator) functoolspartialfind_executable_batch_size)r2r3r4rJrrIrrMws rM device_indexcCsttjjtdjkr z tj|dWStyYnwtr>tt j d}t |ddr>ddl m}||dSt dtj|S)Nz2.6rintel_extension_for_pytorchz>=z2.5) mem_get_infozThe XPU `mem_get_info` API is available in IPEX version >=2.5 or PyTorch >=2.6. The current returned available memory is incorrect. Please consider upgrading your IPEX or PyTorch version.)rparser __version__releaserrPrDr importlibmetadatar intel_extension_for_pytorch.xpuwarningswarnmax_memory_allocated)rN ipex_versionrPrrrget_xpu_available_memorys     r[)F)Nr1N)__doc__rKrrTr<rWr packagingrimportsrrrrrr r r r versionsr rr%rDboolr0callabler5rMr[rrrrs, ,  B