o ۷i/ @spdZddlmZddlZddlZddlmZddlmZm Z m Z m Z erddl Z ddl mZmZmZmZdddd dd Ze jje jjde jjdd Ze jje jjd d d d dd Ze jje jje je jje jd Ze jjee jdddddd Ze jjee jdddddd Z e jj!ee jdddddd Z"e jj#ee jdddddd Z$e%e&Z'zddl(m)Z*Wne+e,fyddZ*YnwdWddZ-dXddZ.dYddZ/dYd d!Z0dYd"d#Z1dYd$d%Z2dYd&d'Z3dYd(d)Z4dYd*d+Z5    dZd[d5d6Z6d\d9d:Z7d;d<Z8d]dAdBZ9d^dGdHZ:dIdJZ;ejd_d`dQdRZ?dSdTZ@dUdVZAer6e=ZBdSdS)az1 PyTorch utilities: Utilities related to PyTorch ) annotationsN)logging)is_torch_availableis_torch_mlu_availableis_torch_npu_availableis_torch_version)fftnfftshiftifftn ifftshiftTF)cudaxpucpumpsdefaultcCdSNrrrrQ/home/ubuntu/vllm_env/lib/python3.10/site-packages/diffusers/utils/torch_utils.py*rcCrrrrrrrr+rreset_peak_memory_statsmax_memory_allocated synchronize)allow_in_graphcCs|SNr)clsrrrmaybe_allow_in_graphWsrdevicestrdispatch_tabledict[str, callable]cOs<||vr |d|i|S||}t|s|S||i|S)Nr)callable)rr!argskwargsfnrrr_device_agnostic_dispatch\s r'seedintcCs t|t|Sr)r'BACKEND_MANUAL_SEED)rr(rrrbackend_manual_seedks r+cC t|tSr)r'BACKEND_SYNCHRONIZErrrrbackend_synchronizeo r/cCr,r)r'BACKEND_EMPTY_CACHEr.rrrbackend_empty_cachesr0r2cCr,r)r'BACKEND_DEVICE_COUNTr.rrrbackend_device_countwr0r4cCr,r)r'BACKEND_RESET_PEAK_MEMORY_STATSr.rrrbackend_reset_peak_memory_stats{r0r6cCr,r)r'"BACKEND_RESET_MAX_MEMORY_ALLOCATEDr.rrr"backend_reset_max_memory_allocatedr0r8cCr,r)r'BACKEND_MAX_MEMORY_ALLOCATEDr.rrrbackend_max_memory_allocatedr0r:cCstsdS|tvr d}t|S)NFr)rBACKEND_SUPPORTS_TRAININGr.rrrbackend_supports_trainings r<shape tuple | list generator2list['torch.Generator'] | 'torch.Generator' | Nonestr | 'torch.device' | Nonedtype'torch.dtype' | Nonelayout'torch.layout' | Nonec sPt|tr t|}|d}ptj|ptd}durbtts)jjndjj}||jkrN|dkrNd|dkrMtd|d|d|dn||jkrb|d krbt d |d |d ttrqt d krqdttrdd dfddt |D}tj |dd |}|Stjd |}|S)zA helper function to create random tensors on the desired `device` with the desired `dtype`. When passing a list of generators, you can seed each batch size individually. If CPU generators are passed, the tensor is always created on the CPU. rrNrzBThe passed generator was created on 'cpu' even though a tensor on zB was expected. Tensors will be created on 'cpu' and then moved to zl. Note that one can probably slightly speed up this function by passing a generator that was created on the z device.r zCannot generate a z! tensor from a generator of type .r)rc s$g|]}tj|dqS)r?rrBrD)torchrandn).0irBr?rD rand_devicer=rr sz randn_tensor..dimrG) isinstancer rHrstridedlisttypeloggerinfo ValueErrorlenrangecattorI)r=r?rrBrD batch_sizegen_device_typelatentsrrLr randn_tensors@   r_returnboolcCs(tdds ttds dSt|tjjjS)z:Check whether the module was compiled with torch.compile()r?r@rrArBrCrDrE)r`ra)rlrmrnr)ror)r`rm)rr)rrmrrmr`rr)rr)C__doc__ __future__r functoolsrrr import_utilsrrrrrH torch.fftr r r r r;r rrrr1 device_countr3 manual_seedr*rrr5reset_max_memory_allocatedr7rr9rr- get_logger__name__rU torch._dynamorr ImportErrorModuleNotFoundErrorr'r+r/r2r4r6r8r:r<r_rirkrrr lru_cacherrrrr torch_devicerrrrs                   2  "