o piZ @s.dZddlmZmZmZmZddlmZddlm Z m Z e r/ddl Z ddl m Z mZmZmZeeZzddlmZWneefyKd d ZYnw    d%d eeefd eeed d fdeddeddedf ddZdefddZdddededdfddZded dd!dded"fd#d$ZdS)&z1 PyTorch utilities: Utilities related to PyTorch )ListOptionalTupleUnion)logging)is_torch_availableis_torch_versionN)fftnfftshiftifftn ifftshift)allow_in_graphcCs|S)N)clsrrY/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/utils/torch_utils.pymaybe_allow_in_graph"srshape generatorztorch.Generatordevicez torch.devicedtypez torch.dtypelayoutz torch.layoutc s<|d}p tj|ptd}durXttsjjndjj}||jkrD|dkrDd|dkrCtd|d|d|dn||jkrX|d krXtd |d |d ttrgt d krgdttrdd dfddt |D}tj |dd |}|Stj d |}|S)zA helper function to create random tensors on the desired `device` with the desired `dtype`. When passing a list of generators, you can seed each batch size individually. If CPU generators are passed, the tensor is always created on the CPU. rcpuNmpszBThe passed generator was created on 'cpu' even though a tensor on zB was expected. Tensors will be created on 'cpu' and then moved to zk. Note that one can probably slighly speed up this function by passing a generator that was created on the z device.cudazCannot generate a z! tensor from a generator of type .r)rc s$g|]}tj|dqS)rrrr)torchrandn).0irrr rand_devicerrr Ksz randn_tensor..dimr)rstridedr isinstancelisttypeloggerinfo ValueErrorlenrangecattor)rrrrr batch_sizegen_device_typelatentsrr!r randn_tensor&s<   r4returncCs(tdds ttds dSt|tjjjS)z:Check whether the module was compiled with torch.compile()r>cKs|dkr*|jdd}|ddd|f|d|ddd|f<t|d|dd}|dkrT|jdd}|ddd|f|d|ddd|f<t|d|d d}||fS) a]Applies the FreeU mechanism as introduced in https: //arxiv.org/abs/2309.11497. Adapted from the official code repository: https://github.com/ChenyangSi/FreeU. Args: resolution_idx (`int`): Integer denoting the UNet block where FreeU is being applied. hidden_states (`torch.Tensor`): Inputs to the underlying block. res_hidden_states (`torch.Tensor`): Features from the skip block corresponding to the underlying block. s1 (`float`): Scaling factor for stage 1 to attenuate the contributions of the skip features. s2 (`float`): Scaling factor for stage 2 to attenuate the contributions of the skip features. b1 (`float`): Scaling factor for stage 1 to amplify the contributions of backbone features. b2 (`float`): Scaling factor for stage 2 to amplify the contributions of backbone features. rrrCNb1s1)r?r@b2s2)rrQ)rRrSrT freeu_kwargsnum_half_channelsrrr apply_freeu|s,,r[)NNNN)__doc__typingrrrrr import_utilsrr r torch.fftr r r r get_logger__name__r* torch._dynamorr ImportErrorModuleNotFoundErrorr4boolr<intrQr[rrrrsN      0