o Ii @sddlZddlmZmZddlZddlZddlmZddl m Z ddl m Z m Z ddlmZmZmZmZmZmZmZmZerHddlmmZdd ed ed efd dZddee deejfddZdde ee!e fdeejfddZ"dS)N)OptionalUnion)AcceleratorState)CUDA_DISTRIBUTED_TYPES)DistributedTypeRNGType)is_hpu_availableis_mlu_availableis_musa_availableis_neuron_availableis_npu_availableis_sdaa_availableis_torch_xla_availableis_xpu_availableFseeddevice_specific deterministiccCs|r|tj7}t|tj|t|tr"tj |nBt r,tj |n8t r6tj |n.tr@tj |n$trJtj |ntrTtj |ntr^tj |ntj |trlt||rutddSdS)a Helper function for reproducible behavior to set the seed in `random`, `numpy`, `torch`. Args: seed (`int`): The seed to set. device_specific (`bool`, *optional*, defaults to `False`): Whether to differ the seed on each device slightly with `self.process_index`. deterministic (`bool`, *optional*, defaults to `False`): Whether to use deterministic algorithms where available. Can slow down training. TN)r process_indexrandomrnptorch manual_seedrxpumanual_seed_allrnpur mlursdaar musar hpur neuroncudarxm set_rng_stateuse_deterministic_algorithms)rrrr&K/home/ubuntu/.local/lib/python3.10/site-packages/accelerate/utils/random.pyset_seed(s2      r(rng_type generatorcCs|tjkr t}n|tjkrtj}n|tjkr)ts!Jdtt }n|tj kr;t s5Jdtj }n}|tj krMtsGJdtj}nk|tjkr_tsYJdtj}nY|tjkrqtskJdtj}nG|tjkrts}Jdtj}n5|tjkrtsJdtj}n#|tjkrtsJdtj}n|tjkr|dusJd | }t!}|j"t#jkr|$t %}t &|gt '|(}nW|j"t)vs|j"t#j*ks|j"t#j+ks|j"t#j,ks|j"t#j-ks|j"t#j.ks|j"t#j/ks|j"t#j0kr |$|j1}tj23|d |(}n|j"t#j4kr.tj23|d |tjkr;t5|dS|tjkrItj5|dS|tj krWtj 5|dS|tj kretj5|dS|tjkrstj5|dS|tjkrtj5|dS|tjkrtj5|dS|tjkrtj5|dS|tjkrtj5|dS|tjkrt 5|6dS|tjkr|7|dSdS) Nz8Can't synchronize XLA seeds as torch_xla is unavailable.z;Can't synchronize NPU seeds on an environment without NPUs.z;Can't synchronize MLU seeds on an environment without MLUs.z=Can't synchronize SDAA seeds on an environment without SDAAs.z=Can't synchronize MUSA seeds on an environment without MUSAs.z;Can't synchronize XPU seeds on an environment without XPUs.z;Can't synchronize HPU seeds on an environment without HPUs.zFCan't synchronize Neuron seeds on an environment without Neuron Cores.z)Need a generator to synchronize its seed.r)8r TORCHr get_rng_stateCUDAr"XLArtensorr#NPUrrMLUr rSDAArrMUSAr rXPUrrHPUr r NEURONr r! GENERATOR get_staterdistributed_typerto xla_devicecollective_broadcast mark_stepcpur MULTI_MLU MULTI_SDAA MULTI_MUSA MULTI_NPU MULTI_XPU MULTI_HPU MULTI_NEURONdevice distributed broadcast MULTI_CPUr$item set_state)r)r* rng_statestater&r&r'synchronize_rng_stateQs                                     rN rng_typescCs|D] }tt||dqdS)N)r*)rNr )rOr*r)r&r&r'synchronize_rng_statessrP)FF)NN)N)#rtypingrrnumpyrrrMr constantsr dataclassesrr importsr r r r rrrrtorch_xla.core.xla_modelcore xla_modelr#intboolr( GeneratorrNliststrrPr&r&r&r's  ( )*R