o ̳im @sUddlZddlZddlZddlmZmZddlZddlZddl m Z ddl m Z m Z e Zejed< d deedeeeefdefd d ZdS) N)OptionalUnion)_broadcast_tensor) get_loggerget_world_size_and_rank_logseed debug_modereturnc CsJt\}}ttjj|d}ttjj}|dur)t||d}t|d }||ks1||kr?t d|d|d|d||}|dkrUt d |d |d |t |tj|t||durt d |t|t}|dkrt d dtjj_dtjj_|St ddtjj_dtjj_dtjd<|S)a Function that sets seed for pseudo-random number generators across commonly used libraries. This seeds PyTorch, NumPy, and the python.random module. For distributed jobs, each local process sets its own seed, computed seed + rank. For more details, see https://pytorch.org/docs/stable/notes/randomness.html. Args: seed (Optional[int]): the integer value seed. If `None`, a random seed will be generated and set. debug_mode (Optional[Union[str, int]]): Controls debug_mode settings for deterministic operations within PyTorch. * If `None`, don't set any PyTorch global values. * If "default" or 0, don't error or warn on nondeterministic operations and additionally enable PyTorch CuDNN benchmark. * If "warn" or 1, warn on nondeterministic operations and disable PyTorch CuDNN benchmark. * If "error" or 2, error on nondeterministic operations and disable PyTorch CuDNN benchmark. * For more details, see :func:`torch.set_deterministic_debug_mode` and https://pytorch.org/docs/stable/notes/randomness.html#avoiding-nondeterministic-algorithms. Returns: int: the current seed Raises: ValueError: If the input seed value is outside the required range. N)r rzInvalid seed value provided: z. Value must be in the range [z, ]z"Setting manual seed to local seed z. Local seed is seed + rank = z + z$Setting deterministic debug mode to z"Disabling cuDNN deterministic modeFTz!Enabling cuDNN deterministic modez:4096:8CUBLAS_WORKSPACE_CONFIG)rnpiinfouint32maxmintorchrandintritem ValueErrorrdebug manual_seedrandomrset_deterministic_debug_modeget_deterministic_debug_modebackendscudnn deterministic benchmarkosenviron) rr world_sizerankmax_valmin_val rand_seed local_seeddeterministic_debug_moder)K/home/ubuntu/.local/lib/python3.10/site-packages/torchtune/training/seed.pyset_seeds@           r+)NN)loggingr rtypingrrnumpyrrtorchtune.training._distributedrtorchtune.utilsrrrLogger__annotations__intstrr+r)r)r)r*s"