o i|6 @s:UdZddlZddlZddlmZmZmZmZddlZddlm Z m Z ddl m Z mZmZddlmZddlmZee eefZejZejZd agaeeegdfeefed <eZe Z!eZ"e#ej$d rpej$j%Z&n d ed efddZ&e#ej$drej$j'Z(n d ed efddZ(ddZ)ddZ*d e+fddZ,dUddZ-Gddde.Z/d e+fddZ0d e+fdd Z1dVd eed dfd!d"Z2d efd#d$Z3d efd%d&Z4dVd eed efd'd(Z5dVd eed efd)d*Z6 dWd-eed.ed/ed dfd0d1Z7d e8eeffd2d3Z9d4eeeeegdfd dfd5d6Z:dXd8e+fd9d:Z;dVd eed eeeffd;d<ZZ=d?efd@dAZ>d ed dfdBdCZ?dVd eed e8eeffdDdEZ@GdFd d Z GdGdHdHZAdIdJZBd?edKd eAfdLd?ZCdYd eeeej fd e fdNdOZD MdYdPe d eeeej fd dfdQdRZEddSlFTgdTZGdS)ZzH This package enables an interface for accessing MTIA backend in python N)AnyCallableOptionalUnion)deviceTensor) _dummy_type_LazySeedTracker classproperty)Device)_get_device_indexF _queued_calls_mtia_exchangeDevicerreturncC|dkrdStdNrz)PyTorch was compiled without MTIA support RuntimeErrorrrP/home/ubuntu/veenaModal/venv/lib/python3.10/site-packages/torch/mtia/__init__.py_exchange_device%r_mtia_maybeExchangeDevicecCrrrrrrr_maybe_exchange_device/rrcCs tdSN) _lazy_initrrrrinit5 rcCs tot S)z9Return whether PyTorch's MTIA state has been initialized.) _initialized_is_in_bad_forkrrrris_initialized9s r#cC tjSr)torch_C_mtia_isInBadForkrrrrr">r r"c Cststtdr dStqtr WddStr tdts'tdtj dt_ t ddtDz1t D]'\}}z|Wq>tye}zdt|dd |}t||d}~wwWttdnttdwdaWddS1swYdS) Nis_initializingzwCannot re-initialize MTIA in forked subprocess. To use MTIA with multiprocessing, you must use the 'spawn' start methodzTorch not compiled with MTIA enabled. Ensure you have `import mtia.host_runtime.torch_mtia.dynamic_library` in your python src file and include `//mtia/host_runtime/torch_mtia:torch_mtia` as your target dependency!Tcss|]}|r|VqdSrr).0callsrrr dsz_lazy_init..z6MTIA call failed lazily at initialization with error: z( MTIA call was originally invoked at: )r#hasattr_tls_initialization_lockr"r _is_compiledAssertionErrorr%r& _mtia_initr(rextend_lazy_seed_tracker get_calls ExceptionstrjoinDeferredMtiaCallErrordelattrr!) queued_callorig_tracebackemsgrrrrBsB       "rc@s eZdZdS)r9N)__name__ __module__ __qualname__rrrrr9usr9cCr$)z*Return true if compiled with MTIA support.)r%r& _mtia_isBuiltrrrrr0y r0cCstsdStdkS)z'Return true if MTIA device is availableFr)r0 device_countrrrr is_available~s rEcCs:tj| tjWdS1swYdS)z?Waits for all jobs in all streams on a MTIA device to complete.N)r%mtiarr&_mtia_deviceSynchronizerrrr synchronizes$rHcCr$)z,Return the number of MTIA devices available.)r%r&_mtia_getDeviceCountrrrrrD rDcCr$)z0Return the index of a currently selected device.)r%r&%_accelerator_hooks_get_current_devicerrrrcurrent_devicerCrLcCtjt|ddS)aSReturn the currently selected :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the currently selected :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). Toptional)r%r&_mtia_getCurrentStreamr rrrrcurrent_stream rQcCrM)a=Return the default :class:`Stream` for a given device. Args: device (torch.device or int, optional): selected device. Returns the default :class:`Stream` for the current device, given by :func:`~torch.mtia.current_device`, if :attr:`device` is ``None`` (default). TrN)r%r&_mtia_getDefaultStreamr rrrrdefault_streamrRrTallpythonenabledstacks max_entriescCstsdStj|||dS)aEnable/Disable the memory profiler on MTIA allocator Args: enabled (all or state, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). stacks ("python" or "cpp", optional). Select the stack trace to record. max_entries (int, optional). Maximum number of entries to record. N)r#r%r&_mtia_recordMemoryHistory)rWrXrYrrrrecord_memory_historysr[cCr$)z4Return a dictionary of MTIA memory allocator history)r%r&_mtia_memorySnapshotrrrrsnapshotrJr]observercCstj|dS)z9Attach an out-of-memory observer to MTIA memory allocatorN)r%r&_mtia_attachOutOfMemoryObserver)r^rrrattach_out_of_memory_observersr`Tincluding_emulationcCsdS)NTr)rarrris_bf16_supportedsrbcCrM)aReturn capability of a given device as a tuple of (major version, minor version). Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). TrN)r%r&_mtia_getDeviceCapabilityr rrrrget_device_capabilityrdcCr$)zEmpty the MTIA device cache.)r%r&_mtia_emptyCacherrrr empty_cacherCrgstreamcCs|durdStj|dS)aSet the current stream.This is a wrapper API to set the stream. Usage of this function is discouraged in favor of the ``stream`` context manager. Args: stream (Stream): selected stream. This function is a no-op if this argument is ``None``. N)r%r&_mtia_setCurrentStreamrhrrr set_streams rkcCs$t|}|dkrtj|dSdS)zSet the current device. Args: device (torch.device or int): selected device. This function is a no-op if this argument is negative. rN)r r%r&%_accelerator_hooks_set_current_devicerrrr set_devicesrmcCrM)zReturn a dictionary of MTIA device properties Args: device (torch.device or int, optional) selected device. Returns statistics for the current device, given by current_device(), if device is None (default). TrN)r%r&_mtia_getDevicePropertiesr rrrrget_device_propertiesreroc@s<eZdZdZdefddZddZdededefd d Zd S) rzContext-manager that changes the selected device. Args: device (torch.device or int): device index to select. It's a no-op if this argument is a negative integer or ``None``. cCst|dd|_d|_dS)NTrNr)r idxprev_idx)selfrrrr__init__s zdevice.__init__cCstj|j|_dSr)r%r&(_accelerator_hooks_maybe_exchange_devicerprq)rrrrr __enter__szdevice.__enter__typevalue tracebackcCstj|j|_dS)NF)r%r&rtrqrp)rrrvrwrxrrr__exit__szdevice.__exit__N)r?r@rA__doc__rrsruryrrrrrs c@sNeZdZUdZeded<dedfddZddZd ed ed efd d Z dS) StreamContextaContext-manager that selects a given stream. All MTIA kernels queued within its context will be enqueued on a selected stream. Args: Stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: Streams are per-device. torch.mtia.Stream cur_streamrhcCstd|_||_tdd|_tjs|jdurd|_tjs dntjd|_ tjs1d|_ dStjd|_ dS)NTr) r}rhr rpr%jit is_scriptingrFrTsrc_prev_streamdst_prev_stream)rrrhrrrrs*s     zStreamContext.__init__cCs|j}|dus |jdkrdStjd|_|jj|jkr9t|jtj|j|_Wdn1s4wYtj|dSNr) rhrpr%rFrQrrrrk)rrr}rrrru9s zStreamContext.__enter__rvrwrxcCsJ|j}|dus |jdkrdS|jj|jkrtj|jtj|jdSr)rhrprrr%rFrkr)rrrvrwrxr}rrrryHs zStreamContext.__exit__N) r?r@rArzr__annotations__rsrurryrrrrr{s  r{cCstj|||dS)zset stream specified by the stream id, device index and device type Args: stream_id (int): stream id in stream pool device_index (int): device index in topo device_type (int): enum device type N)r%r&_mtia_setStream) stream_id device_index device_typerrr_set_stream_by_idVrerr|cCst|S)a,Wrap around the Context-manager StreamContext that selects a given stream. Arguments: stream (Stream): selected stream. This manager is a no-op if it's ``None``. .. note:: In eager mode stream is of type Stream class while in JIT it doesn't support torch.mtia.stream )r{rjrrrrhasrFcCs$tjdtddtjdgtj|dS)zReturns the random number generator state as a ByteTensor. Args: device (torch.device or int, optional): The device to return the RNG state of. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.get_rng_state is not implemented in torch.mtia stacklevelr )dtyper)warningswarn UserWarningr%zerosuint8rrrr get_rng_statels r new_statecCstjdtdddS)aSets the random number generator state. Args: new_state (torch.ByteTensor): The desired state device (torch.device or int, optional): The device to set the RNG state. Default: ``'mtia'`` (i.e., ``torch.device('mtia')``, the current mtia device). z.set_rng_state is not implemented in torch.mtiarrN)rrr)rrrrr set_rng_state{s   r)*)rrEr#rHrDrLrQrT memory_statsmax_memory_allocatedmemory_allocatedreset_peak_memory_statsrdror[r]r`rgrmrkrhrrrrb)rNr)rUrVr)T)rF)Hrz threadingrtypingrrrrr%r_devicer torch._utilsrr r torch.typesr _utilsr r7int _device_tEventStreamr!rlisttuplerlocalr.Lockr/r4r-r&rrrrrr#boolr"rr6r9r0rErHrDrLrQrTr[dictr]r`rbrdrgrkrmror{rrhrrmemory__all__rrrrs       3        :