o win2@sddlZddlZddlZddlmZeeZeej e e dZ Gddde ZeddZGdd d eZd d ZGd d d ZdddddZddddddZddddZddddZdS)N)contextmanagerzInstall GPU packages via `pip install --extra-index-url https://pypi.nvidia.com nemo-curator[cuda12x]` or use `pip install --extra-index-url https://pypi.nvidia.com ".[cuda12x]"` if installing from sourcec@seZdZdZdS)UnavailableErrorzDError thrown if a symbol is unavailable due to an issue importing itN)__name__ __module__ __qualname____doc__rrT/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/nemo/utils/import_utils.pyr#srcOs8t|dkrt|dkrt|dr|dSdd}|S)null_decoratorrcSs|SNr)funcrrr inner.znull_decorator..inner)lencallable)argskwargsrrrr r 's$r cseZdZdZfddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zd.d/Zd0d1Zd2d3Zd4d5Zd6d7Zd8d9Zd:d;Z dd?Z"d@dAZ#dBdCZ$dDdEZ%dFdGZ&dHdIZ'dJdKZ(dLdMZ)dNdOZ*dPdQZ+dRdSZ,dTdUZ-dVdWZ.dXdYZ/dZd[Z0d\d]Z1d^d_Z2Z3S)`UnavailableMetaauA metaclass for generating placeholder objects for unavailable symbols This metaclass allows errors to be deferred from import time to the time that a symbol is actually used in order to streamline the usage of optional dependencies. This is particularly useful for attempted imports of GPU-only modules which will only be invoked if GPU-only functionality is specifically used. If an attempt to import a symbol fails, this metaclass is used to generate a class which stands in for that symbol. Any attempt to call the symbol (instantiate the class) or access its attributes will throw an UnavailableError exception. Furthermore, this class can be used in e.g. isinstance checks, since it will (correctly) fail to match any instance it is compared against. In addition to calls and attribute access, a number of dunder methods are implemented so that other common usages of imported symbols (e.g. arithmetic) throw an UnavailableError, but this is not guaranteed for all possible uses. In such cases, other exception types (typically TypeErrors) will be thrown instead. cs>|dddur|d|d<d|}tt|||||S)N_msg could not be importedMISSING)getsuperr__new__)metanamebasesdct __class__rr rKs zUnavailableMeta.__new__cO t|jr rrclsrrrrr __call__Q zUnavailableMeta.__call__cCr!r r"r$rrrr __getattr__Tr&zUnavailableMeta.__getattr__cCr!r r"r$otherrrr __eq__Wr&zUnavailableMeta.__eq__cCr!r r"r)rrr __lt__Zr&zUnavailableMeta.__lt__cCr!r r"r)rrr __gt__]r&zUnavailableMeta.__gt__cCr!r r"r)rrr __le__`r&zUnavailableMeta.__le__cCr!r r"r)rrr __ge__cr&zUnavailableMeta.__ge__cCr!r r"r)rrr __ne__fr&zUnavailableMeta.__ne__cCr!r r"r$rrr __abs__ir&zUnavailableMeta.__abs__cCr!r r"r)rrr __add__lr&zUnavailableMeta.__add__cCr!r r"r)rrr __radd__or&zUnavailableMeta.__radd__cCr!r r"r)rrr __iadd__rr&zUnavailableMeta.__iadd__cCr!r r"r)rrr __floordiv__ur&zUnavailableMeta.__floordiv__cCr!r r"r)rrr __rfloordiv__xr&zUnavailableMeta.__rfloordiv__cCr!r r"r)rrr __ifloordiv__{r&zUnavailableMeta.__ifloordiv__cCr!r r"r)rrr __lshift__~r&zUnavailableMeta.__lshift__cCr!r r"r)rrr __rlshift__r&zUnavailableMeta.__rlshift__cCr!r r"r)rrr __mul__r&zUnavailableMeta.__mul__cCr!r r"r)rrr __rmul__r&zUnavailableMeta.__rmul__cCr!r r"r)rrr __imul__r&zUnavailableMeta.__imul__cCr!r r"r)rrr __ilshift__r&zUnavailableMeta.__ilshift__cCr!r r"r)rrr __pow__r&zUnavailableMeta.__pow__cCr!r r"r)rrr __rpow__r&zUnavailableMeta.__rpow__cCr!r r"r)rrr __ipow__r&zUnavailableMeta.__ipow__cCr!r r"r)rrr __rshift__r&zUnavailableMeta.__rshift__cCr!r r"r)rrr __rrshift__r&zUnavailableMeta.__rrshift__cCr!r r"r)rrr __irshift__r&zUnavailableMeta.__irshift__cCr!r r"r)rrr __sub__r&zUnavailableMeta.__sub__cCr!r r"r)rrr __rsub__r&zUnavailableMeta.__rsub__cCr!r r"r)rrr __isub__r&zUnavailableMeta.__isub__cCr!r r"r)rrr __truediv__r&zUnavailableMeta.__truediv__cCr!r r"r)rrr __rtruediv__r&zUnavailableMeta.__rtruediv__cCr!r r"r)rrr __itruediv__r&zUnavailableMeta.__itruediv__cCr!r r"r)rrr __divmod__r&zUnavailableMeta.__divmod__cCr!r r"r)rrr __rdivmod__r&zUnavailableMeta.__rdivmod__cCr!r r"r1rrr __neg__r&zUnavailableMeta.__neg__cCr!r r"r1rrr __invert__r&zUnavailableMeta.__invert__cCr!r r"r1rrr __hash__r&zUnavailableMeta.__hash__cCr!r r"r1rrr __index__r&zUnavailableMeta.__index__cCr!r r"r1rrr __iter__r&zUnavailableMeta.__iter__cCr!r r"r'rrr __delitem__r&zUnavailableMeta.__delitem__cCr!r r")r$rvaluerrr __setitem__r&zUnavailableMeta.__setitem__cOr!r r"r#rrr __enter__r&zUnavailableMeta.__enter__cOr!r r"r#rrr __get__r&zUnavailableMeta.__get__cOr!r r"r#rrr __delete__r&zUnavailableMeta.__delete__cCr!r r"r1rrr __len__r&zUnavailableMeta.__len__)4rrrrrr%r(r+r,r-r.r/r0r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQrRrTrUrVrWrX __classcell__rrrr r4sb rcCs t|tuS)z9Helper to check if given symbol is actually a placeholder)typer)objrrr is_unavailables r\c@s(eZdZdZddZddZddZdS) UnavailableNullContextzA placeholder class for unavailable context managers This context manager will return a value which will throw an UnavailableError if used in any way, but the context manager itself can be safely invoked. cOdSr rselfrrrrr __init__rzUnavailableNullContext.__init__cCstddddiS)NMissingContextValuerrz:Attempted to make use of placeholder context return value.)r)r`rrr rUs z UnavailableNullContext.__enter__cOr^r rr_rrr __exit__rzUnavailableNullContext.__exit__N)rrrrrarUrcrrrr r]s  r]msgaltc Cszt|dfWSty t}td|d|Yn ty+t}w|dur5|d}|durHt| dddd |id fS|d fS) aJA function used to import modules that may not be available This function will attempt to import a module with the given name, but it will not throw an ImportError if the module is not found. Instead, it will return a placeholder object which will raise an exception only if used. Parameters ---------- module: str The name of the module to import. msg: str or None An optional error message to be displayed if this module is used after a failed import. alt: object An optional module to be used in place of the given module if it fails to import Returns ------- Tuple(object, bool) The imported module, the given alternate, or a class derived from UnavailableMeta, and a boolean indicating whether the intended import was successful. T Import of  failed with: Nr.rrF) importlib import_module ImportError traceback format_excloggerdebug Exceptionrrsplit)modulererfexception_textrrr safe_imports   rvrerffallback_modulec Csz t|}t||dfWSty%t}td|d|Yn5tyN|dur:t ||||ddYSt}t d|d|d|Yn t yYt}w|durf|d|d}|durtt |d d |id fS|d fS) aA function used to import symbols from modules that may not be available This function will attempt to import a symbol with the given name from the given module, but it will not throw an ImportError if the symbol is not found. Instead, it will return a placeholder object which will raise an exception only if used. Parameters ---------- module: str The name of the module in which the symbol is defined. symbol: str The name of the symbol to import. msg: str or None An optional error message to be displayed if this symbol is used after a failed import. alt: object An optional object to be used in place of the given symbol if it fails to import fallback_module: str Alternative name of the model in which the symbol is defined. The function will first to import using the `module` value and if that fails will also try the `fallback_module`. Returns ------- Tuple(object, bool) The imported symbol, the given alternate, or a class derived from UnavailableMeta, and a boolean indicating whether the intended import was successful. TrgrhNrwz from rirrrF) rkrlgetattrrmrnrorprqAttributeErrorsafe_import_frominforrr)rtsymbolrerfrximported_modulerurrr r{s(     r{)rfcCst||dt|dS)a)A function used to import modules required only in GPU installs This function will attempt to import a module with the given name. This function will attempt to import a symbol with the given name from the given module, but it will not throw an ImportError if the symbol is not found. Instead, it will return a placeholder object which will raise an exception only if used with instructions on installing a GPU build. Parameters ---------- module: str The name of the module to import. alt: object An optional module to be used in place of the given module if it fails to import in a non-GPU-enabled install Returns ------- object The imported module, the given alternate, or a class derived from UnavailableMeta. zB is not enabled in non GPU-enabled installations or environemnts. rd)rvGPU_INSTALL_STRING)rtrfrrr gpu_only_importTs  rcCs t|||d|dt|dS)acA function used to import symbols required only in GPU installs This function will attempt to import a module with the given name. This function will attempt to import a symbol with the given name from the given module, but it will not throw an ImportError if the symbol is not found. Instead, it will return a placeholder object which will raise an exception only if used with instructions on installing a GPU build. Parameters ---------- module: str The name of the module to import. symbol: str The name of the symbol to import. alt: object An optional object to be used in place of the given symbol if it fails to import in a non-GPU-enabled install Returns ------- object The imported symbol, the given alternate, or a class derived from UnavailableMeta. rizB is not enabled in non GPU-enabled installations or environments. rd)r{r)rtr}rfrrr gpu_only_import_fromss r)rkloggingrn contextlibr getLoggerrrpsetLevelINFO addHandler StreamHandlerrrrrr rZrr\r]rvr{rrrrrr s(     )(5