o qoi@sdZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z ej dde dfddZ    dd ed e d efd d ZejddGdddZdede fddZdS)zLazy imports API and utils.N)AnyIteratorOptionalTreturnccsD|stdtj}ztjt|tdt_dVW|t_dS|t_w)a Context manager which replaces import statements with lazy imports. A ProxyObject is returned for the import statement as dummy module. The actual module is not imported until calling `fdl.build()` on Fiddle `Buildable` objects. Usage: ```python with fdl.experimental.lazy_imports(): import foo.bar.baz as model config = fdl.Config(model.MyModel, layer_num=3, dtype='float16') my_model = fdl.build(config) # The module is not actually loaded until now. ``` Warning: Please import modules only when using lazy imports, and do not import classes or functions directly. Because it would confuse the API what's the correct module path to import. ``` # Errors will be raised when importing classes/functions directly. with fdl.experimental.lazy_imports(): from foo.bar.baz.my_module import MyModel # Wrong usage! # Instead, you should import the module directly only. with fdl.experimental.lazy_imports(): from foo.bar.baz import my_module # Correct usage! config = fdl.Config(my_module.MyModel, ...) ``` Be aware that when using lazy imported modules within `fdl.Partial`, lazy imported modules are not loaded until calling the object after `fdl.build()`. This is differet from `fdl.Config` where the modules are loaded when calling `fdl.build()`. ``` with lazy_imports.lazy_imports(kw_only=True): from path.to.the.module import module_1 from path.to.the.module import module_2 config = fdl.Config(module_1.MyModel) my_model = fld.build(config) # module_1 is loaded here. partial = fdl.Partial(module_2.my_function) my_fn = fdl.build(partial) # module_2 not loaded here. assert isinstance(my_fn.func, ProxyObject) out = my_fn(x=1, ...) # module_2 is loaded here during call time. ``` NOTE: When using positional arguments for lazy-imported functions/classes, be aware that the config will likely be different from normal-imported functions/classes. As Fiddle does not have signature information for lazy-imported modules, values for positional arguments, like value 'v' in the example below, will be saved as positional-only arguments, and using the index 0 as the key in `__arguments__`, even if it actually corresponds to a positional-or-keyword argument. # In foo.py def bar(x): return x # In file_1.py with fdl.lazy_import(): import foo cfg_1 = fdl.Config(foo.bar, 'v') assert cfg_1.__arguments__ == {0: 'v'} # In file_2.py import foo cfg_2 = fdl.Config(foo.bar, 'v') assert cfg_2.__arguments__ == {'x': 'v'} Args: kw_only: Whether to only allow keyword args usage for lazy imported modules. Defaults to True. Yields: None zWhen using positional arguments for lazy-imported functions/classes, be aware that the config will likely be different fromnormal-imported functions/classes.)kw_only proxy_clsN)loggingwarningbuiltins __import__ functoolspartial _fake_import ProxyObject)r origin_importrY/home/ubuntu/.local/lib/python3.10/site-packages/fiddle/_src/experimental/lazy_imports.py lazy_importssVrrnamefromlistlevelc Cs~~|r td|d|d^}}|j||d} d| _|s-| } |D]}| |} q#| S|D]}| |} q/|D]}| |q9| S)zMock of `builtins.__import__`.z*Relative import statements not supported (z)..)rrT) ValueErrorsplit from_cache is_import child_import) rglobals_locals_rrrr root_namepartsrootchildsrrrrs     rF)eqc@seZdZUdZeed<dZeed<dZe ded<dZ eed<dZ e ed<e ejd d d d Zed efddZd efddZded e fddZddZddZded dfddZddZdS)rz.Base class to represent a module, function,...rFrNparentr loaded_obji)maxsizecKs|di|S)zFactory to cache all instances of module. Note: The cache is global to all instances of the `fake_import` contextmanager. Args: **kwargs: Init kwargs Returns: New object Nrr)clskwargsrrrrszProxyObject.from_cachercCs:|js|jS|jjr|jsd}nd}|jj||jS)N:r)r$rrqualname)self separatorrrrr*s zProxyObject.qualnamecCst|jd|jdS)N())type__name__r*r+rrr__repr__szProxyObject.__repr__cCs|dkr$tdtjj}tdtjj}|jrt|gSt||gS|dkr+|jS|dkr5d|jSt|j||j|dd}|S) N __signature__argsr( __qualname__r0 ProxyObject_F)rrr$r) inspect ParameterVAR_POSITIONAL VAR_KEYWORDr Signaturer*r/r)r+rr4r(retrrr __getattr__s, zProxyObject.__getattr__cCst|j|j|jfS)N)hashr*r$rr1rrr__hash__szProxyObject.__hash__cCs&t|tsdS|j|jko|j|jkS)NF) isinstancerr*r$)r+otherrrr__eq__s zProxyObject.__eq__cCst||}d|_|S)zReturns the child import.T)getattrr)r+robjrrrrs zProxyObject.child_importcOs@|jdur t|j|_|jr|rtd|dS|j|i|S)NzXThis lazily imported function/class is keyword arguments only.Got positional arguments: z. )r% import_moduler*rr)r+r4r(rrr__call__s   zProxyObject.__call__)r0 __module__r5__doc__str__annotations__rboolr$rrr%r classmethodr lru_cacherpropertyr*r2r=r?rBrrFrrrrrs$        r qualname_strcCs||d}t|dkr|\}}nt|dkr#|\}|jddd\}}ntd|t|}|dD]}t||}q4|S)z4Import module/function/class from a module qualname.r)r)maxsplitzInvalid )rlenrsplitr importlibrErC)rOr import_str attributesrDattrrrrrEs      rE)T)NNrr)rHr contextlib dataclassesr rUr7rtypingrrrcontextmanagerrrIintr dataclassrrErrrrs2j )_