o qoi@sdZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z m Z m Z mZmZddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZddlZd ZdZdZ dZ!dZ"ej#gggggdZ$e%ddej&'DZ(ede de fdZ)ej*ddGdddZ+Gddde,Z-Gdddej.Z/d d!Z0d"ej1d#ej2d$ej1fd%d&Z3d'ej4d(e5fd)d*Z6d'ej4d#ej2fd+d,Z7d-d.Z8d/d0Z9d1d2Z:d3e)d$e)fd4d5Z;ej*ddGd6d7d7Z< dUd8d8dde<dd9d:e=d;e e=dd=ee=d$e f d?d@Z? dUddAd;e e=d$e fdBdCZ@dDe d$e=fdEdFZAdGejBfdHdIZCd#e d$e5fdJdKZDd#e d$e=fdLdMZEGdNdOdOejFZGd#e de fd$e5fdPdQZHdRe de fd$e de ffdSdTZIdS)VaProvides utilities for transforming builder functions into `fdl.Config`s. This module defines the `auto_config` function (and associated helpers), which can be used to convert an existing function that creates an object graph into a function that creates a graph of `Config` and `Partial` objects. When the resulting graph of `Config` and `Partial` objects is built via `fdl.build()`, it will yield same object graph as the original function. N)AnyCallableOptionalTypeTypeVarcast) arg_factory)building)casting)config)copying)mutate_buildable)partial)auto_config_policy)daglish_legacy__auto_config_call_handler__!__auto_config_attr_load_handler__!__auto_config_attr_save_handler___attr_save_temp__auto_config_closure_wrapper__) posonlyargsargs kwonlyargs kw_defaultsdefaultscCs$g|]}t|st|r|qS)inspect isroutineisclass).0builtinrrX/home/ubuntu/.local/lib/python3.10/site-packages/fiddle/_src/experimental/auto_config.py 6sr"_GenericCallable.)boundT)frozencseZdZUdZedefed<edejfed<e ed<e ddZ dd Z d efd d Z d ejfd dZdddZe ddZfddZZS) AutoConfiga<A function wrapper for auto_config'd functions. In order to support auto_config'ing @classmethod's, we need to customize the descriptor protocol for the auto_config'd function. This simple wrapper type is designed to look like a `functool.wraps` wrapper, but implements custom behavior for bound methods. .funcbuildable_func always_inlinecCsdSNTrselfrrr!nowrapNszAutoConfig.nowrapc Cs@dD]}zt|j|}Wn tyYqwt|||qdS)N) __module____name__ __qualname____doc____annotations__)getattrr'AttributeErrorobject __setattr__)r,namevaluerrr! __post_init__Rs zAutoConfig.__post_init__returncO|j|i|SNr'r,rkwargsrrr!__call__czAutoConfig.__call__cOr;r<r(r>rrr! as_buildablefrAzAutoConfig.as_buildableNcCs$t|j|||j|||jdS)Nr'r(r))r&r'__get__r(r))r,objobjtyperrr!rEis   zAutoConfig.__get__cCs|jSr<r=r+rrr! __wrapped__rszAutoConfig.__wrapped__csttd|S)Nr')r3super__getattribute__)r,r7 __class__rr! __getattr__vszAutoConfig.__getattr__r<)r/r.r0r1rrr2r Buildableboolpropertyr-r9r@rCrErHrM __classcell__rrrKr!r&@s    r&c@s eZdZdS)!UnsupportedLanguageConstructErrorN)r/r.r0rrrr!rR|srRc@seZdZdZ d;dededefddZdejfd d Z d;dejd e fd d Z ddZ dej fddZdejfddZdejfddZdejfddZdejfddZdejfddZdejfddZdejfddZdejfd d!Zdejfd"d#Z dej!fd$d%Z"dej#fd&d'Z$dej%fd(d)Z&dej%fd*d+Z'dej(fd,d-Z)dej*fd.d/Z+dej,fd0d1Z-dej fd2d3Z.dej/fd4d5Z0dej1fd6d7Z2dej3fd8d9Z4d:S)<_AutoConfigNodeTransformerzEA NodeTransformer that adds the auto-config call handler into an AST.Fsourcefilename line_numbercCs,||_||_||_||_d|_d|_dS)a>Initializes the auto config node transformer instance. Args: source: The source code of the node that will be transformed by this instance. This is used for better error reporting. filename: The filename `source` is from. line_number: The line number of `source` within `filename`. allow_control_flow: Whether to permit control flow constructs (loops, conditionals, comprehensions, etc). By default, this is `False`, and control flow constructs will cause an `UnsupportedLanguageConstructError` to be raised. rN) splitlines_lines _filename _line_number_allow_control_flow_function_def_depth_temp_var_count)r,rTrUrVallow_control_flowrrr!__init__s  z#_AutoConfigNodeTransformer.__init__nodecCs0|j|jd}|j|jd}|j||j|fSN)rZlinenorXrY col_offset)r,r`rVlinerrr! _location_forsz(_AutoConfigNodeTransformer._location_for activatablecCs6|jr |r ||Sdt|jd}t|||)NzControl flow (z ) is unsupported by auto_config.)r[ generic_visittyper/rRrf)r,r`rgmsgrrr!_handle_control_flows  z/_AutoConfigNodeTransformer._handle_control_flowc Cs:z|jd7_||W|jd8_S|jd8_wra)r\rhr,r`rrr!_generic_visit_inside_functions  z9_AutoConfigNodeTransformer._generic_visit_inside_functionc Csg}|jD]"}t|tjr|j}t|tjr|j}t|tjr"|j}| |qz| d}Wn t y9YdSw|d|D]}|dvr]t d|d|j d|jd|jd|d q@dS) aValidates that decorators are applied in the right order. This is done on a best effort basis to catch cases where @classmethod or @staticmethod are applied on top of @auto_config. Args: node: The `ast.FunctionDef` node to validate decorators for. auto_configN) classmethod staticmethod@z' placed above @auto_config on function z at :z;. Reorder decorators so that @auto_config is placed above @.)decorator_list isinstanceastCallr' AttributeattrNameidappendindex ValueErrorAssertionErrorr7rYrZ)r,r`rt decoratorauto_config_indexrrr!_validate_decorator_orderings6      z7_AutoConfigNodeTransformer._validate_decorator_orderingcsFtjtjttd|jgfdd|jDfdd|jDdS)Nr{ctxc3s|]}|VqdSr<visitrargr+rr! sz8_AutoConfigNodeTransformer.visit_Call..csg|]}|qSrr)rkeywordr+rr!r"z9_AutoConfigNodeTransformer.visit_Call..r'rkeywords)rvrwrz_CALL_HANDLER_IDLoadr'rrrlrr+r! visit_Calls z%_AutoConfigNodeTransformer.visit_CallcCsLt|jtjr!tjtjttd||jtj |j dggdS| |SNr)sr) rurrvrrwrz_ATTR_LOAD_HANDLER_IDrr8Strryrhrlrrr!visit_Attributes z*_AutoConfigNodeTransformer.visit_Attributec sfdd}|j|_t|jdkr,t|jdtjr,||jdj|jdj|jSfdd}g}|jD]}t|tjsEt|tj ryg}|j D]*}t|tjro|}| |||j|j|} | | j d7_ qJ| |qJ||_ q7t|tjr|}||j|j|} | | ||j|j |<j d7_ q7t|tjr|j|_|j|_q7t|tjrq7t|tjrq7td|d|d||S) z"Handler assignment transformation.csHt|tjr |}ttjtjttd|tj |d|ggdSr) rurvrxrExprrwrz_ATTR_SAVE_HANDLER_IDrr)rFryr8r+rr!make_expr_calls  z?_AutoConfigNodeTransformer.visit_Assign..make_expr_callrbrcs"tjtdjtd}|S)N_r)rvrz_ATTR_SAVE_TEMP_VAR_IDr]Store)temp_varr+rr! make_temp_vars z>_AutoConfigNodeTransformer.visit_Assign..make_temp_varz$Cannot handle Assign statement with z as target.)rr8lentargetsrurvrxryTupleListeltsr|r]r} SubscriptsliceStarredrzNotImplementedErrorinsert) r,r`rrtransformed_nodestargetnew_eltseltr expr_noderr+r! visit_AssignsL                z'_AutoConfigNodeTransformer.visit_AssigncC|j|ddSNT)rgrkrlrrr! visit_For.z$_AutoConfigNodeTransformer.visit_ForcCrrrrlrrr! visit_While1rz&_AutoConfigNodeTransformer.visit_WhilecCrrrrlrrr!visit_If4rz#_AutoConfigNodeTransformer.visit_IfcCrrrrlrrr! visit_IfExp7rz&_AutoConfigNodeTransformer.visit_IfExpcCrrrrlrrr!visit_ListComp:rz)_AutoConfigNodeTransformer.visit_ListCompcCrrrrlrrr! visit_SetComp=rz(_AutoConfigNodeTransformer.visit_SetCompcCrrrrlrrr!visit_DictComp@rz)_AutoConfigNodeTransformer.visit_DictCompcCrrrrlrrr!visit_GeneratorExpCrz-_AutoConfigNodeTransformer.visit_GeneratorExpcC ||Sr<rrlrrr! visit_TryF z$_AutoConfigNodeTransformer.visit_TrycCrrrrlrrr! visit_RaiseIrz&_AutoConfigNodeTransformer.visit_RaisecCrr<rrlrrr! visit_WithLrz%_AutoConfigNodeTransformer.visit_WithcCrr<rrlrrr! visit_YieldOrz&_AutoConfigNodeTransformer.visit_YieldcCrr<rrlrrr!visit_YieldFromRrz*_AutoConfigNodeTransformer.visit_YieldFromcCsH|jdkrd}t||||||j}g|_||}||_|S)zTransforms a FunctionDef node.rz=Nested function definitions are not supported by auto_config.)r\rRrfrrtrm)r,r`rjrtrrr!visit_FunctionDefUs   z,_AutoConfigNodeTransformer.visit_FunctionDefcCs(|jdkrd}t|||||S)Nrz4Lambda definitions are not supported by auto_config.)r\rRrfrmr,r`rjrrr! visit_Lambdads  z'_AutoConfigNodeTransformer.visit_LambdacCd}t|||)Nz3Class definitions are not supported by auto_config.rRrfrrrr!visit_ClassDefkz)_AutoConfigNodeTransformer.visit_ClassDefcCr)Nz.traverse)rtraverse_with_path) structurerrrr!_contains_buildablevs  rmodulefnr:cslddtjddfdd|jjtttfD}tjtjt t g||j gdggd}t |}|S) aWraps `module.body` in a function that defines closure variables for `fn`. If `fn` has any free variables (i.e., it's `__code__.co_freevars` is not empty), we want to make sure that compiling its AST (assumed to be in the body of `module`) will create the same set of free variables in the resulting code object. However, by default this won't happen, since we would be compiling `fn`'s AST in the absence of its original context (e.g., just compiling a nested function, and not the containing one). To work around this issue, this function wraps `module.body` in another `FunctionDef` that defines dummy variables corresponding to `fn`'s free variables. This causes the subsequent compile step to create the right set of free variables, and allows us to use `fn.__closure__` when creating a new function object via `types.FunctionType`. We also add <_CALL_HANDLER_ID> as a final dummy variable, and append its value (the call handler) to `fn.__closure__` when creating the new function object. Effectively, this wrapping looks like the following Python code: def __auto_config_closure_wrapper__(): closure_var_1 = None closure_var_2 = None ... <_CALL_HANDLER_ID> = None def fn(...): # Or some expression involving a lambda. ... # Contains references to the closure variables. Args: module: An `ast.Module` object whose body contains the function definition for `fn` (e.g., as an `ast.FunctionDef` or `ast.Lambda`). fn: The function to create dummy closure variables for (assumed to correspond to the body of `module`). Returns: A new `ast.Module` containing an additional wrapper `ast.FunctionDef` that defines dummy closure variables. cSstj|tdS)Nr)rvrzr)r7rrr!sz4_wrap_ast_for_fn_with_closure_vars..N)r8cs g|] }tj|gdqS))rr8)rvr)rvar_nameast_nameast_nonerr!r"sz6_wrap_ast_for_fn_with_closure_vars..)r7rbodyrt)r type_ignores) rvConstant__code__ co_freevarsrrrModuler_CLOSURE_WRAPPER_ID_EMPTY_ARGUMENTSrfix_missing_locations)rrclosure_var_definitionswrapper_modulerrr!"_wrap_ast_for_fn_with_closure_varss.+    rcodefn_namecs8fdd|jD}t|dksJdd|dS)z?Finds the code object within `code` corresponding to `fn_name`.cs$g|]}t|r|jkr|qSr)riscodeco_name)rconstrrr!r"s z'_find_function_code..rbz Couldn't find function code for rsr) co_constsr)rrrrr!_find_function_codes rcCst|t}t||j}|S)acUnwraps `code` to find the code object for `fn`. This function assumes `code` is the result of compiling an `ast.Module` returned by `_wrap_node_for_fn_with_closure_vars`. Args: code: A code object containing code for `fn`. fn: The function to find a code object for within `code`. Returns: The code object corresponding to `fn`. )rrr/)rrrrr!_unwrap_code_for_fns rcs6ttdr tSfdd}t|jd}|S)z#Returns `types.CellType(contents)`.CellTypecsSr<rrcontentsrr!rsz$_make_closure_cell..r)hasattrtypesrri __closure__)rdummy_fn cell_typerrr!_make_closure_cells   rcCs6t|tjr t||St|r||Std|)aConverts an argument of an arg_factory partial() to Fiddle buildables. In normal Python, one expresses arg factories like, my_fn = arg_factory.partial(fn, foo=foo_factory, bar=bar_factory) where `foo_factory` produces a `foo` and `bar_factory` produces a `bar`. These are called each time `my_fn` is called. The Fiddle configuration for `my_fn`, on the other hand, looks like, my_fn_config = fdl.Partial(fn, foo=fdl.ArgFactory(foo_factory), bar=fdl.ArgFactory(bar_factory)) Therefore, we need to wrap `foo_factory` and `bar_factory` in `partial.ArgFactory`. Or, if they are already callable sub-configs, then we wrap them in ArgFactory. If `foo_factory` or `bar_factory` is not a callable or fdl.Partial, then we raise an error. It's techincally possible to pass `foo_factory` as a fdl.Config object that, when called, returns another fucntion, but this is most likely a mistake in configuration, so we don't allow it. Args: arg_factory_cls: The type to use when creating the ArgFactory (normally this will just be `fdl.ArgFactory`, but can potentially be customized). arg: Intermediate value passed to `arg_factory.partial`. Returns: ArgFactory version of a configuration or callable. z|Couldn't figure out how to handle arg_factory argument; please bind any constant args with a nested functools.partial. Arg: )rurPartialcast_librcallabler~)arg_factory_clsrrrr!_maybe_as_arg_factorys r cOsFt|tjr|rtd|tj|fi|S||g|Ri|S)aMakes a fdl.Partial, but calling appropriate APIs if casting is required. Args: partial_cls: The type to use when creating the Partial (normally this will just be `fdl.Partial`, but can potentially be customized). buildable_or_callable: Callable or existing configuration object to update. *args: Positional arguments, only supported when `config_or_callable` is a Partial already. **kwargs: Keyword arguments. Returns: New callable. zFor chained functools.partial calls inside auto_config, e.g. functools.partial(functools.partial(foo, ...), ...), only keyword arguments can be supplied to the outer call. Got: )rurr r~r copy_with) partial_clsbuildable_or_callablerr?rrr! _make_partials r fn_or_clscCst||ddS)aFWrap a callable so that it's exempted from auto_config. This can be used either as a decorator to exempt a function, or used inside an auto_config function to inline exempt certain calls to a function. During auto_config transformation, exempted function calls will be evaluated normally rather than turned into a config object. For example:: @exempt def my_square(x): return x * x @auto_config def build_model(): return Model(a=np.square(3), b=exempt(np.square)(3), c=my_square(3)) config = build_model.as_buildable() assert config.a == fdl.Config(np.square, 3) assert config.b == config.c == 9 Args: fn_or_cls: Any callable. Returns: A wrapped version of the same callable that will not be transformed to config if called inside an auto_config function. TrDr&)rrrr!exempt=src@sJeZdZUejZeejed<ej Z eej ed<ej Z eej ed<dS) ConfigTypes config_clsrr N) r/r.r0r Configrrr2rr r ArgFactoryr rrrr!r\s rF)-experimental_allow_dataclass_attribute_accessexperimental_allow_control_flowexperimental_always_inlineexperimental_exemption_policyexperimental_config_types*experimental_result_must_contain_buildablerrrrrcshdurddur tjfddd ddd ddfd d }|dur0|S||S) aRewrites the given function to make it generate a ``Config``. This function creates a new function from ``fn`` by rewriting its AST (abstract syntax tree), replacing all ``Call`` nodes with a custom call handler. When the rewritten function is run, the call handler intercepts calls and applies the following rules: - Calls to builtins, methods, callables without an inferrable signature, callables wrapped by `auto_config.exempt`, or other functions that have been ``auto_config``'ed take place as usual. - Calls to ``functools.partial`` are replaced by calling ``fdl.Partial`` with the same arguments; - All other calls are replaced by calling ``fdl.Config`` with the arguments that would have been passed to the called function or class. This function may be used standalone or as a decorator. The returned function is simply a wrapper around ``fn``, but with an additional ``as_buildable`` attribute containing the rewritten function. For example:: def build_model(): return Sequential([ Dense(num_units=128, activation=relu), Dense(num_units=128, activation=relu), Dense(num_units=1, activation=None), ]) config = auto_config(build_model).as_buildable() The resulting ``config`` is equivalent to the following "manually" constructed configuration graph:: fdl.Config(Sequential, layers=[ fdl.Config(Dense, num_units=128, activation=relu), fdl.Config(Dense, num_units=128, activation=relu), fdl.Config(Dense, num_units=1, activation=None), ]) This can then be built with ``fdl.build(config)``. Without modification, this will result in the same model as just calling ``build_model()`` directly. However, ``config`` permits changes to the model hyperparameters, for example:: config.layers[0].num_units = 64 config.layers[0].activation = 'elu' config.layers[1].num_units = 64 config.layers[1].activation = 'elu' modified_model = fdl.build(config) Currently, control flow is not supported by default in ``auto_config``. Experimental support for control flow can be enabled using the ``experimental_allow_control_flow`` argument. If enabled, control flow constructs may be used within the function to construct the resulting config (for example, a ``for`` loop could be used to build a list of layers). Control flow is never encoded directly as part of the resulting ``fdl.Config`` (for example, there is no ``fdl.Config`` that will correspond to a conditional or loop). While many simple constructs (``for _ in range(10)`` etc) work, there will also likely be surprising behavior in some circumstances (for example, using ``itertools`` functions in conjunction with a loop will not work, since the calls to ``itertools`` functions will be turned into ``fdl.Config`` objects). Using ``@auto_config`` is compatible with both ``@staticmethod`` and ``@classmethod``, however the ``@auto_config`` decorator must appear above the ``@classmethod`` or ``@staticmethod`` in the decorator list. Args: fn: The function to create a config-generating function from. experimental_allow_dataclass_attribute_access: Whether to allow attribute access on dataclasses within auto_config. Note that access to dataclass attribute is transformed into access to fdl.Config attributes in the as_buildable path. experimental_allow_control_flow: Whether to allow control flow constructs in ``fn``. By default, control flow constructs will cause an ``UnsupportedLanguageConstructError`` to be thrown. experimental_always_inline: If true, this function (when called in an ``auto_config`` context) will always be ``inline``'d in-place. See the documentation on ``inline`` for an example. The default (if unspecified) is currently ``False``, but this may change in the future. experimental_exemption_policy: An optional policy to control which function calls within the body of ``fn`` should be turned into ``fdl.Config``'s and which ones should simply be executed normally during the ``as_buildable`` interpretation of ``fn``. This predicate should return ``True`` if the given callable should be exempted from auto-configuration. experimental_config_types: A ``ConfigTypes`` instance containing the types to use when generating configs. By default, this just supplies the standard Fiddle types ()``fdl.Config``, ``fdl.Partial``, and ``fdl.ArgFactory``), but projects with custom subclasses can use this to override the default. This is experimental and may be removed in the future. experimental_result_must_contain_buildable: If true, then raise an error if `fn.as_buildable` returns a result that does not contain any `Buildable` values -- e.g., if it returns an empty dict. Returns: A wrapped version of ``fn``, but with an additional ``as_buildable`` attribute containing the rewritten function. NTcst|tr|jr|j|i|Sj}|tjur*t||dg|ddRi|S|tjurTj t||dgfdd|ddDRifdd| DS|t ur_||i|S|rj||i|Sj |g|Ri|S)ayHandles calls in auto_config'ed functions. This intercepts calls in an auto-configed function, and determines whether the called `fn_or_cls` should be wrapped in a `Config` or `Partial`. If `fn_or_cls` is `functools.partial`, the call will instead be converted into a call to Fiddle's `Partial`. If it is "auto-config eligible" (see `experimental_custom_call_policy`), then a `Config` will be create for `fn_or_cls` with the provided arguments. Otherwise, `fn_or_cls` is called directly. Args: fn_or_cls: The function or class being called. *args: The positional arguments with which `fn_or_cls` is being called. **kwargs: The keyword arguments with which `fn_or_cls` is being called. Returns: Depending on `fn_or_cls`, either `Partial`, a `Config`, or the result of calling `fn_or_cls` with the provided arguments. rrbNcsg|]}t|qSrr rr rr!r"rzAauto_config..auto_config_call_handler..csi|] \}}|t|qSrr)rr7rr rr! s zAauto_config..auto_config_call_handler..) rur&r)rCr functoolsrrrr itemsrr)rrr?r)rrr r!auto_config_call_handlers, $   z-auto_config..auto_config_call_handlercSsNt|tjr"|j}|rt|rt||Std|dt|dt||S)z5Handles attribute access in auto_config'ed functions.zCannot load attribute  on object of type p within auto_config, as this could lead to inconsistent behavior between the Python and as_buildable code paths.) rur rN __fn_or_cls__ dataclasses is_dataclassr3r~ri)r8ryallow_dataclassrrrr!auto_config_attr_load_handlers   z2auto_config..auto_config_attr_load_handlercSsNt|tjr%|j}|rt|rt|||dStd|dt|ddS)z6Handles saving attributes in auto_config'ed functions.NzCannot save attribute r%r&) rur rNr'r(r)setattrr~ri)rFryr8r*rrrr!auto_config_attr_save_handlers  z2auto_config..auto_config_attr_save_handlercsttjttfstddtdtttfr%t}jnd}t}t }j j }t |||d}t|}||}t|}t||d}t|tjs[Jt|}t|t d}t|}tjpsd}g} tftffD]\} } | |jvr|j| } ttj| d} | | | fqt!|jvr|jt!} t} | | | ft"| D] \} } |#| | qt$|}tj|j%|d j&_&j'_'rt(fd d } n} |r||| } t)| d S) Nz]`auto_config` is only compatible with functions, `@classmethod`s, and `@staticmethod`s. Got z with type rs)rTrUrVr^rbexecr)r*)closurecs8|i|}t|stdjdt|jd|S)Nz(The `auto_config` rewritten version of `z` returned a `z`, which is not (or did not contain) a `fdl.Buildable`. Please ensure this function returns the result of an `auto_config`-eligible call expression, or a supported container (list, tuple, dict) containing one.)r TypeErrorr0rir/)rr?outputauto_config_fnrrr!rCus z;auto_config..make_auto_config..as_buildable)r))*rur FunctionTyperorpr~ri__func__ _getsourcer getsourcefilerco_firstlinenorSrvparserrincrement_linenorrcompilerlistrrrrr}rr"rr|rsortedrtuple __globals__ __defaults____kwdefaults__wrapsr&)r method_typerTrUrVnode_transformerr`rr/indexed_handlers handler_idhandler handler_idxrC)r+r-r$rrrrr2r!make_auto_configs          z%auto_config..make_auto_config)T)rlatest)rrrrrrrrIr) r+r-r$rrrrrrr!rncsl -  qrnrcs2durddtffdd }|dur|S||S)a>Converts functions that create buildables directly into auto_config form. While most of the time, the benefits of an auto_config representation of object configuration and construction are valuable (e.g. static type checking and tooling / refactoring support), sometimes it is more convenient to manipulate buildable objects directly. ``auto_unconfig`` converts a function that directly manipulates ``fdl.Buildable``'s (e.g. ``fdl.Config``'s) into one that looks identically to an ``auto_config``'d function, and is fully interoperable with the rest of the ``auto_config`` ecosystem. Example:: @auto_unconfig def make_experiment_trainer(name: str) -> fdl.Config[MyTrainer] model = make_model.as_buildable(name) select(model, DropOut).set(rate=0.42) # Full Fiddle API available! dataset = make_dataset.as_buildable() # Build fdl.Config's imperatively. trainer_config = fdl.Config(MyTrainer) trainer_config.model = model trainer_config.train_dataset = dataset trainer_config.skip_eval = True return trainer_config # Return a `fdl.Buildable` # Sample usage within an auto_config'd function. @auto_config def make_driver(): return TrainerDriver( trainer=make_experiment_trainer('my_experiment'), checkpointer=CustomCheckpointer()) # Sample usage outside of auto_config contexts. def main(): # Use instantiated objects: trainer = make_experiment_trainer('my_experiment') for example in trainer.train_dataset: print_prediction(trainer.model.predict(example)) # Or manipulate the configuration before calling `fdl.build`: trainer_config = make_experiment_trainer.as_buildable('my_experiment') trainer_config.skip_eval = False # Tweak configuration. trainer2 = fdl.build(trainer_config) run_trainer(trainer2) Args: fn: The function to convert. experimental_always_inline: Whether the output of ``fn`` should always be inlined into the caller's config. Returns: An ``AutoConfig`` that corresponds to ``fn``. NTr:cs$tfdd}t|dS)Ncs>tjj}dtj_z|i|}t|W|tj_S|tj_w)NF)r _statein_buildbuild)rr?previouscfgrrr!python_implementations  zCauto_unconfig..make_unconfig..python_implementationrD)r"rBr&)rrRrKrQr! make_unconfigs z$auto_unconfig..make_unconfigr)rrrSrrKr! auto_unconfigs 8rTfunction_objectcCs t|tSr<)rur&)rUrrr!is_auto_configrrV buildablecCst|tjstdt|dt|jstd|jdtt|j}|j di|j }t|tj s7tdt j ||ddS) aConverts an ``auto_config``-based ``buildable`` into a DAG of Buildables. ``inline`` updates ``buildable`` in place to preserve aliasing within a larger Fiddle configuration. If you would like to leave ``buildable`` unmodified, make a shallow copy (``copy.copy``) before calling ``inline``. Example:: # shared/input_pipelines.py @auto_config(experimental_always_inline=False) def make_input_pipeline(name: str, batch_size: int) -> InputPipeline: file_path = '/base_path/'+name augmentation = 'my_augmentation_routine' # ... return InputPipeline(file_path, augmentation, ...) # config/main.py @auto_config def make_experiment(): data = make_input_pipeline('normal_dataset', batch_size) model = ... return Experiment(data, model) # experiment_configuration.py def make_experiment(): config = make_experiment.as_buildable() config.data.name = 'advanced_dataset' # config.data.augmentation = 'custom_augmentation' # Not configurable!!! # return fdl.build(config) # Works like normal. auto_config.inline(config.data) print(config.data.file_path) # Prints: '/base_path/advanced_dataset' config.data.augmentation = 'custom_augmentation' # Now exposed. experiment = fdl.build(config) # Works like normal. return experiment Args: buildable: The buildable of an ``auto_config``'d function to replace with the root of a Fiddle DAG that corresponds to it. Raises: ValueError: If ``buildable`` is not a ``Config``, or if ``buildable`` doesn't correspond to an ``auto_config``'d function. z'Cannot `inline` non-Config buildables; z is not compatible.z-Cannot `inline` a non-auto_config function; `z` is not compatible.zJYou cannot currently inline functions that do not return `fdl.Buildable`s.)rT destinationNr)rur rr~rirVr'rr&rC __arguments__rNr move_buildable_internals)rWr3 tmp_configrrr!inlines* ,    r\cCs t|rt|Stt|S)z*Returns the source code for callable `fn`.) _is_lambda_getsource_for_lambdatextwrapdedentr getsourcerQrrr!r6.sr6cCs<t|sdSt|drt|dsdS|jdkp|jjdkS)z*Returns True if `fn` is a lambda function.Fr/rz)r isfunctionrr/rrrQrrr!r]9s r]cs4eZdZdZejjfZfddZdddZ Z S) _LambdaFinderzFCST Visitor that searches for the source code for a given lambda func.cs$t||_|jj|_g|_dSr<)rIr_ lambda_fnrr8rc candidates)r,rdrKrr!r_Gs   z_LambdaFinder.__init__r:NcCs2|tjj|}|jj|jkr|j|dSdSr<) get_metadatacstmetadataPositionProviderstartrercrer|)r,r`locrrr!rMsz_LambdaFinder.visit_Lambda)r:N) r/r.r0r1rgrhriMETADATA_DEPENDENCIESr_rrQrrrKr!rcBs   rccCst|}t|}t||j}d|}t|}t |}tj | |t |jdkr<|jd}tj|gdjS|jsKtd|d|jdtd|d|jd ) z2Returns source code for the given lambda function.rbr)rz:Fiddle auto_config was unable to find the source code for z : could not find lambda on line rsz!: multiple lambdas found on line z); try moving each lambda to its own line.)r getmoduler7 linecachegetlines__dict__joinrg parse_modulercrhMetadataWrapperrrrerrr~rc)rrrUlinesrT module_cst lambda_finder lambda_noderrr!r^Ss2     r^r(csfdd}|S)z4A decorator that adds an auto_config-only code path.cst|ddS)NTrDrr=rBrr!rysz&with_buildable_func..decoratorr)r(rrrBr!with_buildable_functs ryr<)Jr1rvbuiltinsr(r"rror_rtypingrrrrrr fiddle._srcrr r r r r r rfiddle._src.experimentalrrlibcstrgrrrrr argumentsr frozensetrqvalues _BUILTINSr# dataclassr& SyntaxErrorrRNodeTransformerrSrrr4rCodeTyperrrrr rrrrOPolicyrnrTrVrr\r6r] CSTVisitorrcr^ryrrrr!s             ;w E  +  3 SD  !