o ,wiV0 @s2dZddlZddlmZmZmZmZmZddlm Z defddZ ee j egedeeffZ de d ed efd d Z d ed ed eeee j ffddZd ed ed eeeffddZd ed ed ee j effddZee je j egedeeffZdefddZee jegedeeffZdefddZdS)zLibrary for manipulating DAGs.N)AnyCallableDict GeneratorList)daglishvaluecCstt|SN)ris_namedtuple_subclasstype)rr d/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/fiddle/_src/experimental/daglish_legacy.pyis_namedtuple_instancesrfn structurereturncs:ts tddtjdtdtffdd d|S)aTraverses `structure`, applying `fn` at each node. The given traversal function `fn` is applied at each node of `structure`, where a node may be a traversable container object or a leaf (non-traversable) value. Traversable containers are lists, tuples, namedtuples (without signature-changing `__new__` methods), dicts, and `fdl.Buildable` instances. `fn` should be a function taking two parameters, `path` and `value`, where `path` is a `Path` instance (tuple of `PathElement`s) and `value` provides the corresponding value. Additionally, `fn` must contain a single `yield` statement, which relinquishes control back to `traverse_with_path` to continue the traversal. The result of the traversal is then provided back to `fn` as the value of the yield statement (this communication between `fn` and `traverse_with_path` makes `fn` a "coroutine"). For example, a minimal traversal function might just be: def traverse(path, value): new_value = yield return new_value For leaf values, the `new_value` provided by the `yield` statement will be exactly `value`. For containers, `new_value` will have the same type as `value`, but its elements will be the result of the traversal function applied to the elements of `value`. (Note that this common pattern of simply returning the result of `yield` can be further simplified to just `return (yield)`.) If `fn` returns before reaching its `yield` statement, further traversal into `value` does not take place. In all cases, the return value of `fn` is used directly in place of `value` in the output structure. Note: The output of a traversal that returns the post-traversal value (e.g., `return (yield)`) will not maintain object identity relationships between different containers in `structure`. In other words, if (for example) the same list instance appears twice in `structure`, it will be replaced by two separate list instances in the traversal output. If maintaining object identity is important, see `memoized_traverse`. The following is a more involved example that replaces all tuples with a string (returning before `yield` to prevent traversal into the tuple), and also replaces all 2s with `None`, before then replacing `None` values in list containers with a string after traversal: structure = { "a": [1, 2], "b": (1, 2, 3), } def replace_twos_and_tuples(unused_path, value): if value == 2: return None elif isinstance(value, tuple): return "used to be a tuple..." # Provides the post-traversal value! Here, any value of 2 has # already been mapped to None. new_value = yield if isinstance(new_value, list): return ["used to be a two..." if x is None else x for x in new_value] else: return new_value output = daglish_legacy.traverse_with_path( replace_twos_and_tuples, structure) assert output == { "a": [1, "used to be a two..."], "b": "used to be a tuple...", } Args: fn: The function to apply to each node of `structure`. The function should be a coroutine with a single yield statement taking two parameters, `path` and `value`. structure: The structure to traverse. Raises: ValueError: If `fn` is not a generator function (i.e., does not contain a yield statement). RuntimeError: If `fn` yields a non-None value, or yields more than once. Returns: The structured output from the traversal. &`fn` should contain a yield statement.pathrrc s|}z?t|durtdtt|}|r9||}||\}}fddt||D}|||}n|}| |Wtdt yY} z | j WYd} ~ Sd} ~ ww)z#Recursive traversal implementation.Nz0The traversal function yielded a non-None value.c3s$|] \}}|f|VqdSr r ).0 path_elementsubtree)rtraverser r s  z7traverse_with_path..traverse..z,Does the traversal function have two yields?) next RuntimeErrorrfind_node_traverserr path_elementsflattenzip unflattensend StopIterationr) rr generator traverserrvaluesmetadata new_values new_structureerr)rr rzs&     z$traverse_with_path..traverser )inspectisgeneratorfunction ValueErrorrPathr)rrr r)r traverse_with_paths X r.memoizable_onlycs8stdidtjdtffdd }t||S)a]Returns a dict mapping id(v)->paths for all `v` traversable from structure. I.e., if `result = collect_paths_by_id(structure)`, then `result[id(v)]` is the list of every path `p` such that `follow_path(structure, p) is v`. This dict only includes values `v` for which `is_memoizable(v)` is true. Args: structure: The structure for which the id->paths mapping should be created. memoizable_only: If true, then only include values `v` for which `is_memoizable(v)` is true. Currently required to be True, to avoid bugs that can result from Python's interning optimizations. zIncluding non-memoizable objects when collecting paths by id may cause problems, because of Python's interning optimizations. If you are sure this is what you need, contact the Fiddle team, and we can look into enabling this flag.rrc3s,rt|rt|g|dVSr )r is_memoizable setdefaultidappendrrr/ paths_by_idr r collect_pathssz*collect_paths_by_id..collect_paths)r,rr-rr.)rr/r7r r5r collect_paths_by_ids r8c,idtjdtffdd }t||S)a Returns a dict mapping id(v)->v for all `v` traversable from `structure`. Args: structure: The structure for which the id->paths mapping should be created. memoizable_only: If true, then only include values `v` for which `is_memoizable(v)` is true. rrc3s$~r t|r|t|<dVSr )rr0r2r4r/ value_by_idr r collect_values  z*collect_value_by_id..collect_valuerr-rr.rr/r<r r:r collect_value_by_ids  r?cr9)a Returns a dict mapping `path->v` for all `v` traversable from `structure`. Args: structure: The structure for which the `path->value` map should be created. memoizable_only: If true, then only include values `v` for which `is_memoizable(v)` is true. rrc3srt|r ||<dVSr )rr0r4r/ value_by_pathr r r<sz,collect_value_by_path..collect_valuer=r>r r@r collect_value_by_paths  rBcsDts tdtdddtjdtffdd }t|S)aTraverses `structure`, providing all paths to each element. Unlike `traverse_with_path`, this function performs an initial traversal and collects all paths to each object in the provided `structure`. These paths (in addition to the current path) are then provided to `fn` during the main traversal. See `traverse_with_path` for additional details on how `fn` can control the traversal via `return` and `yield` statements. Args: fn: The function to apply to each node of `structure`. The function should be a coroutine with a single yield statement taking three parameters `all_paths`, `current_path`, and `value`. structure: The structure to traverse. Returns: The structured output from the traversal. rT)r/ current_pathrc3sdd}t|rt|}n|r)t|dd}t|}t||d}|||EdHS)N)r )rr0r2 follow_pathadd_path_element)rCr all_pathsparent parent_pathsr paths_memorr r wrap_with_pathss  z0traverse_with_all_paths..wrap_with_paths)r*r+r,r8rr-rr.)rrrLr rJr traverse_with_all_pathss   rMcs@ts tdidtjdtjdtffdd }t||S)a)Traverses `structure`, memoizing outputs. This simplifies the case where the traversal function `fn` should only be applied once to each instance of an object in `structure`. During the traversal, this function memoizes `fn`'s output for a given input object, and reuses it if the same object is encountered again. This behavior preserves instance relationships present in `structure`, and is most commonly the desired behavior when transforming a graph of `fdl.Buildable` objects. For example, `fdl.build()` should only build each `fdl.Buildable` instance once (and then reuse the result if an instance is encountered again). Like `traverse_with_all_paths`, an initial traversal collects all paths to each object in `structure`. However, since each object is encountered only once, no `current_path` parameter is supplied to `fn`, only the tuple of all paths. Note that immutable types that can't contain references (including most primitive types like strings and ints) are excluded from memoization. This avoids surprising interactions with Python's interning optimizations. See `is_memoizable` for additional details. See `traverse_with_path` for additional details on how `fn` can control the traversal via `return` and `yield` statements. Args: fn: The function to apply to each node of `structure`. The function should be a coroutine with a single yield statement taking two parameters, `paths`, and `value`. structure: The structure to traverse. Returns: The structured output from the traversal. rrGrCrc3sB~t|}|vr|S||EdH}t|r||<|Sr )r2rr0)rGrCrkeyoutputrmemor r wrap_with_memo4s z)memoized_traverse..wrap_with_memo)r*r+ TypeErrorrPathsr-rrM)rrrRr rPr memoized_traverse s " rU)__doc__r*typingrrrrr fiddle._srcrrr-TraverseWithPathFnr.boolintr8r?rBrTTraverseWithAllPathsFnrMMemoizedTraverseFnrUr r r r s8 v       (