o ei%@sFdZddlZddlZGdddeZedgdZGdddZdS) aA dependency graph for finding evaluation order. Example ------- >>> # The basic use case is that you have a bunch of keys >>> # and some of them depend on each other: >>> database = [] >>> functions = {'read': {'func': lambda: (0,1,2), ... 'needs': []}, ... 'process': {'func': lambda X: [x**2 for x in X], ... 'needs': ['read']}, ... 'save': {'func': lambda x: database.append(x), ... 'needs': ['process']}, ... 'print': {'func': lambda x,y: print(x, "became", y), ... 'needs': ['read', 'process']}, ... 'auxiliary': {'func': lambda: (1,2,3), ... 'needs': []}} >>> # If this is user supplied info, so you can't just hardcode the order, >>> # a dependency graph may be needed. >>> dg = DependencyGraph() >>> # In simple cases, you can just encode the dependencies directly: >>> for key, conf in functions.items(): ... for needed in conf["needs"]: ... dg.add_edge(key, needed) >>> # Now we can evaluate: >>> outputs = {} >>> for node in dg.get_evaluation_order(): ... f = functions[node.key]['func'] ... args = [outputs[needed] for needed in functions[node.key]['needs']] ... outputs[node.key] = f(*args) (0, 1, 2) became [0, 1, 4] >>> # This added nodes implicitly. >>> # However, since 'auxiliary' didn't depend on anything, >>> # it didn't get added! >>> assert 'auxiliary' not in outputs >>> # So to be careful, we should also manually add nodes for any thing that >>> # is not an intermediate step. >>> _ = dg.add_node('auxiliary') >>> assert 'auxiliary' in (node.key for node in dg.get_evaluation_order()) >>> # Arbitrary data can be added to nodes: >>> dg2 = DependencyGraph() >>> for key, conf in functions.items(): ... _ = dg2.add_node(key, conf) ... for needed in conf["needs"]: ... dg2.add_edge(key, needed) >>> # Now we get access to the data in evaluation: >>> outputs2 = {} >>> for key, _, conf in dg2.get_evaluation_order(): ... f = conf['func'] ... args = [outputs[needed] for needed in conf['needs']] ... outputs[key] = f(*args) (0, 1, 2) became [0, 1, 4] Authors: * Aku Rouhe 2020 Nc@seZdZdZdS)CircularDependencyErrorz An error caused by running into circular dependencies while searching for an evaluation order in a DependencyGraph. N)__name__ __module__ __qualname____doc__rrX/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/speechbrain/utils/depgraph.pyr>srDGNode)keyedgesdatac@s`eZdZdZddZeddZdddZd d Zd d Z d dZ dddZ ddZ ddZ dS)DependencyGraphaWGeneral-purpose dependency graph. Essentially a directed acyclic graph. Usually used to find an evaluation order for e.g. variable substitution The relation that an edge between A and B represents is: "A depends on B, i.e. B should be evaluated before A" Nodes can be added explicitly or they can be created implicitly while adding edges. Nodes have keys, which should be some hashable value that identifies the elements the graph represents in your use case. E.G. they can just be the variable name you want to substitute. However, if needed, more generally you can attach any data to a node (e.g. a path in your tree), and if so desired, a unique key can be created for you. You'll only need to know that key while adding edges to/from it. Implicit keys and explicit keys can also be mixed. cCsg|_i|_g|_dSN)digraphkey2ind_manually_added_keysselfrrr__init___s zDependencyGraph.__init__cCstS)z%Returns a unique hashable identifier.)uuiduuid4rrrrget_unique_keyeszDependencyGraph.get_unique_keyNcCs|dur |}n||jvrtdj|d|j|||jvr8|j|}|j|}t|j|j ||j|<|St |j|j|<|jt|g||S)aAdds a node explicitly. Arguments --------- key : hashable, optional If not given, a key is created for you. data : Any, optional Any additional data you wish to attach to this node. Returns ------- hashable The key that was used (either yours or generated). Raises ------ ValueError If node with the given key has already been added explicitly (with this method, not "add_edge"). NzAdding duplicate node: {key})r ) rr ValueErrorformatappendrrr r r len)rr r indnoderrradd_nodejs      zDependencyGraph.add_nodecCs:||}||}|j|j}||vr||dSdS)aAdds an edge, and implicitly also creates nodes for keys which have not been seen before. This will not let you add data to your nodes. The relation encodes: "from_key depends on to_key" (to_key must be evaluated before from_key). Arguments --------- from_key : hashable The key which depends on. to_key : hashable The key which is depended on. N)_get_ind_and_add_if_newrr r)rfrom_keyto_keyfrom_indto_ind edges_listrrradd_edges  zDependencyGraph.add_edgecCs8||jvrt|j|j|<|jt|gd|j|Sr)rrrrr rr rrrrs  z'DependencyGraph._get_ind_and_add_if_newcCs | S)a Checks if an evaluation order can be found. A dependency graph is evaluatable if there are no circular dependencies, i.e., the graph is acyclic. Returns ------- bool Indicating if the graph is evaluatable. )_find_first_cyclerrrris_valids zDependencyGraph.is_validc#sltfdd|durttj}n fdd|D}|D]}|gD]}j|Vq*q#dS)aFinds one valid evaluation order. There can be many different valid orders. NOTE: Generates output one DGNode at a time. May generate DGNodes before it finds a circular dependency. If you really need to know whether an order can be found, check is_valid() first. However, the algorithm for finding cycles is essentially the same as the one used for finding an evaluation order, so for very large graphs... Ah well, but maybe then you should be using some other solution anyway. Arguments --------- selected_keys : list, None List of keys. If not None, only the selected keys are guaranteed in the evaluation order (along with the keys they depend on). Yields ------ DGNode The added DGNodes in a valid evaluation order. See the DGNode namedtuple above. Raises ------ CircularDependencyError If a circular dependency is found. c3s||g}||vrtdjdfdd|Dd|vr"dSt|gj|jD]}||dD]}|Vq8q0|VdS)zImplementation of toposort.z{cycle}z -> c3s |] }tj|jVqdSr)strrr ).0irrr s zIDependencyGraph.get_evaluation_order..toposort..)cycleN)visited)rrjoinunionsetrr )root_indr.herer#r seen_everrtoposortrrr6s$  z6DependencyGraph.get_evaluation_order..toposortNcsg|]}j|qSrr)r*r rrr sz8DependencyGraph.get_evaluation_order..r1rangerr)r selected_keys start_inds start_indrrr4rget_evaluation_ordersz$DependencyGraph.get_evaluation_ordercsLtfddttjD]}|vr#|g}|r#|SqgS)zCDepth-first search based algorithm for finding cycles in the graph.csht||||g}||vr|S|vrgSt|gj|jD] }||}|r1|Sq$gS)zImplementation of cycle_dfs.)printr0r1rr )r2r.r3r#r- cycle_dfsr5rrrrAs   z4DependencyGraph._find_first_cycle..cycle_dfsr9)rrr-rr@rr's z!DependencyGraph._find_first_cyclecCs ||jvSrr7r&rrr __contains__s zDependencyGraph.__contains__)NNr)rrrrr staticmethodrrr%rr(r>r'rBrrrrr Ks  % = r )r collectionsrrr namedtupler r rrrrs 9