o Xi@sUdZddlmZddlZddlZddlZddlmZmZm Z ddl Z ddl m mZddlm mZddlm mZddlm mZddlmmZddl mZddlmZmZe dZejZdZej Gd d d Z!d3d dZ"GdddZ#GdddZ$d4ddZ%Gddde#Z&Gdddej'Z(Gddde(Z)d5d$d%Z*d6d+d,Z+e,eej-iZ.d-e/d.<d/Z0d e/d0<Gd1d2d2Z1dS)7zRewrite rules for ONNX models.) annotationsN)CallableSequenceTypeVar)ir)_tape convenienceTz!pkg.onnxscript.rewriter.rule_namec@s:eZdZUdZded<ded<ded<ded<d ed <d S) ReplacementSubgraphz1A subgraph that will replace the matched pattern._basics.MatchResultmatchSequence[ir.Value] new_outputsSequence[ir.Node] new_nodesnew_initializersz_tape.UsedOpsets used_opsetsN)__name__ __module__ __qualname____doc____annotations__rrU/home/ubuntu/.local/lib/python3.10/site-packages/onnxscript/rewriter/_rewrite_rule.pyr #s  r returnboolcOdS)zA condition function that always returns True. This is used when no condition function is provided for a rewrite rule. Tr)argskwargsrrr always_true.rc@sBeZdZdZ    d%d&ddZd'ddZddddd(d#d$ZdS))PatternzA pattern that can be matched against nodes in an ONNX graph. This class encapsulates pattern matching functionality, providing the ability to match patterns against nodes without requiring replacement functionality. Nrtarget_pattern#_pattern_ir.GraphPattern | Callablecondition_functionCallable | Nonematcher^_matcher.PatternMatcher | Callable[[_pattern_ir.GraphPattern], _matcher.PatternMatcher] | Noneverboseintname str | NonerNonecCsnt|tjs t|}||_|pt|_t|tjr||_n|dur)t |j|_n||j|_||_ ||_ dS)aCreate a pattern matcher. Args: target_pattern: The _pattern_ir.GraphPattern that will be matched against the IR. If a callable is provided, it will be converted to a _pattern_ir.GraphPattern. condition_function: The condition function that will be used to check if the pattern match found should be rewritten. matcher: The pattern matcher that will be used to match the pattern. If not provided, a default matcher will be used. verbose: The verbosity level of the rule. name: An optional name for the pattern that will show up in verbose logging. N) isinstance _pattern_ir GraphPattern_to_graph_pattern_target_patternr_condition_function_matcherPatternMatcherSimplePatternMatcher_verboser*)selfr"r$r&r(r*rrr__init__=s      zPattern.__init__strcC|jr|jSdS)NzAnonymous Patternr*r7rrr__str__azPattern.__str__Tr(check_nodes_are_removabletracermodelir.Modelgraph_or_functionir.Graph | ir.Functionnodeir.Node int | Noner@rrA_basics.MatchingTracer | None_basics.MatchResult | Nonecs|r |dkr td|dur|nj}jj|||drt|}jjD]}|jdurC|jj vrC |jdq/d fdd } dd} j D]\} } | j durv| | j || } | sv| | d | | SqXj D]\}}|j dur| |j ||} | s| | d ||Sq|| j|fij }|s| |d SrtjjSrʈtjjS) tCheck if the node matches the pattern and return the match result. Args: model: The model containing the graph or function. graph_or_function: The graph or function to match against. node: The node to try to match the pattern against. verbose: The verbosity level of messages. check_nodes_are_removable: If True, validate that matched nodes can be safely removed. tracer: The tracer for debugging. Returns: MatchResult if the pattern matches successfully and passes the condition function, None otherwise. z[match] N)r( remove_nodescsHt|tjr|j|jn||r"tjjdS)z4Local utility to handle check failures consistently.N) r-_basics MatchResultfailreasonfailure_nodes_and_valueslog MatchStatusCONDITION_FAILED) check_resultdefault_messagefailure_objectrDr rFr7rArrrPs  zPattern.match..failcsfdd}|S)z4Encapsulates try-except pattern for check functions.c sXz|i|WStjy+}zt}||jt|j|WYd}~Sd}~wwN)rNMatchFailureErrorrOrPrQlistfailure_sources)rreresultfrrwrappedsz0Pattern.match..wrap_try..wrappedr)rarbrr`rwrap_trys zPattern.match..wrap_tryz)Node-level check failed for pattern node z+Value-level check failed for pattern value zCondition function check failedrZ)printr6r3r rN MatchContextr1inputsr*bindingsbind node_bindingsitems check_methodvalue_bindingsr2rSrTSUCCESSNO_MATCH)r7rBrDrFr(r@rAcontextvarrPrc pattern_nodeir_noderV pattern_valueir_valuecheck_match_resultrrYrr ds`       z Pattern.match)NNrN) r"r#r$r%r&r'r(r)r*r+rr,rr9rBrCrDrErFrGr(rHr@rrArIrrJ)rrrrr8r=r rrrrr!6s  $ r!c@s$eZdZdZd ddZd d d Zd S)ReplacementPatternFunctionzThe replacement pattern that will replace the targeted pattern. Attributes: function (Callable): The replacement function that will be used to replace the matched pattern. rr,cCs ||_dSrZ) _function)r7functionrrrr8s z#ReplacementPatternFunction.__init__r r ReplacementSubgraph | NonecCsLt}|j|fi|j}|durdSt|ts|g}t|||j|j|jSrZ) RewriterContextryrgr-rr nodes initializersr)r7r rorrrrget_replacements z*ReplacementPatternFunction.get_replacementN)rr,)r r rr{)rrrrr8rrrrrrxs rxrDrEdeltac Csl|j}|jD]-\}}||vr|dur|nd||<q|dur3|||kr3td|d||d|dqdS)NzMultiple versions of opset z used. Expected version , but got .) opset_importsr ValueError)rDrimportsdomainversionrrr_update_opset_importssrcsjeZdZ        d2d3fdd Zd4ddZdddd5d)d*Zdddd+d6d-d.Zd7d0d1ZZS)8 RewriteRuleNrTFr"r#replacement_pattern%ReplacementPatternFunction | Callabler$r%r&r'r(r)r*r+rMrgraph_pre_visitorCallable[[], None] | Nonegraph_post_visitor as_functionrr,c sX| r|stdt|||||t|tst|}||_||_||_| |_| |_ dS)aCreate a rewrite rule. Args: target_pattern: The _pattern_ir.GraphPattern that will be matched against the IR. If a callable is provided, it will be converted to a _pattern_ir.GraphPattern. replacement_pattern: The ReplacementPatternFunction that will be used to replace the matched pattern. If a callable is provided, it will be converted to a ReplacementPatternFunction. condition_function: The condition function that will be used to check if the pattern match found should be rewritten. matcher: The pattern matcher that will be used to match the pattern. If not provided, a default matcher will be used. verbose: The verbosity level of the rule. name: An optional name for the pattern that will show up in verbose logging. remove_nodes: If True, the matched nodes will be removed from the graph. graph_pre_visitor: A function that will be called before applying the rewriting to the top-level graph or a function. graph_post_visitor: A function that will be called after the rewriting is complete for a graph or function. as_function: If True, the matched nodes will be extracted into a model local function. This is only supported when remove_nodes=True and when the replacement subgraph has a single node, representing the function call. z:as_function=True is only supported when remove_nodes=True.N) rsuperr8r-rx_replacement_patternrMrrr) r7r"rr$r&r(r*rMrrr __class__rrr8s'  zRewriteRule.__init__r9cCr:)NzAnonymous Ruler;r<rrrr=+r>zRewriteRule.__str__r(rArBrCrDrErFrGrHrArIr{cCs|j|||||j|d}|sdS|j|}|dur)|r'|||||tjjdSt|j |j j krBt d|j j dt|j dt ||t |j||S)zTIf the node matches the pattern, then replace the node with the replacement pattern.r?NzsNumber of outputs from replacement function does not match the number of outputs from the target pattern. Expected rr)r rMrrrSrNrTREPLACEMENT_FAILEDlenrr1 num_outputsrrgraph)r7rBrDrFr(rAr replacement_subgraphrrr try_rewrite.s@    zRewriteRule.try_rewrite)commuter(rArcCst|g|dj|||dS)Nrr)RewriteRuleSetapply_to_model)r7rBrr(rArrrrYs zRewriteRule.apply_to_modelSequence[RewriteRule]cs$fddfddjDS)Nc s8tj}t|jj||jjjjj j S)zHReturn a shallow copy of self with node_pattern replaced by new_pattern.) typer3rrr2r6r*rMrrr) new_pattern matcher_classr<rrreplace_patternhs z,RewriteRule.commute..replace_patterncg|]}|qSrr).0p)rrr yz'RewriteRule.commute..)r1rr<r)rr7rrgs zRewriteRule.commute)NNrNTNNF)r"r#rrr$r%r&r'r(r)r*r+rMrrrrrrrrr,rv) rBrCrDrErFrGr(rHrArIrr{)rBrCrrr(rHrArI)rr) rrrr8r=rrr __classcell__rrrrrs& 5 /rc@sHeZdZdZdd ddZejd d Zd!d d Zddddd"ddZ dS)# PatternBaseaBase class for implementing pattern matching as a class. This class encapsulates the pattern definition and condition checking without the replacement functionality. Example:: class TransposePattern(PatternBase): def pattern(cls, op, x, perm): return op.Transpose(x, perm=perm) def check(cls, context, x: ir.Value, perm: ir.Attr) -> bool: if perm.is_ref(): return False if perm.type == ir.AttributeType.INTS: if list(perm.as_ints()) == list(range(len(perm.as_ints()))): return True return False Nr*r+rr,cKs|p|jj|_d|_||_dSrZ)rrr*_compiled_pattern_pattern_kwargs)r7r*rrrrr8s zPatternBase.__init__cOtd)Nz6Method 'pattern' must be implemented by derived class.NotImplementedErrorr7oprrrrrpatternzPatternBase.patternr cOstS)zaDefault check function that returns a _basics.MatchResult object with success always set to True.)rNrOrrrrcheckrzPatternBase.checkTr?rBrCrDrErFrGr(rHr@rrArIrJcCsB|jdurt|j|jfd|ji|j|_|jj||||||dS)rKNr*r?)rr!rrr*rr )r7rBrDrFr(r@rArrrr s zPatternBase.matchrZ)r*r+rr,)rr rw) rrrrr8abcabstractmethodrrr rrrrr|s    rcsPeZdZdZeddZ ddfdd ZejddZ ddZ ddZ Z S)RewriteRuleClassBasea Base class for implementing rewrite rules as a class. Example:: class TransposeIdentity(RewriteRuleClassBase): def pattern(cls, op, x, perm): return op.Transpose(x, perm=perm) def check(cls, context, x: ir.Value, perm: ir.Attr) -> bool: if perm.is_ref(): return False if perm.type == ir.AttributeType.INTS: if list(perm.as_ints()) == list(range(len(perm.as_ints()))): return True return False def rewrite(cls, op, x: ir.Value, perm: ir.Attr | None = None): return op.Identity(x) # Then use # TransposeIdentity.rule() # to create a RewriteRule object. c Os6||i|}t|j|j|j|j|j|j|j|jdS)N)r*rMrrr) rrrewriterr*rMsetupcleanupr)clsrrinstancerrrruleszRewriteRuleClassBase.ruleNTFr*r+rMrrrr,cst|||_||_dSrZ)rr8rMr)r7r*rMrrrrr8s  zRewriteRuleClassBase.__init__cOr)Nz6Method 'rewrite' must be implemented by derived class.rrrrrrrzRewriteRuleClassBase.rewritecCr)zOptional setup function that can be overridden by derived classes. Used to do per model/function initialization. Nrr<rrrrr zRewriteRuleClassBase.setupcCr)z}Optional cleanup function that can be overridden by derived classes. Used to do per model/function cleanup. Nrr<rrrrr zRewriteRuleClassBase.cleanup)NTF)r*r+rMrrrrr,) rrrr classmethodrr8rrrrrrrrrrrs  rrfSequence[ir.Value | None]r}routputsr csig}g|D]"}|rtj|j|j|j|jdnt}|dur%||<||qdfdd dd d dfdd fdd|D}fdd|D}|||fS)z9Utility function to extract a subgraph out as a function.)r*shaper doc_stringNvalueir.Value | Nonercsx|durdS|vr8|j}|dur0td|}tddg|g}||jd|<}|Std|d|S)NrConstantrzValue z not found in value_map.) const_valuer AttrTensorNodeappendrr)rr value_attr const_noder_)constant_nodes value_maprr copy_values  z&_copy_for_function..copy_valueattrir.AttrcSs2|rtd|jtjjtjjhvrtd|S)NzRefAttr not supported.zGraph attributes not supported.)is_refrrr AttributeTypeGRAPHGRAPHS)rrrrcopy_attr_value.s z+_copy_for_function..copy_attr_valuerFrGc sfdd|jD}fdd|jD}tj|j|j|||jt|j d|j |j |j d }|j }t|j D]\}}|||<|j durM|j ||_ q8|S)Ncrrrrvrrrr;rz9_copy_for_function..copy_node..crrrr)rrrr<r)overloadrrr*rmetadata_props)rf attributesvaluesrrrop_typerrrr*rrcopy enumerate)rF new_inputsnew_attributesnew_noderioutput)rrrrr copy_node:s*    z%_copy_for_function..copy_nodecrrrrrF)rrrrPrz&_copy_for_function..crrrrrrrrQr)rrrr)rrrr)rFrGrrG)rValuer*rrrr)rfr}rfunction_inputsinput new_valuefunction_nodesfunction_outputsr)rrrrrr_copy_for_functions,     rrBrCrr9r*cCs0|j}d} t|}|||f|vr|S|d7}q)aGet a new overload for the given domain and name. Args: model: The model to which the new overload will be added. domain: The domain of the new overload. name: The opname of the new overload. Returns: The new overload name. r) functionsr9)rBrr*existing_functionsr overload_namerrr_get_new_overloadUs rzmetadata_merger.MetadataMerger_default_metadata_mergerTmerge_metadatac@sPeZdZddd d d Zd!d d Zddd"ddZdddd#ddZddZdS)$rFrrulesrrrrr,cCsH|std|rttjdd|D}||_tdd|D|_dS)Nz$rules must contain at least one rulecSsg|]}|qSrrrrrrrrxrz+RewriteRuleSet.__init__..css|]}|j VqdSrZ)rMrrrr |sz*RewriteRuleSet.__init__..)rr\ itertoolschain from_iterableranyremove_unused_nodes)r7rrrrrr8ts zRewriteRuleSet.__init__r9cCs|jjd|jdS)N())rrrr<rrr__repr__~szRewriteRuleSet.__repr__N)rArBrCrDrEr(rHrArIr)c sd}|jD] }|jr|q|D]i}|jD])}|j|||||d}|dus,|dur-qt|ts4J|jrlt|tjrG|rFtd|q|j } |jD]} | j | vr`|r_td| j dqMqM|jD]} | | | j <qdt j |j|jrt|jdkrtd|jd} | j} | j} t|| | }|| _t|jjfd d |D}t| j||jj\}}}d d |D|j}fd d|D}tj||||d}tj| | ||dd}||j| <|r |j r|j dnd} td| dt!"|jjtdt!"|jtd|j r|jD] }|j |j#t$<qt%&|||j'r'|jjng|j|jj|j(t)r=t*+|jj|j|d7}|j,-D]3}|j.tj/j0kr_||j1||j2||d7}qG|j.tj/j3kry|j2D]}||j1||||d7}qjqGq|jD] }|j4r|4q|S)a Apply the rewrite rules to the given graph or function. Args: model: The model to which the rewrite rules are applied. graph_or_function: The graph or function to which the rewrite rules are applied. verbose: The verbosity level. Defaults to None. tracer: The tracer for debugging. Defaults to None. Returns: The number of rewrite rules applied. rrNz:Rewrites adding initializers not supported for functions: z Initializer z already exists.rzOas_function=True is only supported for patterns with a single replacement node.csg|]}|vr|qSrr)rn)unsorted_nodesrrrsz>RewriteRuleSet._apply_to_graph_or_function..cSsh|]}|jqSr)rrrrr sz=RewriteRuleSet._apply_to_graph_or_function..csi|] \}}|vr||qSrr)rkr) used_domainsrr sz>RewriteRuleSet._apply_to_graph_or_function..)r}rr)rrz: rz----zMatched Nodes----z++++Replacement Nodes++++z++++End Replacement Nodes++++)5rrrr-r rrFunctionrdr~r* onnxscript optimizerbasic_constant_propagationrrrrrrrrsetr r}rrfrrrjGraphr identifier _ir_utils display_nodesr RULE_NAME_TAGrreplace_nodes_and_valuesrMrrrcopy_merged_metadatarrrrr_apply_to_graph_or_functionrrr)r7rBrDr(rAcountrrFrr~ initializer call_noderr*roriginal_nodesrfr}rparent_opset_importsused_opset_importsrrarrr)rrrrs                        z*RewriteRuleSet._apply_to_graph_or_functionrcCst|tjsJtj|jt|j }|j ||j||d}|D]}tj|||j ||||d7}q"|j r?tj ||S)aApply the rewrite rules in the set to the model. Args: model: The model to which the rewrite rules are applied. verbose: The verbosity level of messages. Defaults to None. tracer: if specified, no changes are made to the model, only information about the best matches found is computed. Returns: The number of applications of rewrite rules. r) r-rModelrr r rr\rrrr)r7rBr(rAoriginal_functionsrrzrrrr s    zRewriteRuleSet.apply_to_modelccs|jEdHdSrZ)rr<rrr__iter__.szRewriteRuleSet.__iter__)rrrrrr,rv) rBrCrDrEr(rHrArIrr))rBrCr(rHrArIrr))rrrr8rrrrrrrrrss   #r)rr)rDrErr )rfrr}rrr )rBrCrr9r*r9rr9)2r __future__rr dataclassesrtypingrrronnxscript.optimizerronnxscript.rewriter._basicsrewriterrNonnxscript.rewriter._ir_utilsronnxscript.rewriter._matcherr3onnxscript.rewriter._pattern_irr. onnxscript.utils.metadata_mergerutilsmetadata_mergerr onnxscript.irrrr Builderr|r dataclassr rr!rxrrABCrrrrMetadataMergercomma_separator_mergerrrrrrrrrsF    J B M