o ߗiH@sXddlmZ ddlZddlZddlZddlZddlZddlZddlm Z m Z ddl m Z m Z mZmZddlZddlmZmZmZmZmZmZmZmZerSddlmZeeZejedZ gdZ!dd l"m#Z$d d Z%Gd d d Z&idej'dej(dej)dej*dej*dej+dej,dej-dej.dej/dej0dej1ddddej2dej3dej4d ej5id!ej6d"ej7d#ej8d$ej9d%ej9d&ej:d'ej:d(ej;d)ej<d*ej=d+ed,ed-ed.ed/ed0ej>d1ej>Z?hd2Z@d3d4ZAd5ZBeBD]&ZCd6eCZDd7eDZEeFe&eDeAeCeGeeEe?eD<e@)eDe!HeDqdhZIe@eIBZJhd8ZKhd9ZLdhZMeKeMBZNhd:ZOdd$hZPdd$d;ZQhd<ZReBD] ZCd6eCZDeR)eDqXhd=ZShd>ZTd?d@ZUdAdBZVdCdDZWdEdFZXdGdHZYdIdJZZdKdLZ[dMdNZ\dOdPZ]dQdRZ^dSdTZ_ UddVdWZ`dXdYZadZd[Zbe`ej=ej6eXeYeZe[eae\ebeUeVeWe]e^d\Zcd]d^Zdd_d`ZedadbZfdcddZgdedfZhdgdhZididjZjdkdlZkdmdnZldodpZmdqdrZndsdtZodudvZpejqeZrdwdxZseBD]ZCdyeCZteseCZueteu_veu_eFereteuq[u[C[tdzd{Zwdd|d}Zxd~dZyddZziecid/ej{dej'dehd"eidejdekdeldemdeedefd+eydegd#ej8d.end-eod,epdewexezdZ|eBD]ZCd6eCZDeGerdyeCe|eD<qk[C[D[B[rddZ}ddZ~ddZddZddZddZddZddZe}eeeeedZejejdZddZddZddZddZddZe|D] \ZZeeeqeD] \ZZeeeqddZe|D];\ZZeeKvreeeqeeOvreeeqeeMvseeLvreeeeeeeeQvr&eeeq[[dS)) annotationsN) lru_cacheupdate_wrapper)OptionalType TYPE_CHECKINGUnion) sym_floatsym_itesym_maxsym_minsym_notSymBoolSymFloatSymInt)ShapeEnvsym_node)SymNodemethod_to_operator magic_methods) py_sym_typescCs(|turtS|tur tS|turtS|SN)boolrintrfloatr)tr\/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/torch/fx/experimental/sym_node.py _to_symtype5src@seZdZUdZdZded<   dddd Zdd dZdddZdddZ e ddZ e ddZ ddZ dddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0Zd1d2Zd3d4Zd5d6Zd7d8Zdd:d;Zddd?Zddd@dAZ ddBdCZ!ddDdEZ"ddFdGZ#ddHdIZ$ddJdKZ%ddLdMZ&ddNdOZ'ddPdQZ(ddRdSZ)ddTdUZ*ddVdWZ+ddXdYZ,ddZd[Z-dd\d]Z.dd^d_Z/dd`daZ0ddbdcZ1ddddeZ2ddfdgZ3ddhdiZ4ddjdkZ5ddldmZ6ddndoZ7ddpdqZ8ddrdsZ9ddtduZ:ddvdwZ;ddxdyZdd~dZ?dddZ@dddZAdddZBdddZCdddZDddZEddZFddZGddZHddZIdddZJddZKddZLddZMdddZNddZOddZPddZQddZRddZSddZTddZUddZVddZWddZXdS)rz This is a type erased SymInt/SymFloat which we use to do actual operations. End users don't touch this. Magic methods are NOT defined on this object. Fr_optimized_summationNhint!Optional[Union[int, float, bool]]c s|_|_|_|_fdd}|durMt||us0t|t|us0Jd|dt|jrLjjrL|} || ksLJ|d| djdn|}|_|_ jo\jj} | o`|_ dS)NcsPddlm}|jr dSjjjdd}|dur&t|ts$|n|}|S)Nr)has_free_unbacked_symbolsT) compute_hint)%torch.fx.experimental.symbolic_shapesr"expr shape_env_maybe_evaluate_static isinstanceSymTypespytype)r"r selfrrr#{s  z&SymNode.__init__..compute_hintzCannot create SymNode of type z with incompatible hint of type z != z (for )) _exprr&r*rtyper_translation_validation_enabledr%_hintconstantfx_node) r,r%r&r*r r2r3optimized_summationr# computed_hinttx_validation_enrr+r__init__Qs2    zSymNode.__init__r&rreturncCst|j||j|j|j|jSr)rr.r*r1r2r3)r,r&rrrwith_shape_envszSymNode.with_shape_envothercCs<|j|jko|j|jko|j|jko|j|jko|j|jkSr)r.r*r1r2r3r,r:rrr _value_eqs     zSymNode._value_eqrcCst|j|j|j|j|jfSr)hashr.r*r1r2r3r+rrr _value_hashszSymNode._value_hashcCs|j|jSr)r&replacer.r+rrrr%sz SymNode.exprcCs|jSrr1r+rrrr sz SymNode.hintcCs |jduSrr@r+rrrhas_hint zSymNode.has_hintcCs*|jdur|dur |S|j|jS|jSr)r1r& size_hintr%)r,fallbackrrr require_hints zSymNode.require_hintcCs|jjr t|jSdSr)r% is_numberrr+rrr maybe_as_ints zSymNode.maybe_as_intcCs$ddl}t|j|jrt|jSdSNr)sympyr(r%Floatrr,rIrrrmaybe_as_floats zSymNode.maybe_as_floatcCs,ddl}|j|jur dS|j|jurdSdS)NrTF)rIr%truefalserKrrr maybe_as_bools   zSymNode.maybe_as_boolcC |jtuSr)r*rr+rrris_intrBzSymNode.is_intcCrPr)r*rr+rrris_floatrBzSymNode.is_floatcCrPr)r*rr+rrris_boolrBzSymNode.is_boolcCs"|jduot|jto|jjSr)r1r(rnode is_nested_intr+rrrrUs   zSymNode.is_nested_intcC4t|tusJddl}t|||jt|||dSNr)r2r3)r/rrIrIntegerr&r,numrIrrrwrap_int zSymNode.wrap_intcCrVrW)r/rrIrrJr&rYrrr wrap_floatr\zSymNode.wrap_floatcCs:t|tusJddl}t|r|jn|j|jt|||dSrW)r/rrIrrMrNr&rYrrr wrap_boolszSymNode.wrap_boolcC|Srrr+rrrclonez SymNode.clonecCs|jSrr%r+rrrstrz SymNode.strcC|Srrcr+rrr__str__rdzSymNode.__str__cCsd|jd|jd|jg}|jdur|d|j|jdur+|d|j|jdur9|d|jd|dS) NzSymNode(z , shape_env=z , pytype=zhint=z constant=zfx_node=z, r-)r.r&r*r1appendr2r3join)r,reprrr__repr__s   zSymNode.__repr__ builtins.strcCrerrfr+rrr _graph_repr%szSymNode._graph_reprcCrer)_absr+rrrabs+rdz SymNode.abscCrer)_posr+rrrpos.rdz SymNode.poscC ||Sr)_roundr,ndigitsrrrround1rBz SymNode.roundcCrer)_truncr+rrrtrunc4rdz SymNode.trunccCrrr)_addr;rrradd7rBz SymNode.addcCrrr)_subr;rrrsub:rBz SymNode.subcCrrr)_mulr;rrrmul=rBz SymNode.mulcCrrr)_modr;rrrmod@rBz SymNode.modcCrrr) _float_powr;rrr float_powCrBzSymNode.float_powcCrrr)_pow_by_naturalr;rrrpow_by_naturalFrBzSymNode.pow_by_naturalcCrrr)_and_r;rrrand_IrBz SymNode.and_cCrrr)_or_r;rrror_LrBz SymNode.or_cCrrr)_float_truedivr;rrr float_truedivOrBzSymNode.float_truedivcCrrr) _int_truedivr;rrr int_truedivRrBzSymNode.int_truedivcCrrr) _int_floordivr;rrr int_floordivUrBzSymNode.int_floordivcCrrr)_lshiftr;rrrlshiftXrBzSymNode.lshiftcCrrr)_rshiftr;rrrrshift[rBzSymNode.rshiftcCrer)_sym_notr+rrrr ^rdzSymNode.sym_notcCrrr)_eqr;rrreqarBz SymNode.eqcCrrr)_ner;rrrnedrBz SymNode.necCrrr)_gtr;rrrgtgrBz SymNode.gtcCrrr)_ltr;rrrltjrBz SymNode.ltcCrrr)_ler;rrrlemrBz SymNode.lecCrrr)_ger;rrrgeprBz SymNode.gecCrer)_floorr+rrrfloorsrdz SymNode.floorcCrer) _is_integerr+rrr is_integervrdzSymNode.is_integercCrer) _sym_floatr+rrrr yrdzSymNode.sym_floatcCrer)_sym_intr+rrrsym_int|rdzSymNode.sym_intcCrer)_ceilr+rrrceilrdz SymNode.ceilcCrer)_negr+rrrnegrdz SymNode.negcCrrr)_sym_minr;rrrr rBzSymNode.sym_mincCrrr)_sym_maxr;rrrr rBzSymNode.sym_maxcC |||Sr)_sym_ite)r,then_valelse_valrrrr  zSymNode.sym_itecCrr)_is_contiguousr,sizesstridesrrr is_contiguousrzSymNode.is_contiguouscCrr)_is_channels_last_contiguous_2drrrris_channels_last_contiguous_2drz&SymNode.is_channels_last_contiguous_2dcCrr)_is_channels_last_contiguous_3drrrris_channels_last_contiguous_3drz&SymNode.is_channels_last_contiguous_3dcCrr)_is_channels_last_strides_2drrrris_channels_last_strides_2drz#SymNode.is_channels_last_strides_2dcCrr)_is_channels_last_strides_3drrrris_channels_last_strides_3drz#SymNode.is_channels_last_strides_3dcCrr)'_is_non_overlapping_and_dense_indicatorrrrr&is_non_overlapping_and_dense_indicatorrz.SymNode.is_non_overlapping_and_dense_indicatorcCrrr)rr;rrrsym_orrBzSymNode.sym_orcCrrr)rr;rrrsym_andrBzSymNode.sym_andcCrrr) _bitwise_andr;rrr bitwise_andrBzSymNode.bitwise_andcCrrr) _bitwise_orr;rrr bitwise_orrBzSymNode.bitwise_orcCrrr)rr;rrrtruedivrBzSymNode.truedivcCrrr)rr;rrrfloordivrBzSymNode.floordivcCrrr)rr;rrrpowrBz SymNode.powcCs|||t|dS)N)rrto_noderrrris_non_overlapping_and_densesz$SymNode.is_non_overlapping_and_densecC |ddSNr) guard_intr+rrrint_rz SymNode.int_c Csddl}ddlm}m}|r!t||tjtdd|DfiSdd|D}|j|}g}d}|D]} | j dur<n | | j q3t |}|j tjtdd|Df\} } t||j t|| dS) Nrget_proxy_modehandle_sym_dispatchcss|]}t|VqdSr wrap_node.0arrr sz"SymNode.sym_sum..cSg|]}|jqSrrbrrrr z#SymNode.sym_sum..css|]}|jVqdSrr3rrrrrsr)rI"torch.fx.experimental.proxy_tensorrrrtorchsym_sumtupleAddr rhsumr&_create_fx_call_functionrr) r,argsrIrrexprsout size_hintsout_hintrr3_rrrrs0  zSymNode.sym_sumcC@|jj|j|j|jd}zt|WStytd|w)NrzFailed to convert to int: %s) r& evaluate_exprr%r r3r Exceptionlogwarningr,filelinerrrrr   zSymNode.guard_intcCr)NrzFailed to convert to float: %s) r&rr%r r3rrrrrrrr guard_floatrzSymNode.guard_floatcCr)NrFailed to convert to bool: %s r&rr%r r3rrrrrrrr guard_boolrzSymNode.guard_boolcCsRddlm}|r||js|jjs|||S|jj|j|d||jdS)Nr)free_unbacked_symbols:r) r$rrAr%r&+prefer_deferred_runtime_asserts_over_guardsrdefer_runtime_assertr3)r,rrrrrr expect_trues  zSymNode.expect_truecCsDddlm}||d}|||}|r |s |t||S)Nr)_advise_is_size)r$rrr[rrAr)r,rrrbrrrr expect_sizes    zSymNode.expect_sizecCsB|jj|j|j|jdd}zt|WSty td|w)aN Like guard_bool, but if we encounter unbacked symbols, if those symbols are size-like, we will treat them as >= 2 for the purposes of the analysis. This CHANGES the runtime semantics, but all size-oblivious sites have been audited to ensure that the runtime semantics don't change in a material way. Acceptable runtime semantic changes are, e.g., squeeze() no longer dropping an unbacked one size, or a tensor reporting as non-contiguous even if it's contiguous if it would have been reported contiguous due to being empty. T)r3size_obliviousrrrrrrguard_size_oblivious's    zSymNode.guard_size_obliviouscCrr)rr+rrrbool_=rz SymNode.bool_cCdS)NTrr+rrr is_symbolic@razSymNode.is_symboliccCsdSrrr+rrr nested_intCrazSymNode.nested_intcCr)NFrr+rrr is_constantFrazSymNode.is_constant)NNF)r r!)r&rr8r)r:rr8r)r8rr)r8rl)r8r)Y__name__ __module__ __qualname____doc__r__annotations__r7r9r<r>propertyr%r rArErGrLrOrQrRrSrUr[r]r^r`rcrgrkrmrorqrvrxrzr|r~rrrrrrrrrrr rrrrrrrrr rrrr r r rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrBs   R                                                &    rrqrorzandrrrrrxrrrrcCrer)rxrrrXsrrrrrr~rrorrrrrvrr|r r r r r rr> rorrqrrrxrr r cfdd}|S)Ncst|dS)N_sym_)getattrr+namerrfnsz_get_sym_node_fn..fnrrrrrr_get_sym_node_fn~s r) sqrtcoscoshsinsinhtantanhasinacosatanlog2sym_r>rrr r >rzr~r|>rvrsym_log2r)rr>rr rr>rrrxr> rrrrrrrrr rrcCddlm}|||S)Nr) FloatTrueDiv)torch.utils._sympy.functionsr*)rrr*rrr_sympy_float_truediv  r,cCr))Nr) IntTrueDiv)r+r.)rrr.rrr_sympy_int_truedivr-r/cCr))Nr)FloorDiv)r+r0)rrr0rrr_sympy_floordivr-r1cCs0ddlm}m}|jr|jr|||S|||S)NrMod PythonMod)r+r3r4is_nonnegative)rrr3r4rrr _sympy_mods   r6cCr))Nr) PowByNatural)r+r7)rrr7rrr_sympy_pow_by_naturalr-r8cCr))Nr)FloatPow)r+r9)rrr9rrr_sympy_float_powr-r:cCddl}|||SrH)rIAndrrrIrrr _sympy_and r>cCr;rH)rIOrr=rrr _sympy_orr?rAcCr))Nr)LShift)r+rB)rrrBrrr _sympy_lshiftr-rCcCr))Nr)RShift)r+rD)rrrDrrr _sympy_rshift r-rEcCst|dkr |gSddlm}m}||d||kr ||gS||d||kr/|g|Sdt|d}}||kra||d}||||}|dkrPdS|dkrY|d}n|d}||ks<||||S)zs If new_arg is found in ordered_args None is returned, else the new ordered_args with new_arg inserted r) _args_sortkeyBasicrN)lensympy.core.basicrFrGcompareinsert) ordered_argsnew_argsort_keyrGlowhighmidcompare_resultrrr_binary_search_insert_args&      rUFc sddlddlm}fdd}ddlm}|||O}|||O}|rR|rR||jd||jdkr<||j|jS||jd||jdkrR||j|jS|rg|jrgtt|j|}|durg||S|r||jr|tt|j|}|dur|||S ||}|||fS)a Custom optimization for Add used to optimize incremental binary summations of certain properties. The idea is when we know the expression is a summation of unique symbols all we need to know is the correct order of symbols, and no other optimizations are needed. We pass evaluate=false, with the correct order of args and save the following. 1. Avoid running other optimizations when the Add is constructed. 2. Manually figure out the order of the args for the new expression in log(n) comparisons instead of nLog(n) (comparing terms is expensive and shows in the profiles). The function returns a tuple of (1) a boolean that indicates whether the output is a summation of unique symbols, (2) the result sympy expression. rN)rFcsj|ddi}d|fS)NevaluateFT)r)rNresultrIrrmake_optimizedAsz&_optimized_add..make_optimized)_is_symbols_binary_summationrH) rIrKrFr+rZ_args is_symbolrUlistr) lhsrhslhs_is_optimized_summationrhs_is_optimized_summationsortkeyrYrZnew_argsrWrrXr_optimized_add1s*         rdcCr))Nr)BitwiseFn_bitwise_and)r+re)rrrerrrrbr-rcCr))Nr)BitwiseFn_bitwise_or)r+rf)rrrfrrrrhr-r)rzr|r~rrrrrrrrrrrrcCsddl}t||jr3|j}t|dkr3t|d|jr3|djr3||d}|d|kr3||dSt||jr@|||ksFt||jrK||S||S)NrrIr)rIr(MulrrJrJrrX)rrrIaacoefrrr_floor_ceil_helpers &     rjcCddlm}||S)Nr) FloorToInt)r+rl)rrlrrr _sympy_floor rmcCrk)Nr) TruncToInt)r+ro)rrorrr _sympy_truncrnrpcCrk)Nr) CeilToInt)r+rq)rrqrrr _sympy_ceilrnrrcCr;rH)rIEqr=rrr _sympy_eqr?rtcCr;rH)rINer=rrr _sympy_ner?rvcCr;rH)rIGtr=rrr _sympy_gtr?rxcCr;rH)rILtr=rrr _sympy_ltr?rzcCr;rH)rILer=rrr _sympy_ler?r|cCr;rH)rIGer=rrr _sympy_ger?r~cCr))Nr)Min)r+r)rrrrrr _sympy_minr-rcCr))NrMax)r+r)rrrrrr _sympy_maxr-rcCsddl}|||f|dfS)NrT)rI Piecewise)rrfrIrrr _sympy_itesrcr)Ncs"ddl}t|jjjd|S)NrOpaqueUnaryFn_)r+rutils_sympy functions)rrrrrrsz_get_sym_math_fn..fnrrrrr_get_sym_math_fns r_sympy_cCsddl}||SrH)rIAbsrrIrrr _sympy_abss rcCs*ddlm}m}|dur||S|||S)Nr) RoundDecimal RoundToInt)r+rr)numberrurrrrr _sympy_rounds rcCrkNr)ToFloat)r+r)rrrrr_sympy_sym_floats rcCs*ddl}ddlm}|||||Sr)rIr+rrsr)rrIrrrr_sympy_is_integer s r)rvrcCs$t|}t||tt|dddS)NrrH)rJsympy_is_contiguous_genericr]range)rrdimrrrsympy_is_contiguous0srcCsddl}t|}t||kr|jS|j}|jj}|D]}|||||jj||||BM}|||9}qt|D]}|||||jjO}q;|SrH) rIrJrNrMSOnersrZero)rr dim_orderrIrrzdrrrr5s ( rcCt||gdSN)rrIrrrrrrr$sympy_is_channels_last_contiguous_2dMrcCrN)rrrIrrrrrr$sympy_is_channels_last_contiguous_3dQrrc Csddl}ddlm}t|}|t|kr|jS|jj}|j}|||ddM}|D]+}||||d|||k@M}|dkrJ||||dM}|||||d}q*|S)Nrrr) rIr+rrJrNrrrMru) rrrrIrrmrrrrr&sympy_is_channels_last_strides_genericUs    rcCrrrrrrr!sympy_is_channels_last_strides_2d}rrcCrrrrrrr!sympy_is_channels_last_strides_3drrcCsddlm}|g||RS)Nr)!IsNonOverlappingAndDenseIndicator)r+r)rrrrrr-_sympy_is_non_overlapping_and_dense_indicators r)rrrrrr)r r cCsVt|tr|jSt|tur||St|tur||St|tur)| |St Sr) r(r)rTr/rr^rr[rr]NotImplemented)r,rZrrrrs       rcCsXt|tr |jdur |jS|rt|S|rt|S|r%t|St d|)Nzunrecognized return type ) r(rr2rQrrRrrSrAssertionErrorrrrrrsrcCst|Sr)METHOD_TO_OPERATORmethodrrrrrdrcstdtvrd}n}fdd}fdd}tvr/ttd||dSdkrEfdd }ttd||dSd kr\d fd d }ttd||dSttd||dS)Nrc sddlm}m}t}d}|jdur|jdur||j|j}t}|r6|dur6t||t|t|S|rHt|||t|t|fiSt |t sOJd}zTdkrddl m }m } |j} |jjsn| |jjdkr|jjs{| |jjdkr||j|j} n!| |j|j} ndkr|j|j|j|j\}} n|j|j} Wntytd|j|jwtd|j|j| tvrt} ntvrt} n|jtus|jturt} n|j} | dur|durt |ts| |}|j||j |j f\} }t | |j| || |d }|S) NrrFrr2rzfailed to eval %s(%s, %s)z%s %s %s -> %s)r3r4)!rrrrr alternate_impl_if_hinted_methodsgetrrr(rr+r3r4r&r%r5 bound_sympylowerrrrr sym_node_logdebugalways_float_magic_methodsralways_bool_magic_methodsrr*r)rr3)r,r:rropralternate_implr4r3r4r&rr*r3rrWfuncrrrbinary_magic_impls      z+_make_node_magic..binary_magic_implc sddlm}m}t}|rt|||t|fiS|j}dks&dkr,|j|}z|}Wnt yAt d|wt d||d}|jdurV||j}tvr]t}ntvrdt}n tvrkt}n|j}|j||jf\}} t||j|||dS)Nrrrceilingzfailed to eval %s(%s)z %s %s -> %sr)rrrrrrr%r&_simplify_floor_divrrrrrr always_int_magic_methodsrrrrrr*rr3r) r,rrrr%rrr*r3rrrrunary_magic_impl#s4     z*_make_node_magic..unary_magic_implr c sddlm}m}|jr|jn|j}|r&t||tt|t|t|fiSz |j|j|j}WntyEt d|j|j|jw|j t|j |j |j f\}}t||j |j||dS)Nrrzfailed to eval %s(%s, %s, %s)r)rrrr rr rr%rrrr&rr3rr*) pred_node then_node else_noderrrrr3rrrr sym_ite_implKs@   z&_make_node_magic..sym_ite_implrvc sddlm}m}tj}|rt|||t||fiS|j}z||}Wnty5t d||w|dur=t }n|j }d}|j durM||j |}|jg} |durZ| ||j|t| \} } t||j||| dS)Nrrz!failed to eval %s(%s, ndigits=%s)r)rrrbuiltinsrvrrr%rrrrr*r r3rhr&rrr) r,rurrrr%rr*rrr3rrrr round_implvs0     z$_make_node_magic..round_implr)r2magic_methods_on_operator_with_trailing_underscore unary_methodssetattrr)rr method_attrrrrrrrr_make_node_magics  ^$()rcsXfdd}ttd|fdd}ttjts*ttjt|dSdS)Nc sddlm}m}ttjt}|r(t|||dd|Ddd|DfiSdd|D}dd|D}z||}WntyMt d||wg} d} |D]} | j dur]n | | j qTg} |D]} | j durqn | | j qh|| | }  drt} nt} t||j| | S) NrrcSsg|]}t|qSrrrsrrrrszH_make_node_sizes_strides..sizes_strides_impl..cSrrrbrrrrrrcSrrrbrrrrrrr _indicator)rrrrsysmodulesr rrrrr rhendswithrrrr&)r,rrrrr size_exprs stride_exprsrrrr stride_hintsr*rrrsizes_strides_implsF     z4_make_node_sizes_strides..sizes_strides_implrcsddlddlm}t||D]"ttr2ttj fdd|Dfdd|DSqdkr<|||St fdd|Dfdd|DS) Nr)!eval_is_non_overlapping_and_densecg|]}tj|qSrrrTrrrrrrzH_make_node_sizes_strides..sizes_strides_user..crrrrrrrrrrcg|]}|qSrsympifyrrXrrrcrrrrrXrrrr) rIr$r itertoolschainr(rrrrTr)rrrrrrsizes_strides_users&    z4_make_node_sizes_strides..sizes_strides_user)rrhasattrrrr )rrrrrrr_make_node_sizes_stridess 0rc stvr dndddddtvrdd nd d fd d fd d}fdd}fdd}tvrXt|dd|dStvrlt|}t|t||dSdkrfdd}t|dd|dSdkrdfdd }t|dd|dS}tvrt}t|d|d|tvrt|d|d|dSdS)Nr'r2Union[SymInt, int, SymFloat, float, SymBool, bool]cSs4t|tttfr |St|tr|jddStd)Nrrz*expect to be called with constant SymBools)r(rrrrrTrrrrrr get_constant s  z&_make_user_magic..get_constantcSs2t|tttfr dSt|tttfr|jSdS)NTF) r(rrrrrrrTrrrrrrs  z%_make_user_magic..is_constantcSs"t|trt|jt|S|S)z;Implements True+True=2, which works in python but not sympy)r(rrrTr[rrrrrpromote5s z!_make_user_magic..promotecSr_rrrrrrr=racs\dvr||fSt|ttjf}t|ttjf}|s|r*|s#t|}|s*t|}||fS)N)rzr|r~rrrrr r rrrrrr)r(rrrr )r,r:f_selff_otherrrrpromote2@s  z"_make_user_magic..promote2cs2|}|rt|Stt|jSr)rrrrTr+)rrrrrrrrjsz*_make_user_magic..unary_magic_implcst|ttttttfs tSt d|||}|}||\}}|r1t ||S|r9|}t |j |}|turEtSt t|j |}|rW|S|S)NzMAGIC %s %s %s)r(rrrrrrrrrrrrTrrr,r: other_noderetrrrrrrrrrps z+_make_user_magic..binary_magic_implcst|ttttttfs tS|}|}||\}}|r)t||S|r1|}t |j |}|tur=tSt t ||j }|rO|S|Sr) r(rrrrrrrrrrTrrrrrrrbinary_magic_impls z,_make_user_magic..rbinary_magic_impl__r cs~|j}t||}t||}|tus|turtSt|tr't|tr'|j|jks)Jtt|j||}|jr=|S|Sr) rTrrr(rr*rrr)predrrrrrr)rrrrsym_ite_magic_impls  z,_make_user_magic..sym_ite_magic_implrvcs,|r t||Stt|j|Sr)rrvrrrTrt)rrrrrround_magic_implsz*_make_user_magic..round_magic_impl__r)rrr) rbool_becomes_int_magic_methodsunary_magic_methodsrunary_nonmagic_methodsrr bitwise_opsreflectable_magic_methods) r user_typerrrorigrr method_namerrr_make_user_magics<  !  * r)FFr) __future__rrrloggingmathoperatorr functoolsrrtypingrrrrrr r r r r rrrr$r getLoggerr r_logginggetArtifactLoggerr__all__ torch.typesrr)rrrqrorzrrrrrxrrrrrrrr~rrrrrvrr|rrrr math_op_namesrsym_name priv_sym_namerrrhrronly_bool_magic_methodsralso_bool_magic_methodsbool_magic_methodsonly_float_magic_methodsrrrrrr,r/r1r6r8r:r>rArCrErUrdrrrrjrmrprrrtrvrxrzr|r~rrrrcurrent_modulerpriv_sympy_namerr rrrrinvertrrrrrrrrrsizes_strides_methodsminmaxrrrrrritemsrrrrrrrs  (           !"%     # 1           (  hW8