o ϯi_ @snUddlZddlZddlZddlmZddlmZddlmZddlm Z ddlm Z m Z m Z m Z mZmZmZmZmZddlZddlZddlmZddlmZddlmZdd lmZmZd d lmZed d dZ hdZ!ee"e#d<eGdddZ$dej%de ee"ej%ffddZ&dej%deeeedffde ee"ej%fe"fdej'j(fddZ)Gdd d Z*dS)N)Counter)copy) dataclass)Number) AnyDictIteratorListOptionalSetTupleTypeVarUnion_named_modules_with_dup)Tensor)_get_trace_graph TracerWarning)HandleTJitModelAnalysis)bound>:aten::taten::eqaten::geaten::gtaten::to aten::Int aten::cat aten::max aten::full aten::ones aten::relu aten::roll aten::size aten::view aten::chunk aten::clamp aten::copy_ aten::empty aten::floor aten::index aten::relu_ aten::slice aten::split aten::stack aten::zeros aten::arange aten::clamp_ aten::detach aten::expand aten::gather aten::narrow aten::select aten::unbind aten::__and__ aten::dropout aten::flatten aten::nonzero aten::permute aten::reshape aten::squeeze aten::type_asaten::new_fullaten::full_likeaten::new_emptyaten::new_zerosaten::remainderaten::transposeaten::unsqueezeaten::contiguousaten::index_put_aten::unsqueeze_aten::zeros_likeaten::bitwise_notaten::masked_fillaten::floor_divideaten::ScalarImplicitaten::constant_pad_ndaten::split_with_sizes _IGNORED_OPSc@s*eZdZUdZded<ded<ded<dS) StatisticszY For keeping track of the various model statistics recorded during analysis. zDict[str, Counter[str]]countsunsupported_opszSet[str] uncalled_modsN)__name__ __module__ __qualname____doc____annotations__r]r]J/home/ubuntu/.local/lib/python3.10/site-packages/fvcore/nn/jit_analysis.pyrTZs  rTmodelreturnccs:t}t|D]\}}||vr||||fVqdS)zd Like .named_modules(), but the results are slightly different for some wrapped models. N)setradd)r_seennamemodr]r]r^_named_modules_without_dupfs  rfmoduleinputs.aliasesc sGdddGdddgdtjdtddffd d }t|tjjjtjfr7||}|j}|||t |D] \}}||}|||q;t ||\}}D]} | qR|S) a Traces the provided module using torch.jit._get_trace_graph, but adds submodule scope information to each graph node. The resulting graph is in-lined and has all model parameters treated as inputs. The input model has the scope name '', while its descendants have names of the form 'child.grandchild.grandgrandchild...'. Args: model (nn.Module) : The module to trace inputs (tuple) : Inputs used during the trace of the model aliases (dict(str or nn.Module, str) : maps modules and module names to the canonical name to be used as the scope for that module. Returns: graph (torch._C.Graph) : The pytorch JIT trace of the model c@s6eZdZdeddfddZdejdedefdd ZdS) z._get_scoped_trace_graph..ScopePushHookrdr`NcSs ||_dSN)rdselfrdr]r]r^__init__s z7_get_scoped_trace_graph..ScopePushHook.__init__rgrhcSstj}|r ||j|Srj)torch_C_get_tracing_state push_scoperd)rlrgrh tracing_stater]r]r^__call__s  z7_get_scoped_trace_graph..ScopePushHook.__call__) rXrYrZstrrmnnModulerrsr]r]r]r^ ScopePushHooksrwc@s(eZdZdejdededefddZdS)z-_get_scoped_trace_graph..ScopePopHookrgrhoutputsr`cSstj}|r ||Srj)rnrorp pop_scope)rlrgrhrxrrr]r]r^rss z6_get_scoped_trace_graph..ScopePopHook.__call__N)rXrYrZrurvrrsr]r]r]r^ ScopePopHooks rzrerdr`Ncs2||}|}||dSrj)register_forward_pre_hookregister_forward_hookappend)rerdprehookposthookrzrw hook_handlesr]r^register_hookss  z/_get_scoped_trace_graph..register_hooks) rurvrt isinstanceparallel distributedDistributedDataParallel DataParallelrgrfrremove) rgrhrir root_namerdregraph_handler]rr^_get_scoped_trace_graphrs"     rc @s&eZdZdZdejdeeeedffddfddZ dCd e de fd d Z dCd e de je fd dZdee e je ffddZde je fddZdCd e de je fddZdee fddZdeeddfddZdDddZde de fddZ  dEdeejd edeeedffddfd!d"Zd#ed$e defd%d&Zd#ed$e defd'd(Zd#ed)edefd*d+Z d#ed)edefd,d-Z!d.e je ddfd/d0Z"d1ee ddfd2d3Z#dejdeee ejfe ffd4d5Z$d e dee fd6d7Z%dFd9d:Z&d;e de fde defd?d@Z(defdAdBZ)dS)GraK Provides access to per-submodule model statistics obtained by tracing a model with pytorch's jit tracing functionality. Calculates a statistic on a per-operator basis using the provided set of functions that acts on the inputs and outputs to the operator, then aggregates this over modules in the model. Can return the aggregate statistic for any submodule in the model. Is lazily evaluated, and will perform the trace when a statistic is first requested. Changing the operator handles will cause the trace to be rerun on the next request. Submodules may be referred to using the module's name. The input model has name "", while its descendants have names of the form "child.grandchild.grandgrandchild...". An operator is treated as within the scope of a module if calling that module directly resulted in that operator being run. In particular, this means that calls to other functions owned by a module or explicit calls to module.forward(...) will not register resulting operators as contributing statistics to that module. r_rh.r`NcCsh||_||_i|_tt||_|||_d|_t t |_ | d| d|d|ddS)a Args: model: The model to analyze inputs: The inputs to the model for analysis. We will trace the execution of `model.forward(inputs)`. This means inputs have to be tensors or tuple of tensors (see https://pytorch.org/docs/stable/generated/torch.jit.trace.html#torch.jit.trace). In order to trace other methods or unsupported input types, you may need to implement a wrapper module. NTno_tracer_warningowner)_model_inputs _op_handlesdictr_named_modules _get_aliases_aliases_statsrrS _ignored_opsunsupported_ops_warningsuncalled_modules_warningstracer_warnings ancestor_mode)rlr_rhr]r]r^rms     zJitModelAnalysis.__init__ module_namecCs(|}||}t|j|}|S)a( Returns the total aggregated statistic across all operators for the requested module. Args: module_name (str) : The submodule to get data for. Defaults to the entire model. Returns: int : The aggregated statistic. )_analyzecanonical_module_namesumrUvalues)rlrstats total_countr]r]r^totals zJitModelAnalysis.totalcCs|}||}|j|S)a Returns the statistics for a requested module, grouped by operator type. The operator handle determines the name associated with each operator type. Args: module_name (str) : The submodule to get data for. Defaults to the entire model. Returns: Counter(str) : The statistics for each operator. )rrrU)rlrrr]r]r^ by_operators  zJitModelAnalysis.by_operatorcC|}|jS)a Returns the statistics for all submodules, separated out by operator type for each submodule. The operator handle determines the name associated with each operator type. Returns: dict(str, Counter(str)): The statistics for each submodule and each operator. Grouped by submodule names, then by operator name. )rrUrlrr]r]r^by_module_and_operator s z'JitModelAnalysis.by_module_and_operatorcCs6|}t}|jD] \}}t|||<q |S)z Returns the statistics for all submodules, aggregated over all operators. Returns: Counter(str): statistics counter grouped by submodule names )rrrUitemsrr)rlr summed_countsreresultsr]r]r^ by_modules zJitModelAnalysis.by_modulecCs(|jdur td||}|jj|S)a Lists the number of operators that were encountered but unsupported because no operator handle is available for them. Does not include operators that are explicitly ignored. Args: module_name (str) : The submodule to list unsupported ops. Defaults to the entire model. Returns: Counter(str) : The number of occurences each unsupported operator. NzDAnalysis results should be computed before calling unsupported_ops())r RuntimeErrorrrV)rlrr]r]r^rV)s   z JitModelAnalysis.unsupported_opscCr)a Returns a set of submodules that were never called during the trace of the graph. This may be because they were unused, or because they were accessed via direct calls .forward() or with other python methods. In the latter case, statistics will not be attributed to the submodule, though the statistics will be included in the parent module. Returns: set(str) : The set of submodule names that were never called during the trace of the model. )rrWrr]r]r^uncalled_modules>s z!JitModelAnalysis.uncalled_moduleskwargscOsd|_t|ddkrtdt|ddd|dddD]\}}|||<q|D]\}}|dur:|j|q+||j|<q+|S)a@ Sets additional operator handles, or replaces existing ones. Args: args: (str, Handle) pairs of operator names and handles. kwargs: mapping from operator names to handles. If a handle is ``None``, the op will be explicitly ignored. Otherwise, handle should be a function that calculates the desirable statistic from an operator. The function must take two arguments, which are the inputs and outputs of the operator, in the form of ``list(torch._C.Value)``. The function should return a counter object with per-operator statistics. Examples :: handlers = {"aten::linear": my_handler} counter.set_op_handle("aten::matmul", None, "aten::bmm", my_handler2) .set_op_handle(**handlers) Nrz?set_op_handle should be called with pairs of names and handles!r)rlen TypeErrorziprrrbr)rlargsrrdrr]r]r^ set_op_handleNs&  zJitModelAnalysis.set_op_handlecCsi|_tt|_d|_|S)z< Clears all operator handles currently set. N)rrrSrrrlr]r]r^clear_op_handlesps z!JitModelAnalysis.clear_op_handlesrdcCs.t|ts Jd||jvr|j|Std)a Returns the canonical module name of the given ``name``, which might be different from the given ``name`` if the module is shared. This is the name that will be used as a key when statistics are output using .by_module() and .by_module_and_operator(). Args: name (str) : The name of the module to find the canonical name for. Returns: str : The canonical name of the module. zModule name must be a string.zIRequested module name is not among the descendants of the analyzed model.)rrtrKeyErrorrkr]r]r^rys   z&JitModelAnalysis.canonical_module_name new_model new_inputscCsV|dur|jn|}|dur|jn|}t||djdi|j|j|j |j S)a Returns a copy of the :class:`JitModelAnalysis` object, keeping all settings, but on a new model or new inputs. Args: new_model (nn.Module or None) : a new model for the new JitModelAnalysis. If None, uses the original model. new_inputs (typing.Tuple[object, ...] or None) : new inputs for the new JitModelAnalysis. If None, uses the original inputs. Returns: JitModelAnalysis : the new model analysis object N)r_rhr]) rrrrrr_enable_warn_unsupported_opsr_enable_warn_uncalled_modsr _warn_trace)rlrrr_rhr]r]r^rs zJitModelAnalysis.copyrlmodecCs"|dvr td|d||_|S)a Sets which warnings to print when tracing the graph to calculate statistics. There are three modes. Defaults to 'no_tracer_warning'. Allowed values are: * 'all' : keeps all warnings raised while tracing * 'no_tracer_warning' : suppress torch.jit.TracerWarning only * 'none' : suppress all warnings raised while tracing Args: mode (str) : warning mode in one of the above values. )allrnonez!Unrecognized tracer warning mode .) ValueErrorrrlrr]r]r^rs z JitModelAnalysis.tracer_warningscCs |dvr td|||_|S)a Sets how to determine the ancestor modules of an operator. Must be one of "owner" or "caller". * "caller": an operator belongs to all modules that is currently executing `forward()` at the time the operator is called. * "owner": an operator belongs to the last module that's executing `forward()` at the time the operator is called, plus this module's recursive parents. If an module has multiple parents (e.g. a shared module), only one will be picked. For most cases, a module only calls submodules it owns, so both options would work identically. In certain edge cases, this option will affect the hierarchy of results, but won't affect the total count. )rcallerzUnrecognized ancestor mode: )r_ancestor_moderr]r]r^rszJitModelAnalysis.ancestor_modeenabledcC ||_|S)aC Sets if warnings for unsupported operators are shown. Defaults to True. Counts of unsupported operators may be obtained from :meth:`unsupported_ops` regardless of this setting. Args: enabled (bool) : Set to 'True' to show unsupported operator warnings. )rrlrr]r]r^rs z)JitModelAnalysis.unsupported_ops_warningscCr)a Sets if warnings from uncalled submodules are shown. Defaults to true. A submodule is considered "uncalled" if it is never called during tracing. This may be because it is actually unused, or because it is accessed via calls to ``.forward()`` or other methods of the module. The set of uncalled modules may be obtained from :meth:`uncalled_modules` regardless of this setting. Args: enabled (bool) : Set to 'True' to show warnings. )rrr]r]r^rs z*JitModelAnalysis.uncalled_modules_warningsopscCs<|jsdStt}|D] \}}|d||qdS)Nz.Unsupported operator {} encountered {} time(s))rlogging getLoggerrXrwarningformat)rlrloggeropfreqr]r]r^_warn_unsupported_opss  z&JitModelAnalysis._warn_unsupported_opsrWcsRjsdSfdd|D}t|dkrdStt}|ddt|dS)Ncsh|] }|r|qSr]) _has_forward).0xrr]r^ sz7JitModelAnalysis._warn_uncalled_mods..raBThe following submodules of the model were never called during the trace of the graph. They may be unused, or they were accessed by direct calls to .forward() or via other python methods. In the latter case they will have zeros for statistics, though their statistics will still contribute to their parent calling module. z, )rrrrrXrjoinsorted)rlrWrr]rr^_warn_uncalled_modss   z$JitModelAnalysis._warn_uncalled_modscCs6i}t|D]\}}||vr|||<||||<q|Srjr)rlr_rirdrgr]r]r^r s zJitModelAnalysis._get_aliasescCsH||d}dh}tt|dD]}|d|d|q|S)z Get all ancestors of the given module, defined by ownership. If the given module has multiple owners, use its canonical name. rrrN)rsplitrangerrbr)rlrpartsreskr]r]r^_get_all_ancestorss z#JitModelAnalysis._get_all_ancestorsrTcCs<|j}|dur |St(|jdkrtdn |jdkr%tjdtdt|j|j |j }Wdn1s8wYi}i}t |jD]\}}|j |}t ||<t ||<qFt }|D]} | } | dkrs| d| } | d} || |jdkrt | } n || d } || | |jvr|| rqa| D] }||| d 7<qqat| t| } }|j| | |}t|trt || |i}|D]}t|tt t!j"t!j#fst$d t%|d q| D] }|||7<qqat |j |}t&|||d }||_|'|d|(||S)Nrignorer)categoryprim::PythonOpr/rrz Invalid type z? for the flop count! Please use a wider type to avoid overflow.)rUrVrWr))rwarningscatch_warningsr simplefilterfilterwarningsrrrrrrrranodeskindpyname scopeNamerupdaterrr_should_ignore_nodelistrhrxrr_simplify_op_namerintfloatnpfloat64int64rtyperTrr)rlrrrUrVrrerdall_seennoder scope_names ancestorsrhrx op_countsvrWr]r]r^rsl                 zJitModelAnalysis._analyze full_op_namecCs&|d}|dkr||ddS|S)z| Get simplified name of the op without the preceding namespace, e.g. aten::batch_norm -> batch_norm z::rrN)find)rlrpr]r]r^r_s z"JitModelAnalysis._simplify_op_namemod_namecCsT|j|}|dur dSt|}tjtjtjtjh}|D] }|j|jur'dSqdS)NFT) rgetrru ModuleList ModuleDictrvIdentityforward)rlrrg module_typeno_forward_modsrer]r]r^rjs  zJitModelAnalysis._has_forwardcCs@|}||jvr dS|ds|drdS|drdSdS)NTrzprim::CallFunctionFzprim::)rr startswith)rlrrr]r]r^r~s  z$JitModelAnalysis._should_ignore_node)r)r`r)NN)r`rT)*rXrYrZr[rurvrrr rmrtrrtypingrrrrrrVr rr rrrrrrrrboolrrrrrrrrrrr]r]r]r^rsN  "    & C )+rr r collectionsrr dataclassesrnumbersrrrrr r r r r rnumpyrrntorch.nnrufvcore.common.checkpointrr torch.jitrr jit_handlesrrrSrtr\rTrvrfroGraphrrr]r]r]r^s<     ,     >"   F