o Ơix/@sddlZddlZddlZddlZddlmZmZddlmZm Z m Z m Z ddl Z ddl mZddlmZddlmZeeZeGdddZeGd d d Zd ed e eefd dfddZdeded efddZde jjd e efddZdS)N) dataclassfield)AnyDictListUnion)CompilationSettings)ConverterRegistry) get_node_namec@sreZdZUdZdZeed<dZeed<e e dZ e ed<e e dZ e ed<e e dZe ed <e e dZe ed <d S) PerSubgraphDataaClass to track data on a per-subgraph level Args: subgraph_name (str): Name of the subgraph in the GraphModule subgraph_op_count (int): Number of operations in the subgraph input_shapes (Any): Shapes of input Tensors of the subgraph input_dtypes (Any): Input data types of the subgraph output_shapes (Any): Shapes of output Tensors of the subgraph output_dtypes (Any): Output data types of the subgraph subgraph_namersubgraph_op_countdefault_factory input_shapes input_dtypes output_shapes output_dtypesN)__name__ __module__ __qualname____doc__r str__annotations__rintrlistrrrrrrrX/home/ubuntu/.local/lib/python3.10/site-packages/torch_tensorrt/dynamo/_DryRunTracker.pyr s  r c@seZdZUdZdZeed<dZeed<ee dZ e ed<ee dZ e ed<ee dZ e ed<ee dZe ed <ee dZeeed <dZeed <eedZeed <eedZeeefed <ee dZeeed<dS) DryRunTrackeraJClass to track data on a graph-wide level Args: total_ops_in_graph (int): Total number of operators in graph supported_ops_in_graph (int): Number of supported operators in graph input_shapes (Any): Shapes of input Tensors of the graph input_dtypes (Any): Input data types of the graph output_shapes (Any): Shapes of output Tensors of the graph output_dtypes (Any): Output data types of the graph per_subgraph_data (List[PerSubgraphData]): Per-subgraph data, see above class tensorrt_graph_count (int): Number of TensorRT engines to be generated compilation_settings (CompilationSettings): User Compilation Settings unsupported_ops (Dict[str, int]): Set of operators not supported in TRT to_run_in_torch (List[str]): Set of nodes to run in Torch rtotal_ops_in_graphsupported_ops_in_graphrrrrrper_subgraph_datatensorrt_graph_countcompilation_settingsunsupported_opsto_run_in_torchN)rrrrr rrr!rrrrrrrr"rr r#rr$dictr%rrr&rrrrr%s    rdryrun_trackerdryrun_enabledreturnc sPd}|d7}|d|jd|jdt|jd|jdd7}|jr7dd d |jD}|d |d 7}|jrHd|j}|d |d7}|d|jd 7}t|j |j ks[J|d7}|ddt |j |j d7}t|j D]A\}}|d7}|dd|dd|jd7}|ddt |j |j d7}|dd|jd7}|ddt |j|jd7}qs|d7}|ddt |j|jd7}|j dkr^tdd|j Dtd d|j D|j tdtd!d|j D|d"7}|d#7}|d$7}|d%7}|d&d'tfd(d |j Dd)7}tkrE|d*td'tfd+d |j Dd)7}|d,d-tfd.d |j Dd)7}n|d/7}|rt|t|trtj|rtd0|d1d3St |d2}|!|Wd3d3S1swYd3Sd3St"|d3S)4zCDisplays statistics about the dryrun either to debug logs or stdout z++++++++++++++++++++++++++++++++++++++++++++++++++ Dry-Run Results for Graph ++++++++++++++++++++++++++++++++++++++++++++++++++ zThe graph consists of z Total Operators, of which z operators are supported, dz % coverage cSs&g|]\}}t|dt|qS)z: )r).0kvrrr Us&z(dryrun_stats_display..z{The following ops are currently unsupported or excluded from conversion, and are listed with their op-count in the graph: z z7The following nodes are currently set to run in Torch: zj Note: Some of the above nodes may be supported, but were not included in a TRT graph by the partitioner zCompiled with: z Graph Structure: z zInputs: z ... z z TRT Engine #z - Submodule name: z zEngine Inputs: zNumber of Operators in Engine: zEngine Outputs: z Outputs: rcs|]}|jVqdSNrr. trt_subgraphrrr  z'dryrun_stats_display..csr3r4r5r6rrrr8r9csr3r4r5r6rrrr8r9zF ------------------------- Aggregate Stats -------------------------z1 Average Number of Operators per TRT Engine: z$ Most Operators in a TRT Engine: z) ********** Recommendations **********z= - For minimal graph segmentation, select min_block_size=z which would generate cg|] }|jkrdqSr2r5r6)most_ops_in_an_enginerrr1z TRT engine(s)z= - For moderate graph segmentation, select min_block_size=cs g|] }|jtkrdqSr;)rmathceilr6)avg_ops_per_enginerrr1s zX - The current level of graph segmentation is equivalent to selecting min_block_size=z which generates cr:r;r5r6)min_ops_in_an_enginerrr1r=zE Aggregate stats not available since no TRT Engines were generated.zFile already exists at path z, not saving dryrun outputzw+N)#r r!roundr%joinitemsr&r$lenr"r#input_formatterrr enumerater rrrminsummaxr>r?print isinstancerospathexistsloggerwarningopenwritedebug)r(r)formatted_stats parsed_opsformatted_nodesitrt_subgraph_datafr)r@rAr<rdryrun_stats_displayFs           $ r[shapesdtypescs,dtdtdtffdd ||ddS)z@Format shapes and dtypes of input Tensors into a readable stringr\r]r*cst|trGtdd|DrGd}|D]}t|tr)|d|dd|dd7}q||d 7}q|d d d d t|dd d }|St|trlt|dkrltdd|Drld|d t|dd d St|ttfrt|trzdnd}t||D] \}}|||7}q|d d t|trd}|Sd}|St|trd}|D] \}}||||7}q|d d d}|St dt |d)zHelper for input formattercss|] }t|ttfVqdSr4)rLrtupler.eltrrrr8s zBinput_formatter..input_formatter_helper..z Tensor: (z(min=rz, max=r2z), z, N)@css8|]\}}t|totdd|Do|dvVqdS)css|]}t|tVqdSr4)rLrr_rrrr8szLinput_formatter..input_formatter_helper...) min_shape opt_shape max_shapeN)rLr^all)r.r/shaperrrr8s  zTensor: zList[zTuple(z], zDict{z}, zInvalid input type z5 encountered in parse_complex_tensor_structs parsing.) rLr^rirr'rErDrzip ValueErrortype)r\r]input_shape_stringr` formatted_strrjdtypekeyinput_formatter_helperrrrssL  (     z/input_formatter..input_formatter_helperNra)rr)r\r]rrrrrFs7rF graph_modulecCsNg}|jjD]}|jdvr$|jtjkr$|dt|jdt |q|S)zParses call_function and call_method nodes from a GraphModule Excludes getitem nodes Returns a string representation of the nodes ) call_function call_methodzNode: z, with layer location: ) graphnodesoptargetoperatorgetitemappendr qualified_name_or_strr )rtr&noderrrparse_non_trt_nodess   r)loggingr>r{rM dataclassesrrtypingrrrrtorchtorch_tensorrt.dynamo._settingsr3torch_tensorrt.dynamo.conversion._ConverterRegistryr 0torch_tensorrt.dynamo.conversion.converter_utilsr getLoggerrrPr rboolrr[rFfx GraphModulerrrrrs2       =