o Ơi@sddlmZddlZddlmZddlmZmZmZm Z ddl Z ddl m Z m Z ddlmZddlmZddlmZmZeeZ d#ddd d$ddZd%ddZd&ddZd'd!d"ZdS)() annotationsN) signature)AnyOptionalTupleUnion)Dimexport)Input)default_device)get_torch_inputsto_torch_device) arg_inputs kwarg_inputsmod&torch.nn.Module | torch.fx.GraphModuleinputsOptional[Tuple[Any, ...]]rrOptional[dict[Any, Any]]kwargsrreturntorch.export.ExportedProgramc Ks|dur |dur td|dur|durtd|p|}|dur"i}t|dt}t||}t||}t||}|t|t|t ||||ddd} | S)a Exports a ``torch.export.ExportedProgram`` from a ``torch.nn.Module`` or ``torch.fx.GraphModule`` specifically targeting being compiled with Torch-TensorRT Exports a ``torch.export.ExportedProgram`` from either a ``torch.nn.Module`` or torch.fx.GraphModule``. Runs specific operator decompositions geared towards compilation by Torch-TensorRT's dynamo frontend. Arguments: mod (torch.nn.Module | torch.fx.GraphModule): Source module to later be compiled by Torch-TensorRT's dynamo fronted inputs (Tuple[Any, ...]): List of specifications of input shape, dtype and memory layout for inputs to the module. This argument is required. Input Sizes can be specified as torch sizes, tuples or lists. dtypes can be specified using torch datatypes or torch_tensorrt datatypes and you can use either torch devices or the torch_tensorrt device type enum to select device type. :: input=[ torch_tensorrt.Input((1, 3, 224, 224)), # Static NCHW input shape for input #1 torch_tensorrt.Input( min_shape=(1, 224, 224, 3), opt_shape=(1, 512, 512, 3), max_shape=(1, 1024, 1024, 3), dtype=torch.int32 format=torch.channel_last ), # Dynamic input shape for input #2 torch.randn((1, 3, 224, 244)) # Use an example tensor and let torch_tensorrt infer settings ] Keyword Arguments: arg_inputs (Tuple[Any, ...]): Same as inputs. Alias for better understanding with kwarg_inputs. kwarg_inputs (dict[Any, ...]): Optional, kwarg inputs to the module forward function. device (Union(torch.device, dict)): Target device for TensorRT engines to run on :: device=torch.device("cuda:0") debug (bool): Enable debuggable engine enable_experimental_decompositions (bool): Use the full set of operator decompositions. These decompositions may not be tested but serve to make the graph easier to convert to TensorRT, potentially increasing the amount of graphs run in TensorRT. **kwargs: Any, Returns: torch.fx.GraphModule: Compiled FX Module, when run it will execute via TensorRT Nz2'arg_inputs' and 'inputs' should not both be None.z>'arg_inputs' and 'inputs' should not be used at the same time.devicestrictF)rdynamic_shapesr) AssertionErrorr getr r get_dynamic_shapes_argsupdateget_dynamic_shapes_kwargsr tuple) rrrrrrtorch_arg_inputstorch_kwarg_inputsr exp_programr$Q/home/ubuntu/.local/lib/python3.10/site-packages/torch_tensorrt/dynamo/_tracer.pytraces,-    r& Union[dict[str, Any], list[Any]]cCst|tri}|D] \}}t|||<q |St|tr!t|St|ttfr8g}|D] }|t|q,|St dt |d)Nz Unknown type .) isinstancedictitemsrr get_dynamic_shapeslistr append TypeErrortype)rdynamic_shapes_kwargkvrinputr$r$r%rZs  rtorch.nn.Moduledict[str, Any]cCsHtt|jj}i}t||dt|D] \}}t|||<q|SN)r-rforward parameterskeysziplenr,)rrargsrr4 input_namer$r$r%rms rr4r dict[Any, Any]cCst|tsiSi}|jtjjkrc|jd}|jd}|jd}t|t|kr0t|ks3JJtt|D])}||||krK||krNnnq9t|j dt |||||d||<q9|S)N min_shape opt_shape max_shape_)minmax) r)r shape_mode _ShapeModeDYNAMICshaper<rangernamestr)r4 dynamic_dimsr@rArBdimr$r$r%r,ws"    ($ r,r7) rrrrrrrrrrrr)rrrr')rr5rrrr6)r4r rr?) __future__rlogginginspectrtypingrrrrtorch torch.exportrr torch_tensorrt._Inputr torch_tensorrt.dynamo._defaultsr torch_tensorrt.dynamo.utilsr r getLogger__name__loggerr&rrr,r$r$r$r%s$      J