o ƠiF@sdddlmZddlmZddlmZmZmZmZm Z m Z ddl Z ddl m Z mZGdddeZdS)) annotations)Enum)AnyDictListOptionalSequenceTupleN)dtype memory_formatc@s(eZdZUdZGdddeZdZded<dZded<e j Z d ed <d Z d ed <e j Zd ed<dZded<dZded<eeZded<dZded<dZded<d Zd ed<d@d d!ZdAd"d#ZdAd$d%ZedBd(d)ZedCd+d,ZedDd0d1Ze dEdFd5d6Ze dEdGd:d;Z dHdId>d?ZdS)JInputa Defines an input to a module in terms of expected shape, data type and tensor format. Attributes: shape_mode (torch_tensorrt.Input._ShapeMode): Is input statically or dynamically shaped shape (Tuple or Dict): Either a single Tuple or a dict of tuples defining the input shape. Static shaped inputs will have a single tuple. Dynamic inputs will have a dict of the form .. code-block:: py {"min_shape": Tuple, "opt_shape": Tuple, "max_shape": Tuple} dtype (torch_tensorrt.dtype): The expected data type of the input tensor (default: torch_tensorrt.dtype.float32) format (torch_tensorrt.TensorFormat): The expected format of the input tensor (default: torch_tensorrt.TensorFormat.NCHW) c@seZdZdZdZdS)zInput._ShapeModerN)__name__ __module__ __qualname__STATICDYNAMICrrI/home/ubuntu/.local/lib/python3.10/site-packages/torch_tensorrt/_Input.py _ShapeModesrNzOptional[_ShapeMode] shape_modez6Optional[Tuple[int, ...] | Dict[str, Tuple[int, ...]]]shaper Fbool_explicit_set_dtyper formatg@float DOMAIN_OFFSETglow_tensor_domain_inclhigh_tensor_domain_excl torch.Tensor torch_tensorstrnameis_shape_tensorargsrkwargsreturnNonecsPdvr=dvr=tddkrtdddksJd=dddd<dddd<dddd <t|dkrrt|dsVtd tt|dtfd d d Dretdt|d|_ tj j |_ nt|dkrdvrt fdd d Dstddvrt fdd d Drtddvrtdstd ttdtd|_ tj j |_ nctdstd ttddtdstd ttddtd std ttd dtdtdtd d |_ tj j|_ n tdt|ddvr3td|_|jtjkr>d|_nd|_dvrKd|_dvrXtd|_dvrbd}nd}t||_dvrud|_n%|jrtjddd |_n|j tj jkr|d|_n||_d!vrd!|_dSdS)"aF __init__ Method for torch_tensorrt.Input Input accepts one of a few construction patterns Args: shape (Tuple or List, optional): Static shape of input tensor Keyword Arguments: shape (Tuple or List, optional): Static shape of input tensor min_shape (Tuple or List, optional): Min size of input tensor's shape range Note: All three of min_shape, opt_shape, max_shape must be provided, there must be no positional arguments, shape must not be defined and implicitly this sets Input's shape_mode to DYNAMIC opt_shape (Tuple or List, optional): Opt size of input tensor's shape range Note: All three of min_shape, opt_shape, max_shape must be provided, there must be no positional arguments, shape must not be defined and implicitly this sets Input's shape_mode to DYNAMIC max_shape (Tuple or List, optional): Max size of input tensor's shape range Note: All three of min_shape, opt_shape, max_shape must be provided, there must be no positional arguments, shape must not be defined and implicitly this sets Input's shape_mode to DYNAMIC dtype (torch.dtype or torch_tensorrt.dtype): Expected data type for input tensor (default: torch_tensorrt.dtype.float32) format (torch.memory_format or torch_tensorrt.TensorFormat): The expected format of the input tensor (default: torch_tensorrt.TensorFormat.NCHW) tensor_domain (Tuple(float, float), optional): The domain of allowed values for the tensor, as interval notation: [tensor_domain[0], tensor_domain[1]). Note: Entering "None" (or not specifying) will set the bound to [0, 2) torch_tensor (torch.Tensor): Holds a corresponding torch tensor with this Input. name (str, optional): Name of this input in the input nn.Module's forward function. Used to specify dynamic shapes for the corresponding input in dynamo tracer. Examples: - Input([1,3,32,32], dtype=torch.float32, format=torch.channel_last) - Input(shape=(1,3,32,32), dtype=torch_tensorrt.dtype.int32, format=torch_tensorrt.TensorFormat.NCHW) - Input(min_shape=(1,3,32,32), opt_shape=[2,3,32,32], max_shape=(3,3,32,32)) #Implicitly dtype=torch_tensorrt.dtype.float32, format=torch_tensorrt.TensorFormat.NCHW r shape_rangesr r min_shape opt_shape max_shapezbInput shape specifications for inputs are required to be a List, tuple or torch.Size, found type: c3|]}|vVqdSNr.0kr&rr `z!Input.__init__..r+r,r.zFound that both shape (as a positional argument), and one or more of min_shape, opt_shape, max_shape were specified class Input expects that only either shape or all three of min_shape, opt_shape, max_shape are definedc3r/r0rr1r4rrr5ir6zwMissing required arguments for class Input Either shape or all three of min_shape, opt_shape, max_shape must be definedc3r/r0rr1r4rrr5ns zFound that both shape, and one or more of min_shape, opt_shape, max_shape were specified class Input expects that only either shape or all three of min_shape, opt_shape, max_shape are definedz for min_shapez for opt_shapez for max_shapezEUnexpected number of positional arguments for class Input Found zA arguments, expected either zero or a single positional argumentsr TFr$r tensor_domainNr r r#)lenr _supported_input_size_type TypeErrorr"typeany ValueErrortuplerrrrallrr _fromunknownrr$r r_parse_tensor_domainr8r torchtensorexample_tensorr#)selfr%r&domainrr4r__init__4s(                    zInput.__init__c Cs|jtjjkr!d|jt|jt|jt|jdt|jdS|jtjj kr[t |jt rRd|jd|jd|jdt|jt|jt|jdt|jdSt d|jd t d ) Nz5Input(shape={}, dtype={}, format={}, domain=[{}, {}))rr zUInput(min_shape={}, opt_shape={}, max_shape={}, dtype={}, format={}, domain=[{}, {}))r+r,r.IInput shape is dynamic but shapes are not provided as dictionary (found: )zUnknown input shape mode) rr rrrrr"r r8r isinstancedict RuntimeErrorrHrrr__str__s.      z Input.__str__cCs|Sr0)rQrPrrr__repr__szInput.__repr__abcCs|j|jkrdS|jtjjkrTt|jtsJt|jtsJ|jd|jdk|jd|jdk|jd|jdk|j|jk|j|jk|j |j k|j |j kg}t |S|j|jk|j|jk|j|jk|j |j k|j |j kg}t |S)NFr+r,r.) rr rrrMrrNr rrrrA)rSrTchecksrrrequivalent_specs*          zInput.equivalent_spec input_sizecCs0t|tjrdSt|trdSt|trdSdS)NTF)rMrESizer@list)rWrrrr;s   z Input._supported_input_size_typerIOptional[Tuple[float, float]]Tuple[float, float]cCs|dur tjtjf}|St|dkrQ|\}}t|ttf}t|ttf}|s-td||s6td|||krGtdd|d|dt|t|f}|Std t|d |) a Produce a tuple of integers which specifies a tensor domain in the interval format: [lo, hi) Args: domain (Tuple[int, int]): A tuple of integers (or NoneTypes) to verify Returns: A tuple of two int32_t-valid integers Nr-z4Expected value for tensor domain low specifier, got z5Expected value for tensor domain high specifier, got z@Expected provided integer range to have low tensor domain value z/< high tensor domain value, got invalid range [z, rLz"Expected 2 values for domain, got z: )r rrr:rMintrr?)rI result_domain domain_lo domain_hi valid_type_lo valid_type_hirrrrDs8  zInput._parse_tensor_domaintdisable_memory_format_check'Input'cCsZ|s|jtjds|jtjdstd|s|jtjdr tjntj}||j|j||dS)a7 Produce a Input which contains the information of the given PyTorch tensor. Args: tensor (torch.Tensor): A PyTorch tensor. disable_memory_format_check (bool): Whether to validate the memory formats of input tensors Returns: A Input object. )r z`Tensor does not have a supported memory format, supported formats are contiguous or channel_last)rr rr ) is_contiguousrEcontiguous_format channels_lastr?rr )clsrbrcfrmtrrr from_tensor6s    zInput.from_tensortsSequence[torch.Tensor] List['Input']cs&t|ttfs Jfdd|DS)ab Produce a list of Inputs which contain the information of all the given PyTorch tensors. Args: tensors (Iterable[torch.Tensor]): A list of PyTorch tensors. disable_memory_format_check (bool): Whether to validate the memory formats of input tensors Returns: A list of Inputs. csg|] }j|dqS))rc)rj)r2rbrhrcrr gs z&Input.from_tensors..)rMrYr@)rhrkrcrrnr from_tensorsVs zInput.from_tensorsoptimization_profile_field Optional[str]cs|jtjjkr0durtdt|jtr&t |jj |j j tj dddSt d|jddurqzt fdd d DsBJWn tyNtd wt|jtrht |jj |j j tj dddSt d |jdtd )a Get an example tensor of the shape specified by the Input object Args: optimization_profile_field (Optional(str)): Name of the field to use for shape in the case the Input is dynamically shaped Returns: A PyTorch Tensor Nz>Specified a optimization profile field but the input is staticT) use_defaultr9zGInput shape is dynamic but shapes are not provided as sequence (found: rLc3s|]}|kVqdSr0r)r2 field_namerqrrr5s  z'Input.example_tensor..r7zCInvalid field name, expected one of min_shape, opt_shape, max_shaperKzhRequested an example tensor from a dynamic shaped input but did not specific which profile field to use.)rr rrr?rMrr@rErandtor rOr>AssertionErrorrN)rHrqrrurrGlsB        zInput.example_tensor)r%rr&rr'r()r'r")rSr rTr r'r)rWrr'r)rIrZr'r[)F)rbrrcrr'rd)rkrlrcrr'rmr0)rqrrr'r) rrr__doc__rrr__annotations__rr rCrr linearrrrrr r#r$rJrQrR staticmethodrVr;rD classmethodrjrprGrrrrr sH               /r ) __future__renumrtypingrrrrrr rEtorch_tensorrt._enumsr r objectr rrrrs