o wÖiŸã@s:ddlZddlmZddlTddlmZGdd„dƒZdS)éN)Ú service_pb2)Ú*é)Ú raise_errorc@sZeZdZdZdd„Zdd„Zdd„Zdd „Zd d „Zd d „Z ddd„Z dd„Z dd„Z dS)Ú InferInputaSAn object of InferInput class is used to describe input tensor for an inference request. Parameters ---------- name : str The name of input whose data will be described by this object shape : list The shape of the associated input. datatype : str The datatype of the associated input. cCsBt ¡ ¡|_||j_|j d¡|jj |¡||j_d|_ dS)NÚshape) rÚModelInferRequestÚInferInputTensorÚ_inputÚnameÚ ClearFieldrÚextendÚdatatypeÚ _raw_content)Úselfr rr©rú[/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/tritonclient/grpc/_infer_input.pyÚ__init__3s   zInferInput.__init__cCó|jjS)z†Get the name of input associated with this object. Returns ------- str The name of input )r r ©rrrrr ;ózInferInput.namecCr)zŽGet the datatype of input associated with this object. Returns ------- str The datatype of input )r rrrrrrErzInferInput.datatypecCr)z‰Get the shape of input associated with this object. Returns ------- list The shape of input )r rrrrrrOrzInferInput.shapecCs|j d¡|jj |¡|S)zÝSet the shape of input. Parameters ---------- shape : list The shape of the associated input. Returns ------- InferInput The updated input r)r r rr )rrrrrÚ set_shapeYs zInferInput.set_shapecCs¤t|tjfƒs tdƒ|jjdkr(|jt|jjƒkr'td |jt|jjƒ¡ƒnt |jƒ}|jj|kr=td ||jj¡ƒd}t |jj ƒt |j ƒkrLd}t t |jj ƒƒD]}|jj ||j |krcd}qT|s}td t |j ƒdd …t |jj ƒdd …¡ƒ|jj d d ¡|jj d d ¡|jj d d ¡|jjdkr°t|ƒ}|jdkr«| ¡|_|Sd|_|S|jjdkrËt|ƒ}|jdkrÆ| ¡|_|Sd|_|S| ¡|_|S)aªSet the tensor data from the specified numpy array for input associated with this object. Parameters ---------- input_tensor : numpy array The tensor data in numpy array format Returns ------- InferInput The updated input Raises ------ InferenceServerException If failed to set data for the tensor. z"input_tensor must be a numpy arrayÚBF16zFgot unexpected datatype {} from numpy array, expected {} for BF16 typez8got unexpected datatype {} from numpy array, expected {}TFz4got unexpected numpy array shape [{}], expected [{}]réÿÿÿÿÚshared_memory_regionNÚshared_memory_byte_sizeÚshared_memory_offsetÚBYTESró)Ú isinstanceÚnpÚndarrayrr rÚdtypeÚtriton_to_np_dtypeÚformatÚnp_to_triton_dtypeÚlenrÚrangeÚstrÚ parametersÚpopÚserialize_byte_tensorÚsizeÚitemrÚserialize_bf16_tensorÚtobytes)rÚ input_tensorr"Ú valid_shapeÚiÚserialized_outputrrrÚset_data_from_numpyjsb ÿÿ€  ÿÿ€"ÿÿ    ÷ ø  ý ÿzInferInput.set_data_from_numpyrcCsH|j d¡d|_||jjd_||jjd_|dkr"||jjd_|S)a Set the tensor data from the specified shared memory region. Parameters ---------- region_name : str The name of the shared memory region holding tensor data. byte_size : int The size of the shared memory region holding tensor data. offset : int The offset, in bytes, into the region where the data for the tensor starts. The default value is 0. Returns ------- InferInput The updated input ÚcontentsNrrrr)r r rr)Ú string_paramÚ int64_param)rÚ region_nameÚ byte_sizeÚoffsetrrrÚset_shared_memory°s zInferInput.set_shared_memorycCó|jS)z±Retrieve the underlying InferInputTensor message. Returns ------- protobuf message The underlying InferInputTensor protobuf message. )r rrrrÚ _get_tensorËózInferInput._get_tensorcCr<)z¬Retrieve the contents for this tensor in raw bytes. Returns ------- bytes The associated contents for this tensor in raw bytes. )rrrrrÚ _get_contentÔr>zInferInput._get_contentN)r) Ú__name__Ú __module__Ú __qualname__Ú__doc__rr rrrr4r;r=r?rrrrr$s    F r)Únumpyr Útritonclient.grpcrÚtritonclient.utilsÚ_utilsrrrrrrÚs