o wiu@sddlmZddlZddlmZmZddlZddlmZ ddl Z ddl m Z ddlmZmZmZmZmZmZddlmZmZerFddlmZdmd d Ze jd ed dndoddZdpddZejeddgdZ e jd ed   dqdrd!d"Z!dpd#d$Z"eje"ddgdZ#e jd ed  dsdrd%d&Z$e jd'eddtd+d,Z%dud0d1Z&dvd2d3Z'dwd4d5Z(dxd7d8Z)dyd9d:Z*dzd{d@dAZ+dnd|dEdFZ,dnd}dGdHZ-d~dLdMZ. dddQdRZ/ddVdWZ0dnddYdZZ1dd^d_Z2 dddadbZ3ddddeZ4 fdddkdlZ5dS)) annotationsN) TYPE_CHECKINGAny)MapProto OptionalProto SequenceProto TensorProtohelpersubbyte)load_external_data_for_tensoruses_external_data)Sequencefa Sequence[int]return list[complex]csfddttdDS)Ncs*g|]}t|d|ddqS))complex).0irN/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/onnx/numpy_helper.py s*z-_combine_pairs_to_complex..r)rangelenrrrr_combine_pairs_to_complexsrzpDeprecated since 1.18. Scheduled to remove in 1.20. Consider using libraries like ml_dtypes for dtype conversion)categorydata np.int16 | np.int32 | np.ndarraydimsint | Sequence[int] | None np.ndarraycCsxdd}|dur-t|jdkr!|t|gtjtjdS||tjtjS||tj|tjS)a_Converts ndarray of bf16 (as uint32) to f32 (as uint32). Args: data: A numpy array, empty dimensions are allowed if dims is None. dims: If specified, the function reshapes the results. Returns: A numpy array of float32 with the same dimension if dims is None, or reshaped to dims if specified cSs|d>S)Nrxrrr,sz%bfloat16_to_float32..Nr) rshapenparrayastypeint32viewfloat32reshape)rr!shiftrrrbfloat16_to_float32s $r1ivalintfnbooluz np.float32c CsV|std|dks|dkrt|d|r!d}|dkr tjSnd}|dkr.ttj S|dkr8ttjSt|}|d @d ?}|d@}|d@}|d >}|dkr|dkrd|}|d @dkrm|d M}|d K}|d 8}|d @dkr|d M}|d K}|d 8}||d @d>O}||d>O}n||d>O}|d|7}||d>O}t|tj}|S)Nzfn=False is not implemented.rz is not a float8.xr)NotImplementedError ValueErrorr)nanr.uint32r- r2r4r6 exponent_biasexpomantsignresfrrr_float8e4m3_to_float32_scalar4sL         rO)excludedTFcCs0|stdt|||d}|dur|S||S)aConverts ndarray of float8, e4m3 (as uint32) to f32 (as uint32). See :ref:`onnx-detail-float8` for technical details. Args: data: A numpy array, empty dimensions are allowed if dims is None. dims: If specified, the function reshapes the results. fn: No infinite values. uz: No negative zero. Returns: A numpy array of float32 with the same dimension if dims is None, or reshaped to dims if specified. z4float32_to_float8e4m3 not implemented with fn=False.r4r6N)rD_float8e4m3_to_float32r/rr!r4r6rMrrrfloat8e4m3_to_float32cs rTc CsN|r|r|dkrttjSd}n5|sB|sB|dvr ttj S|dvr*ttjS|dkr5ttj S|dkr?ttjSd}ntdt|}|d@d ?}|d @}|d@}|d >}|d kr|d krd |}|d @d kr{|dM}|dK}|d8}||d@d>O}||d>O}n||d>O}|d |7}||d>O}t|tj}|S)Nr:r$>r8>}~r<|z%fn and uz must be both False or True.rr>r?rr<rrBrA)r)r.rFinfrDrGr-rHrrr_float8e5m2_to_float32_scalarsF          r^cCs$t|||d}|dur |S||S)aConverts ndarray of float8, e5m2 (as uint32) to f32 (as uint32). See :ref:`onnx-detail-float8` for technical details. Args: data: A numpy array, empty dimensions are allowed if dims is None. dims: If specified, the function reshapes the results. fn: No infinite values. uz: No negative zero. Returns: A numpy array of float32 with the same dimension if dims is None, or reshaped to dims if specified rQN)_float8e5m2_to_float32r/rSrrrfloat8e5m2_to_float32s r`z_Deprecated since 1.18. Scheduled to remove in 1.20. Consider implementing your own unpack logicnp.int32 | np.ndarrayint | Sequence[int]signedcsfdd}t|dd}||\}}tj|j|jftjd}||ddd<||ddd<|jt|dkrB|dd}||}|S) aConverts ndarray of int4 (as packed uint8) to f32 See :ref:`onnx-detail-int4` for technical details. Args: data: A numpy array, empty dimensions are allowed if dims is None. dims: The dimensions are used to reshape the unpacked buffer signed: Whether the 4 bit integer is signed or unsigned Returns: A numpy array of float32 reshaped to dims. cs t|SN)r _unpack_single_4bitx2r%rcrrr's zunpack_int4..rrdtyperN)r) frompyfuncravelemptysizer.prodr/)rr!rc single_funcfuncres_highres_lowrMrrfr unpack_int4s  rsr&npt.NDArray[np.uint8]npt.NDArray[np.float32]cCsptt|ddd}|d@tj}|d@d?tj}t|dk||d|d|dd|d}|S)aNEvaluate the numerical value of an array of unpacked float4e2m1 values (as uint8) See :ref:`onnx-detail-int4` for technical details. Args: x: an array of uint8 elements representing a float4e2m1 (using the 4 LSB) Returns: An array of float32 elements representing the values of the float4e2m1 input. r9rirrg@g?)r)where bitwise_andr+r.)r&rLmantissaexponentvalrrrunpacked_float4e2m1_to_float32s  r|cCt||}t|S)a]Converts ndarray of float4e2m1 (as packed uint8) to f32 See :ref:`onnx-detail-float4` for technical details. Args: data: A numpy array, empty dimensions are allowed if dims is None. dims: The dimensions are used to reshape the unpacked buffer Returns: A numpy array of float32 reshaped to dims. ) _unpack_uint4r|)rr!rMrrrunpack_float4e2m1 s rcCstj|jdg|jd}|td@}|td@}|tdL}||ddd<||ddd<|jt|dkr?|dd }|j|d d |S) aConvert a packed uint4 array to unpacked uint4 array represented as uint8. Args: data: A numpy array. dims: The dimensions are used to reshape the unpacked buffer. Returns: A numpy array of int8/uint8 reshaped to dims. rrgr[r@rNrriF)refcheck)r)rlrmrhuint8rnresize)rr!result array_low array_highrrrr~s  r~npt.NDArray[np.int8]cCs"t|d?dk||dBtjS)z4Extend 4-bit signed integer to 8-bit signed integer.r>rr)r)rwr+int8r%rrr_extend_int4_sign_bits8s"rcCr})aHConvert a packed (signed) int4 array to unpacked int4 array represented as int8. The sign bit is extended to the most significant bit of the int8. Args: data: A numpy array. dims: The dimensions are used to reshape the unpacked buffer. Returns: A numpy array of int8 reshaped to dims. )r~r)rr!unpackedrrr _unpack_int4=s rtensorrbase_dirstrc Csv|dr td|jtjkrtd|j}t|}tt|}t |}|j }|jtj krHt ||}dd|D}t |||St|rQt|||dr&|j} tjdkrjt j| |d} |tjkrt j| t jd|} | tjS|tjkrt j| t jd|} | tj S|tj!krt j| t jd|} | tj"S|tj#krt j| t jd|} | tj$S|tj%krt j| t jd|} | tj&S|tj'krt j| t j(d} t)| |tj*S|tj+krt j| t j(d} t,| |tj-S|tj.krt j| t j(d} t)| |tj/St j| |d|S|tj0kr.raw_databigrg)8HasFieldrE data_typer UNDEFINED TypeErrorr tensor_dtype_to_np_dtype$tensor_dtype_to_storage_tensor_dtypetensor_dtype_to_fieldr!STRINGgetattrr)asarrayr+r/r r rsys byteorder frombufferbyteswaptobytesBFLOAT16int16r-custom_np_typesbfloat16 FLOAT8E4M3FNr float8e4m3fnFLOAT8E4M3FNUZfloat8e4m3fnuz FLOAT8E5M2 float8e5m2FLOAT8E5M2FNUZfloat8e5m2fnuzUINT4rr~uint4INT4rint4 FLOAT4E2M1 float4e2m1FLOAT16 int32_datauint16float16r, COMPLEX64 COMPLEX128r) rr tensor_dtypenp_dtypestorage_np_dtype storage_fieldr! utf8_stringsssrrrrrto_arrayOs                                     rarrname str | Nonec Cstt|tjtjfstdt|t}|j|j |r!||_ |j t ks.t |j tjrt|j |_|}|D]K}t|trL|j|dq;t|tjrr|D]}t|tre|j|dqTt|trp|j|qTq;t|tr~|j|q;tdtt||Szt|j }Wnty}z td|j |d}~ww||_||_tjdkrt||S)zConverts a numpy array to a tensor def. Args: arr: a numpy array. name: (optional) the name of the tensor. Returns: TensorProto: the converted tensor def. z2arr must be of type np.generic or np.ndarray, got rzMUnrecognized object in the object array, expect a string, or array of bytes: z$Numpy data type not understood yet: Nr) isinstancer)ndarraygenericrtyperr!extendr(rrhobject issubdtypestr_r np_dtype_to_tensor_dtyperflattenr string_dataappendencodebytesrDKeyError RuntimeErrorrrrr_convert_endian)rrr flat_arrayerrhrrr _from_arraysP            rc Cs*t|tjs t||S|j}|tjkr#|jdddkr#tj }tj }n|tj kr8|jdddkr8tj }tj }n|tj krM|jdddkrMtj}tj }nn|tjkrb|jdddkrbtj}tj }nY|tjkrw|jdddkrwtj}tj}nD|tjkr|jdddkrtj}tj}n/|tjkr|jdddkrtj}tj }n|tjkr|jddd krtj}tj }nt||S|tjtjtjfvr||}|jd d krtd |jd |ddd d@|d dd d>}t}|j !|j|r||_"|#|_$||_%|St|||}||_%|S)a Converts an array into a TensorProto including supported type defined in :mod:`onnx._custom_element_types`. Args: tensor: a numpy array. name: (optional) the name of the tensor. Returns: TensorProto: the converted tensor def. re4m3fne4m3fnuze5m2e5m2fnuzrrrrrrzIThe conversion of a tensor of INT4 or UINT4 requires an even size (shape=z+). Every byte stores two INT4 or two UINT4.Nr[r@)&rr)rrrhrrdescrrrrrrrrrrrrrrrrrrrrr+rkrmrEr(r!rrrrr) rrdttodt_tovaluebufferpbtrrr from_arrays^  $ rsequencer list[Any]cCsv|j}|tjkrdd|jDS|tjkrdd|jDS|tjkr*dd|jDS|tjkr7dd|j DSt d)zConverts a sequence def to a Python list. Args: sequence: a SequenceProto object. Returns: list: the converted list. cSg|]}t|qSrrrvrrrrdzto_list..cSrrrrrrrrfrcSrr)to_listrrrrrhrcSrr)to_dictrrrrrjrz8The element type in the input sequence is not supported.) elem_typerTENSOR tensor_values SPARSE_TENSORsparse_tensor_valuesSEQUENCEsequence_valuesMAP map_valuesr)rrrrrrYs    rlstrh int | Nonec st}|r||_|r |}n#tdkr-d}t|tr tj}nt|tr)tj}ntj}ntj}||_ tdkrHt fddDsHt d|tjkr]D] }|j t|gqO|S|tjkrrD] }|j t|gqd|S|tjkrD] }|j t|gqy|St d)a]Converts a list into a sequence def. Args: lst: a Python list name: (optional) the name of the sequence. dtype: (optional) type of element in the input list, used for specifying sequence values when converting an empty list. Returns: SequenceProto: the converted sequence def. rc3s"|] }t|tdVqdS)rN)rr)relemrrr s zfrom_list..zqThe element type in the input list is not the same for all elements and therefore is not supported as a sequence.zZThe element type in the input list is not a tensor, sequence, or map and is not supported.)rrrrdictrlistrrrallrrrrr from_listr from_dict) rrrhrr first_elemrseqmappingrrrrnsB   "   r map_protordict[Any, Any]cCs`g}|jtjkrt|j}nt|j}t|j}t|t|kr't d|j dt t ||}|S)zConverts a map def to a Python dictionary. Args: map_proto: a MapProto object. Returns: The converted dictionary. z2Length of keys and values for MapProto (map name: z) are not the same.) key_typerrr string_keyskeysrvaluesr IndexErrorrrzip)rkey_list value_list dictionaryrrrrs    rdict_cst}|r||_t|}t|dt}tjtj tj tj tj tj tjtjg}tfdd|Ds9tdt|}t|dtfdd|DsUtdt|}||_|tjkrh|j|n ||vrr|j||j||S)zConverts a Python dictionary into a map def. Args: dict_: Python dictionary name: (optional) the name of the map. Returns: MapProto: the converted map def. rc3|] }t|kVqdSrdr) result_type)rkey) raw_key_typerrrzfrom_dict..zfThe key type in the input dictionary is not the same for all keys and therefore is not valid as a map.c3rrdr)rr{)raw_value_typerrrrzjThe value type in the input dictionary is not the same for all values and therefore is not valid as a map.)rrrr)rr rrINT8INT16INT32INT64UINT8UINT16UINT32UINT64rrr rr rr rr CopyFrom)rrrr r valid_key_int_typesr  value_seqr)rrrrsB       roptionalr Any | NonecCs|j}|tjkr dS|tjkrt|jS|tjkrt|jS|tjkr(t |j S|tj kr2t |j S|tjkrs       * (  !      A @ 9  5 3