o ˳iF@sh dZddlZddlZddlmZddlmZmZmZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZdd lmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7dd l8m9Z9m:Z:e d Z;e dZe dZ?dedefddZ@e dedeAdeBeCBeABejDBeEBe BfddZFeFGee@dedeAdeBeCBeABejDBfddZHeFGeeFGeeFGeeFGe#eFGe eFGeeFGe"eFGe!e@dedeAdeBfddZHeFGeeFGee@dedeAdeCfddZHeFGee@dedeAdeAfddZHeFGe%e@de%deAdejDfddZHeFGeeFGee@dedeAdeEfd dZHeFGee@dedeAde fd!dZHeFGe$e@d"e$deAddfd#dZHe dedeIeEBe BeBeBeCBeBBeABeBejDBdeEfd$d%ZJeJGeded&eIdeEfd'dZHeJGeded&eBdeEfd(dZHeJGeded&eBdeEfd)dZHeJGe eJGe"ded&eeBBdeEfd*dZHeJGeeJGe!ded&eeBBdeEfd+dZHeJGeded&eeBBdeEfd,dZHeJGe#de#d&eeBBdeEfd-dZHeJGeded&eCdeEfd.dZHeJGeded&eCdeEfd/dZHeJGeded&eAdeEfd0dZHeJGe%de%d&ejDeEBdeEfd1dZHeJGeeJGeded&eEdeEfd2dZHeJGeded&e deEfd3dZHe ded4eEdefd5d6ZKeKGeded4eEdeIfd7dZHeKGeeKGeded4eEdeBfd8dZHeKGeeKGe#eKGe eKGe"eKGeeKGe!ded4eEdeBfd9dZHeKGeded4eEdeCfd:dZHeKGeded4eEdeCfd;dZHeKGeded4eEdeAfdeEde fd?dZHeKGe$d"e$d>eEddfd@dZHe dedAedefdBdCZLeLGededAeIdeIfdDdZHeLGeeLGedeeBdAeBdeBfdEdZHeLGededAeeBBdeAfdFdZHeLGe#de#dAeBeBdeAfdGdZHeLGe dedAeBeBdeAfdHdZHeLGe"de"dAeBeBdeAfdIdZHeLGeeLGedeeBdAeCdeCfdJdZHeLGededAeAdeAfdKdZHeLGedLed4eEdeAfdMdZHeLGeded4eEdeAfdNdZHeLGededAe deAfdOdZHeLGe%de%dAejDdeAfdPdZHe dedAedefdQdRZMeMGededAeIdeIfdSdZHeMGeeMGedeeBdAeBdeBfdTdZHeMGededAeAeBBeBdefdUdZHeMGe#de#dAeAeBBeBdefdVdZHeMGe dedAeAeBBeBdefdWdZHeMGe"de"dAeAeBBeBdefdXdZHeMGeeMGedeeBdAeCdeCfdYdZHeMGededAeAdeAfdZdZHeMGedLedAeAeEBdeEfd[dZHeMGededAeEeABdeEfd\dZHeMGededAeAde fd]dZHeMGe%de%dAeAeEBejDBdejDfd^dZHdS)_aUtility module for various conversions around PrimitiveType implementations. This module enables: - Converting partition strings to built-in python objects. - Converting a value to a byte buffer. - Converting a byte buffer to a value. - Converting a json-single field serialized field Note: Conversion logic varies based on the PrimitiveType implementation. Therefore conversion functions are defined here as generic functions using the @singledispatch decorator. For each PrimitiveType implementation, a concrete function is registered for each generic conversion function. For PrimitiveType implementations that share the same conversion logic, registrations can be stacked. N)Callable)datedatetimetimeDecimal)singledispatch)Struct)Any)UTF8L) BinaryType BooleanTypeDateType DecimalType DoubleType FixedType FloatType IntegerTypeLongType PrimitiveType StringTypeTimestampNanoType TimestampTypeTimestamptzNanoTypeTimestamptzTypeTimeType UnknownTypeUUIDType strtobool)date_str_to_days date_to_daysdatetime_to_microsdatetime_to_nanos days_to_datemicros_to_timemicros_to_timestampmicros_to_timestamptztime_str_to_microstime_to_microstimestamp_to_microstimestamptz_to_micros to_human_day to_human_timeto_human_timestampto_human_timestamptz)decimal_to_bytesunscaled_to_decimalz.wrapper)rstrr )r2r;r7r9r: handle_none\sr=r4r5cCstd|d|)zConvert a partition string to a python built-in. Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. value_str (str): A string representation of a partition value. zCannot convert 'z' to unsupported type:  TypeErrorr8r7r7r:partition_to_pymr@cCt|SN)rr8r7r7r:_xrDcCs6t|\}}}|dkrtd|dtt|S)zkConvert a string to an integer value. Raises: ValueError: If the scale/exponent is not 0. rzHCannot convert partition value, value cannot have fractional digits for z partition)ras_tuple ValueErrorintfloat)r4r5rDexponentr7r7r:rD~s cCrBrC)rIrDr5r7r7r:rDscC|SrCr7rKr7r7r:rDcC t|SrC)uuidUUIDrKr7r7r:rDs cCs t|tSrC)bytesr rKr7r7r:rDs cCrBrCrrKr7r7r:rDrEtype_cCdSrCr7)rRrDr7r7r:rDrMcCstd|)a2Convert a built-in python value to bytes. This conversion follows the serialization scheme for storing single values as individual binary values defined in the Iceberg specification that can be found at https://iceberg.apache.org/spec/#appendix-d-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. _: The value to convert to bytes (The type of this value depends on which dispatched function is used--check dispatchable functions for type hints). zscale does not match r>)r4rDr7r7r:to_bytessrTvaluecCst|rdSdS)Nr) _BOOL_STRUCTpackrDrUr7r7r:rDscCrNrC) _INT_STRUCTrXrYr7r7r:rD cCrNrC) _LONG_STRUCTrXrYr7r7r:rDr[cCt|tr t|}t|SrC) isinstancerr"r\rXrYr7r7r:rD  cCr]rC)r^rr#r\rXrYr7r7r:rDr_cCr]rC)r^rr!rZrXrYr7r7r:rD  cCr]rC)r^rr)r\rXrYr7r7r:rDr`cCrN)zConvert a float value into bytes. Note: float in python is implemented using a double in C. Therefore this involves a conversion of a 32-bit (single precision) float to a 64-bit (double precision) float which introduces some imprecision. ) _FLOAT_STRUCTrXrYr7r7r:rDs cCrNrC)_DOUBLE_STRUCTrXrYr7r7r:rDr[cCs |tSrC)encoder rYr7r7r:rDr[cCst|tr|S|jSrC)r^rQrYr7r7r:rD s cCrLrCr7rYr7r7r:rDrMcCsf|\}}}tt|}||jkrtd|d|t||jkr/td|dt|t|S)aConvert a Decimal value to bytes given a DecimalType instance with defined precision and scale. Args: primitive_type (DecimalType): A DecimalType instance with precision and scale. value (Decimal): A Decimal instance. Raises: ValueError: If either the precision or scale of `value` does not match that defined in the DecimalType instance. Returns: bytes: The byte representation of `value`. z;Cannot serialize value, scale of value does not match type z: zMCannot serialize value, precision of value is greater than precision of type )rFabsrHscalerGlen precisionr0)r4rUrDdigitsrJr7r7r:rDs  bcCstd|d|)zConvert bytes to a built-in python value. Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. b (bytes): The bytes to convert. Cannot deserialize bytes, type  not supported: r>)r4rir7r7r: from_bytes4rArlcCst|ddkSNr)rWunpackrDrir7r7r:rD?scCt|dSrm)rZrnror7r7r:rDDscCrprm)r\rnror7r7r:rDJscCrprm)rarnror7r7r:rDTcCrprm)rbrnror7r7r:rDYrqcCst|tSrC)rQdecoder ror7r7r:rD^rqcCrLrCr7ror7r7r:rDcbufcCstj|ddd}t||jS)NbigT)signed)rHrlr1re)r4rtunscaledr7r7r:rDjs cCrSrCr7)rRrtr7r7r:rDpsvalcCstd|d|)a5Convert built-in python values into JSON value types. https://iceberg.apache.org/spec/#json-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. val (Any): The arbitrary built-in value to convert into the right form rjrkr>r4rxr7r7r:to_jsonus rzcCrL)z4Python bool automatically converts into a JSON bool.r7rDrxr7r7r:rDrMcCrL)z0Python int automatically converts to a JSON int.r7r{r7r7r:rDrscCt|tr t|}t|SzJSON date is string encoded.)r^rr!r,r{r7r7r:rD cCr|)zHPython time or microseconds since epoch serializes into an ISO8601 time.)r^rr)r-r{r7r7r:rDr~cCr|)zdPython datetime (without timezone) or microseconds since epoch serializes into an ISO8601 timestamp.)r^rr"r.r{r7r7r:rDs cCr|)zaPython datetime (with timezone) or microseconds since epoch serializes into an ISO8601 timestamp.)r^rr"r/r{r7r7r:rDr~cCrL)z!Float serializes into JSON float.r7r{r7r7r:rDrscCrL)z*Python string serializes into JSON string.r7r{r7r7r:rDrMtcCs>t|t|krtdt|dt|t|dtS)8Python bytes serializes into hexadecimal encoded string.FixedType has length %, which is different from the value: hex)rfrGcodecsrcrrr )rrir7r7r:rDscCst|dtS)rr)rrcrrr ror7r7r:rDscCrB)afPython decimal serializes into string. Stores the string representation of the decimal value, specifically, for values with a positive scale, the number of digits to the right of the decimal point is used to indicate scale, for values with a negative scale, the scientific notation is used and the exponent must equal the negated scale. r<r{r7r7r:rDs cCrB)zSerialize into a JSON string.rr{r7r7r:rDrEcCstd|dt|)a1Convert JSON value types into built-in python values. https://iceberg.apache.org/spec/#json-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. val (Any): The arbitrary JSON value to convert into the right form rjrk)r?r<ryr7r7r: from_jsons rcCrL)z4JSON bool automatically converts into a Python bool.r7r{r7r7r:rDrMcCrL)z0JSON int automatically converts to a Python int.r7r{r7r7r:rDrscC(t|tr t|}t|trt|S|Sr})r^r<r rHr$r{r7r7r:rD  cCr)z%JSON ISO8601 string into Python time.)r^r<r(rHr%r{r7r7r:rDrcCrz)JSON ISO8601 string into Python datetime.)r^r<r*rHr&r{r7r7r:rDrcCrr)r^r<r+rHr'r{r7r7r:rDrcCrL)z,JSON float deserializes into a Python float.r7r{r7r7r:rD$rscCrL)z,JSON string serializes into a Python string.r7r{r7r7r:rD+rMcCsLt|trt|td}t|t|kr$tdt|dt||S)+JSON hexadecimal encoded string into bytes.rrr)r^r<rrrrcr rfrG)rrxr7r7r:rD1s cCs t|trt|tdS|S)rr)r^r<rrrrcr r{r7r7r:rD=s cCrB)z*Convert JSON string into a Python Decimal.rr{r7r7r:rDFrEcCs.t|tr t|St|trtj|dS|S)z%Convert JSON string into Python UUID.)rQ)r^r<rOrPrQr{r7r7r:rDLs    )N__doc__rrOcollections.abcrrrrdecimalr functoolsrstructr typingr pyiceberg.typedefr r pyiceberg.typesr rrrrrrrrrrrrrrrrrrpyiceberg.utils.datetimer r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/pyiceberg.utils.decimalr0r1rWrZr\rarbr=r<rHrIrPrQr@registerrDboolrTrlrzrr7r7r7r:s^     TH. (( (   "         (