o uyi4G@s dZddlZddlZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z mZmZmZddlmZmZdd lmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'dd l(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8dd l9m:Z:m;Z;e d Ze dZ?e dZ@dedefddZAe dedeBdeeCeDeBejEeFeffddZGeGHeeAdedeBdeeCeDeBejEffddZIeGHeeGHeeGHeeGHe$eGHe!eGHe eGHe#eGHe"eAdedeBdeCfddZIeGHeeGHeeAdedeBdeDfddZIeGHeeAdedeBdeBfddZIeGHe&eAde&deBdejEfddZIeGHeeGHeeAdedeBdeFfddZIeGHeeAdedeBdefd dZIeGHe%eAd!e%deBddfd"dZIe dedeeJeFeeeeDeCeBeejEf deFfd#d$ZKeKHeded%eJdeFfd&dZIeKHeded%eCdeFfd'dZIeKHeded%eCdeFfd(dZIeKHe!eKHe#ded%eeeCfdeFfd)dZIeKHe eKHe"ded%eeeCfdeFfd*dZIeKHeded%eeeCfdeFfd+dZIeKHe$de$d%eeeCfdeFfd,dZIeKHeded%eDdeFfd-dZIeKHeded%eDdeFfd.dZIeKHeded%eBdeFfd/dZIeKHe&de&d%eejEeFfdeFfd0dZIeKHeeKHeded%eFdeFfd1dZIeKHeded%edeFfd2dZIe ded3eFdefd4d5ZLeLHeded3eFdeJfd6dZIeLHeeLHeded3eFdeCfd7dZIeLHeeLHe$eLHe!eLHe#eLHe eLHe"ded3eFdeCfd8dZIeLHeded3eFdeDfd9dZIeLHeded3eFdeDfd:dZIeLHeded3eFdeBfd;dZIeLHeeLHeeLHe&ded3eFdeFfddZIeLHe%d!e%d=eFddfd?dZIe ded@e defdAdBZMeMHeded@eJdeJfdCdZIeMHeeMHedeeefd@eCdeCfdDdZIeMHeded@eeeCfdeBfdEdZIeMHe$de$d@eeCefdeBfdFdZIeMHe!ded@eeCefdeBfdGdZIeMHe#de#d@eeCefdeBfdHdZIeMHeeMHedeeefd@eDdeDfdIdZIeMHeded@eBdeBfdJdZIeMHedKed3eFdeBfdLdZIeMHeded3eFdeBfdMdZIeMHeded@edeBfdNdZIeMHe&de&d@ejEdeBfdOdZIe ded@e defdPdQZNeNHeded@eJdeJfdRdZIeNHeeNHedeeefd@eCdeCfdSdZIeNHeded@eeBeCefdefdTdZIeNHe$de$d@eeBeCefdefdUdZIeNHe!ded@eeBeCefdefdVdZIeNHe#de#d@eeBeCefdefdWdZIeNHeeNHedeeefd@eDdeDfdXdZIeNHeded@eBdeBfdYdZIeNHedKed@eeBeFfdeFfdZdZIeNHeded@eeFeBfdeFfd[dZIeNHeded@eBdefd\dZIeNHe&de&d@eeBeFejEfdejEfd]dZIdS)^aUtility module for various conversions around PrimitiveType implementations. This module enables: - Converting partition strings to built-in python objects. - Converting a value to a byte buffer. - Converting a byte buffer to a value. - Converting a json-single field serialized field Note: Conversion logic varies based on the PrimitiveType implementation. Therefore conversion functions are defined here as generic functions using the @singledispatch decorator. For each PrimitiveType implementation, a concrete function is registered for each generic conversion function. For PrimitiveType implementations that share the same conversion logic, registrations can be stacked. N)datedatetimetimeDecimal)singledispatch)Struct)AnyCallableOptionalUnion)UTF8L) BinaryType BooleanTypeDateType DecimalType DoubleType FixedType FloatType IntegerTypeLongType PrimitiveType StringTypeTimestampNanoType TimestampTypeTimestamptzNanoTypeTimestamptzTypeTimeType UnknownTypeUUIDType strtobool)date_str_to_days date_to_daysdatetime_to_microsdatetime_to_nanos days_to_datemicros_to_timemicros_to_timestampmicros_to_timestamptztime_str_to_microstime_to_microstimestamp_to_microstimestamptz_to_micros to_human_day to_human_timeto_human_timestampto_human_timestamptz)decimal_to_bytesunscaled_to_decimalz.wrapper)rr strr )r4r=r9r;r< handle_none^sr?r6r7cCstd|d|)zConvert a partition string to a python built-in. Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. value_str (str): A string representation of a partition value. zCannot convert 'z' to unsupported type:  TypeErrorr:r9r9r<partition_to_pyorBcCt|SN)r!r:r9r9r<_zrFcCs6t|\}}}|dkrtd|dtt|S)zkConvert a string to an integer value. Raises: ValueError: If the scale/exponent is not 0. rzHCannot convert partition value, value cannot have fractional digits for z partition)ras_tuple ValueErrorintfloat)r6r7rFexponentr9r9r<rFs cCrDrE)rKrFr7r9r9r<rFscC|SrEr9rMr9r9r<rFcC t|SrE)uuidUUIDrMr9r9r<rFs cCs t|tSrE)bytesr rMr9r9r<rFs cCrDrErrMr9r9r<rFrGtype_cCdSrEr9)rTrFr9r9r<rFrOcCstd|)a.Convert a built-in python value to bytes. This conversion follows the serialization scheme for storing single values as individual binary values defined in the Iceberg specification that can be found at https://iceberg.apache.org/spec/#appendix-d-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. _: The value to convert to bytes (The type of this value depends on which dispatched function is used--check dispatchable functions for type hints). zscale does not match r@)r6rFr9r9r<to_bytessrVvaluecCst|rdSdS)Nr) _BOOL_STRUCTpackrFrWr9r9r<rFscCrPrE) _INT_STRUCTrZr[r9r9r<rF cCrPrE) _LONG_STRUCTrZr[r9r9r<rFr]cCt|tr t|}t|SrE) isinstancerr$r^rZr[r9r9r<rF  cCr_rE)r`rr%r^rZr[r9r9r<rFracCr_rE)r`rr#r\rZr[r9r9r<rF  cCr_rE)r`rr+r^rZr[r9r9r<rFrbcCrP)zConvert a float value into bytes. Note: float in python is implemented using a double in C. Therefore this involves a conversion of a 32-bit (single precision) float to a 64-bit (double precision) float which introduces some imprecision. ) _FLOAT_STRUCTrZr[r9r9r<rFs cCrPrE)_DOUBLE_STRUCTrZr[r9r9r<rFr]cCs |tSrE)encoder r[r9r9r<rFr]cCst|tr|S|jSrE)r`rSr[r9r9r<rF s cCrNrEr9r[r9r9r<rFrOcCsf|\}}}tt|}||jkrtd|d|t||jkr/td|dt|t|S)aConvert a Decimal value to bytes given a DecimalType instance with defined precision and scale. Args: primitive_type (DecimalType): A DecimalType instance with precision and scale. value (Decimal): A Decimal instance. Raises: ValueError: If either the precision or scale of `value` does not match that defined in the DecimalType instance. Returns: bytes: The byte representation of `value`. z;Cannot serialize value, scale of value does not match type z: zMCannot serialize value, precision of value is greater than precision of type )rHabsrJscalerIlen precisionr2)r6rWrFdigitsrLr9r9r<rFs  bcCstd|d|)zConvert bytes to a built-in python value. Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. b (bytes): The bytes to convert. Cannot deserialize bytes, type  not supported: r@)r6rkr9r9r< from_bytes5rCrncCst|ddkSNr)rYunpackrFrkr9r9r<rF@scCt|dSro)r\rprqr9r9r<rFEscCrrro)r^rprqr9r9r<rFKscCrrro)rcrprqr9r9r<rFUcCrrro)rdrprqr9r9r<rFZrscCst|tSrE)rSdecoder rqr9r9r<rF_rscCrNrEr9rqr9r9r<rFdbufcCstj|ddd}t||jS)NbigT)signed)rJrnr3rg)r6rvunscaledr9r9r<rFks cCrUrEr9)rTrvr9r9r<rFqsvalcCstd|d|)a5Convert built-in python values into JSON value types. https://iceberg.apache.org/spec/#json-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. val (Any): The arbitrary built-in value to convert into the right form rlrmr@r6rzr9r9r<to_jsonvs r|cCrN)z4Python bool automatically converts into a JSON bool.r9rFrzr9r9r<rFrOcCrN)z0Python int automatically converts to a JSON int.r9r}r9r9r<rFrucCt|tr t|}t|SzJSON date is string encoded.)r`rr#r.r}r9r9r<rF cCr~)zHPython time or microseconds since epoch serializes into an ISO8601 time.)r`rr+r/r}r9r9r<rFrcCr~)zdPython datetime (without timezone) or microseconds since epoch serializes into an ISO8601 timestamp.)r`rr$r0r}r9r9r<rFs cCr~)zaPython datetime (with timezone) or microseconds since epoch serializes into an ISO8601 timestamp.)r`rr$r1r}r9r9r<rFrcCrN)z!Float serializes into JSON float.r9r}r9r9r<rFrucCrN)z*Python string serializes into JSON string.r9r}r9r9r<rFrOtcCs>t|t|krtdt|dt|t|dtS)8Python bytes serializes into hexadecimal encoded string.FixedType has length %, which is different from the value: hex)rhrIcodecsrertr )rrkr9r9r<rFscCst|dtS)rr)rrertr rqr9r9r<rFscCrD)afPython decimal serializes into string. Stores the string representation of the decimal value, specifically, for values with a positive scale, the number of digits to the right of the decimal point is used to indicate scale, for values with a negative scale, the scientific notation is used and the exponent must equal the negated scale. r>r}r9r9r<rFs cCrD)zSerialize into a JSON string.rr}r9r9r<rFrGcCstd|dt|)a1Convert JSON value types into built-in python values. https://iceberg.apache.org/spec/#json-single-value-serialization Args: primitive_type (PrimitiveType): An implementation of the PrimitiveType base class. val (Any): The arbitrary JSON value to convert into the right form rlrm)rAr>r{r9r9r< from_jsons rcCrN)z4JSON bool automatically converts into a Python bool.r9r}r9r9r<rFrOcCrN)z0JSON int automatically converts to a Python int.r9r}r9r9r<rFrucC(t|tr t|}t|trt|S|Sr)r`r>r"rJr&r}r9r9r<rF  cCr)z%JSON ISO8601 string into Python time.)r`r>r*rJr'r}r9r9r<rFrcCrz)JSON ISO8601 string into Python datetime.)r`r>r,rJr(r}r9r9r<rFrcCrr)r`r>r-rJr)r}r9r9r<rFrcCrN)z,JSON float deserializes into a Python float.r9r}r9r9r<rF%rucCrN)z,JSON string serializes into a Python string.r9r}r9r9r<rF,rOcCsLt|trt|td}t|t|kr$tdt|dt||S)+JSON hexadecimal encoded string into bytes.rrr)r`r>rrtrer rhrI)rrzr9r9r<rF2s cCs t|trt|tdS|S)rr)r`r>rrtrer r}r9r9r<rF>s cCrD)z*Convert JSON string into a Python Decimal.rr}r9r9r<rFGrGcCs.t|tr t|St|trtj|dS|S)z%Convert JSON string into Python UUID.)rS)r`r>rQrRrSr}r9r9r<rFMs    )O__doc__rrQrrrdecimalr functoolsrstructrtypingr r r r pyiceberg.typedefr rpyiceberg.typesrrrrrrrrrrrrrrrrrr r!pyiceberg.utils.datetimer"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1pyiceberg.utils.decimalr2r3rYr\r^rcrdr?r>rJrKrRrSrBregisterrFboolrVrnr|rr9r9r9r<s\   TH* ((  ""   " " "    "  "" " " " "  *