o 8wiY?@sUddlZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z m Z ddlZddlZddlmZddlmZddlmZdd lmZmZdd lmZmZmZe raddlZ d d l!m"Z"da#e e$e%e&d <ej'dkrrdndZ(e)de)de)de)de)de)de)de)de)de)de)de)de)de)dgZ*eGdd d Z+d!e$e%fd"d#Z,d$d%d!e-fd&d'Z.d$d%d!e/fd(d)Z0d*ej1d!e/fd+d,Z2d-e e$e%e$e/e$ej1e$d%fd!e$e/fd.d/Z3dS)0N) dataclassfield)BytesIO) TYPE_CHECKINGAnyClassVarOptionalUnion)config)DownloadConfig) array_cast) is_local_pathxopen)first_non_null_valueno_op_if_value_is_nullstring_to_dict) FeatureType_IMAGE_COMPRESSION_FORMATSlittle<>z|b1|u1zu2zi2zu4zi4zf4zf8c@s*eZdZUdZdZeeed<dZe ed<e dddZ eeed<d Z e eed <eeed Ze eed <e dddd Zeed<ddZdeeeeeejd fdefddZddedd fddZdedeedfffddZdeejej ej!fdej fddZ"ddej dej fddZ#dS) ImageaImage [`Feature`] to read image data from an image file. Input: The Image feature accepts as input: - A `str`: Absolute path to the image file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the image file to the archive file. - `bytes`: Bytes of the image file. This is useful for parquet or webdataset files which embed image files. - An `np.ndarray`: NumPy array representing an image. - A `PIL.Image.Image`: PIL image object. Output: The Image features output data as `PIL.Image.Image` objects. Args: mode (`str`, *optional*): The mode to convert the image to. If `None`, the native mode of the image is used. decode (`bool`, defaults to `True`): Whether to decode the image data. If `False`, returns the underlying dictionary in the format `{"path": image_path, "bytes": image_bytes}`. Examples: ```py >>> from datasets import load_dataset, Image >>> ds = load_dataset("AI-Lab-Makerere/beans", split="train") >>> ds.features["image"] Image(decode=True, id=None) >>> ds[0]["image"] >>> ds = ds.cast_column('image', Image(decode=False)) {'bytes': None, 'path': '/root/.cache/huggingface/datasets/downloads/extracted/b0a21163f78769a2cf11f58dfc767fb458fc7cea5c05dccc0144a2c0f0bc1292/train/healthy/healthy_train.85.jpg'} ``` NmodeTdecodeF)defaultrepridPIL.Image.Imagedtypebytespathpa_type)rinitr_typecCs|jSN)r%)selfr*T/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/datasets/features/image.py__call__]szImage.__call__valuereturncCstjrddl}ntdt|trt|}t|tr |ddSt|t t fr,d|dSt|tj r6t |St||j j rAt|S|ddurXtj|drXd|ddS|ddusf|ddurq|d|ddStd|d ) aEncode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `PIL.Image.Image` or `dict`): Data passed as input to Image feature. Returns: `dict` with "path" and "bytes" fields rN4To support encoding images, please install 'Pillow'.r$r#r$r"r#zUAn image sample should have one of 'path' or 'bytes' but they are missing or None in .)r PIL_AVAILABLE PIL.Image ImportError isinstancelistnparraystrr# bytearrayndarrayencode_np_arrayrencode_pil_imagegetosr$isfile ValueError)r)r-PILr*r*r+encode_example`s(        zImage.encode_examplec Cs~|jstdtjrddl}ddl}ntd|duri}|d|d}}|dur|dur6td|dt|rA|j |}nV| d d }| tj rQtjntj}t||} | durd|| d nd} t| d } t|d | d} t| }Wdn1swY|j |}n|j t|}|||j jjjdur|j|}|jr|j|jkr||j}|S)aqDecode example image file into image data. Args: value (`str` or `dict`): A string with the absolute image file path, a dictionary with keys: - `path`: String with absolute or relative image file path. - `bytes`: The bytes of the image file. token_per_repo_id (`dict`, *optional*): To access and decode image files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `PIL.Image.Image` zMDecoding is disabled for this feature. Please use Image(decode=True) instead.rNz4To support decoding images, please install 'Pillow'.r$r#zCAn image should have one of 'path' or 'bytes' but both are None in r1::repo_idtokenrbdownload_config)r RuntimeErrorr r2r3 PIL.ImageOpsr4rArropensplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrr>r rrreadloadgetexifExifTagsBase OrientationImageOpsexif_transposerconvert) r)r-token_per_repo_idrBr$bytes_image source_urlpatternsource_url_fieldsrHrKfr*r*r+decode_examplesD      zImage.decode_examplercCs(ddlm}|jr |S|d|ddS)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr")featuresrer)r)rer*r*r+flattens z Image.flattenstoragecCstj|jr%tjdgt|td}tjj||gddg| d}ntj |jrJtjdgt|t d}tjj||gddg| d}ntj |jr|j ddkr_|d}ntjdgt|td}|j ddkr{|d}ntjdgt|t d}tjj||gddg| d}n4tj|jrtjdd|Dtd}tjdgt|t d}tjj||gddg| d}t||jS) a'Cast an Arrow array to the Image arrow storage type. The Arrow types that can be converted to the Image pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the image bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the image array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Ntyper#r$maskrcSs*g|]}|durtt|dndqS)Nr#)r<r7r8).0arrr*r*r+ s*z&Image.cast_storage..)patypes is_stringrlr8lenrf StructArray from_arraysis_null is_binaryrg is_structget_field_indexris_list to_pylistr r%)r)rj bytes_array path_arrayr*r*r+ cast_storages0      zImage.cast_storagecsduritfddtjfdd|Dtd}tjdd|dDtd}tjj||gd dg| d }t ||j S) a8Embed image files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Ncs|dd}|tjrtjntj}t||}|dur#|dnd}t|d}t |d|d }| WdS1s@wYdS)NrDrErFrGrIrJ) rOrPr rQrRrSrr>r rrT)r$r`rarbrHrKrc)r]r*r+ path_to_bytes s  $z*Image.embed_storage..path_to_bytescs8g|]}|dur|ddur|dn|dndqS)Nr#r$r*)rox)rr*r+rqs*z'Image.embed_storage..rkcSs$g|]}|durtj|ndqSr()r?r$basename)ror$r*r*r+rq!s$r$r#rm) rrrr8r}rfrrgrvrwrxr r%)r)rjr]r~rr*)rr]r+ embed_storages    zImage.embed_storager()$__name__ __module__ __qualname____doc__rrr9__annotations__rboolrrr!rrrstructrfrgr%rr'r,r r#r:dictr7r;rCrdri StringArrayrv ListArrayrrr*r*r*r+r-s & $$'=$ .rr.cCsRtjrddl}ntdtdur'|jtt|jj t|jj @atS)Nrr/) r r2r3r4rrr&r6setOPENkeysSAVE)rBr*r*r+list_image_compression_formats(s  $rr_r cCsBt}|jtvr |j}n |jdvrdnd}|j||d|S)zmConvert a PIL Image object to bytes using native compression if possible, otherwise use PNG/TIFF compression.)1LLARGBRGBAPNGTIFF)format)rrrrsavegetvalue)r_bufferrr*r*r+image_to_bytes5s  rcCs.t|dr|jdkr|jddSdt|dS)Nfilenamer0)hasattrrr)r_r*r*r+r=@s r=r8c Cs>tjrddl}ntd|j}|jdkr|jnt}|j}|j}d}|j ddrM|dvr7t d|d|dt d }||krLt d |d |d nB|tvrT|}n;|d kr||t|}t |tvryt |}t d |d |d n|d}|d ksX|durt d|dt|j||}dt|dS)Nrr/=r )uizUnsupported array dtype z for image encoding. Only z' is supported for multi-channel arrays.rzDowncasting array dtype z to z to be compatible with 'Pillow'rzCannot downcast dtype z- to a valid image dtype. Valid image dtypes: r0)r r2r3r4r! byteorder_NATIVE_BYTEORDERkinditemsizeshape TypeErrorr7warningswarn_VALID_IMAGE_ARRAY_DTPYESr9r fromarrayastyper) r8rBr!dtype_byteorder dtype_kinddtype_itemsize dest_dtype dtype_strr_r*r*r+r<GsB   r<objscstjrddl}ntd|rIt|\}}t|tr dd|DSt|tjr3t t fdd|DSt||j j rGt t fdd|DS|S|S)zmEncode a list of objects into a format suitable for creating an extension array of type `ImageExtensionType`.rNr/cSs"g|] }|dur |ddndqS)Nr0r*roobjr*r*r+rq~s"z2objects_to_list_of_image_dicts..cg|]}|qSr*r*robj_to_image_dict_funcr*r+rqcrr*r*rrr*r+rqr) r r2r3r4rr5r9r7r;rr<rr=)rrB_rr*rr+objects_to_list_of_image_dictsrs    r)4r?sysr dataclassesrriortypingrrrrr numpyr7pyarrowrrrr download.download_configr tabler utils.file_utilsrrutils.py_utilsrrrr3rBrhrrr6r9rrrr!rrrr#rrr=r;r<rr*r*r*r+sZ      {  +