o .i<@sUddlZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z m Z ddlZddlZddlmZddlmZddlmZdd lmZmZdd lmZmZmZe raddlZ d d l!m"Z"da#e e$e%e&d <ej'dkrrdndZ(e)de)de)de)de)de)de)de)de)de)de)de)de)de)dgZ*eGdd d Z+d!e$e%fd"d#Z,d$d%d!e-fd&d'Z.d$d%d!e/fd(d)Z0d*ej1d!e/fd+d,Z2d-e e$e%e$e/e$ej1e$d%fd!e$e/fd.d/Z3dS)0N) dataclassfield)BytesIO) TYPE_CHECKINGAnyClassVarOptionalUnion)config)DownloadConfig) array_cast) is_local_pathxopen)first_non_null_valueno_op_if_value_is_nullstring_to_dict) FeatureType_IMAGE_COMPRESSION_FORMATSlittle<>z|b1|u1zu2zi2zu4zi4zf4zf8c@seZdZUdZdZeeed<dZe ed<dZ eeed<dZ e eed<e e e d Ze eed <edd d d Zeed <ddZdeeeeejdfdefddZddeddfddZdedeedfffddZdee je je j fde jfddZ!de jde jfddZ"dS)ImageaImage [`Feature`] to read image data from an image file. Input: The Image feature accepts as input: - A `str`: Absolute path to the image file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the image file to the archive file. - `bytes`: Bytes of the image file. This is useful for archived files with sequential access. - An `np.ndarray`: NumPy array representing an image. - A `PIL.Image.Image`: PIL image object. Args: mode (`str`, *optional*): The mode to convert the image to. If `None`, the native mode of the image is used. decode (`bool`, defaults to `True`): Whether to decode the image data. If `False`, returns the underlying dictionary in the format `{"path": image_path, "bytes": image_bytes}`. Examples: ```py >>> from datasets import load_dataset, Image >>> ds = load_dataset("AI-Lab-Makerere/beans", split="train") >>> ds.features["image"] Image(decode=True, id=None) >>> ds[0]["image"] >>> ds = ds.cast_column('image', Image(decode=False)) {'bytes': None, 'path': '/root/.cache/huggingface/datasets/downloads/extracted/b0a21163f78769a2cf11f58dfc767fb458fc7cea5c05dccc0144a2c0f0bc1292/train/healthy/healthy_train.85.jpg'} ``` NmodeTdecodeidPIL.Image.Imagedtypebytespathpa_typeF)defaultinitrepr_typecCs|jSN)r#)selfr*K/home/ubuntu/.local/lib/python3.10/site-packages/datasets/features/image.py__call__[szImage.__call__valuereturncCstjrddl}ntdt|trt|}t|tr |ddSt|t r*d|dSt|tj r4t |St||j j r?t |S|ddurVtj|drVd|ddS|ddusd|dduro|d|ddStd|d ) aEncode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `PIL.Image.Image` or `dict`): Data passed as input to Image feature. Returns: `dict` with "path" and "bytes" fields rN4To support encoding images, please install 'Pillow'.r"r!r"r r!zUAn image sample should have one of 'path' or 'bytes' but they are missing or None in .)r PIL_AVAILABLE PIL.Image ImportError isinstancelistnparraystrr!ndarrayencode_np_arrayrencode_pil_imagegetosr"isfile ValueError)r)r-PILr*r*r+encode_example^s(         zImage.encode_examplec Cs~|jstdtjrddl}ddl}ntd|duri}|d|d}}|dur|dur6td|dt|rA|j |}nV| d d }| tj rQtjntj}t||} | durd|| d nd} t| d } t|d | d} t| }Wdn1swY|j |}n|j t|}|||j jjjdur|j|}|jr|j|jkr||j}|S)aqDecode example image file into image data. Args: value (`str` or `dict`): A string with the absolute image file path, a dictionary with keys: - `path`: String with absolute or relative image file path. - `bytes`: The bytes of the image file. token_per_repo_id (`dict`, *optional*): To access and decode image files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `PIL.Image.Image` zMDecoding is disabled for this feature. Please use Image(decode=True) instead.rNz4To support decoding images, please install 'Pillow'.r"r!zCAn image should have one of 'path' or 'bytes' but both are None in r1z::repo_id)tokenrb)download_config)r RuntimeErrorr r2r3 PIL.ImageOpsr4r@rropensplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrr=r rrreadloadgetexifExifTagsBase OrientationImageOpsexif_transposerconvert) r)r-token_per_repo_idrAr"bytes_image source_urlpatternsource_url_fieldsrErGfr*r*r+decode_examplesD      zImage.decode_examplercCs(ddlm}|jr |S|d|ddS)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr )featuresrar)r)rar*r*r+flattens z Image.flattenstoragecCstj|jr%tjdgt|td}tjj||gddg| d}ntj |jrJtjdgt|t d}tjj||gddg| d}ntj |jr|j ddkr_|d}ntjdgt|td}|j ddkr{|d}ntjdgt|t d}tjj||gddg| d}n4tj|jrtjdd|Dtd}tjdgt|t d}tjj||gddg| d}t||jS) a'Cast an Arrow array to the Image arrow storage type. The Arrow types that can be converted to the Image pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the image bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the image array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Ntyper!r"maskrcSs*g|]}|durtt|dndqS)Nr!)r;r7r8).0arrr*r*r+ s*z&Image.cast_storage..)patypes is_stringrhr8lenrb StructArray from_arraysis_null is_binaryrc is_structget_field_indexris_list to_pylistr r#r)rf bytes_array path_arrayr*r*r+ cast_storages0      zImage.cast_storagecstddtjfdd|Dtd}tjdd|dDtd}tjj||gddg| d }t ||j S) a8Embed image files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Image arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. cSs8t|d }|}Wd|S1swY|S)NrF)rrP)r"r_rZr*r*r+ path_to_bytes s   z*Image.embed_storage..path_to_bytescs8g|]}|dur|ddur|dn|dndqS)Nr!r"r*)rkxr~r*r+rms*z'Image.embed_storage..rgcSs$g|]}|durtj|ndqSr()r>r"basename)rkr"r*r*r+rms$r"r!ri) rrnr8ryrbrrcrrrsrtr r#rzr*rr+ embed_storages   zImage.embed_storager()#__name__ __module__ __qualname____doc__rrr9__annotations__rboolrrrrnstructrbrcr#rrr'r,r r!dictr7r:rBr`re StringArrayrr ListArrayr}rr*r*r*r+r-s $ $"'=$ .rr.cCsRtjrddl}ntdtdur'|jtt|jj t|jj @atS)Nrr/) r r2r3r4rrr%r6setOPENkeysSAVE)rAr*r*r+list_image_compression_formatss  $rr[rcCsBt}|jtvr |j}n |jdvrdnd}|j||d|S)zmConvert a PIL Image object to bytes using native compression if possible, otherwise use PNG/TIFF compression.)1LLARGBRGBAPNGTIFF)format)rrrrsavegetvalue)r[bufferrr*r*r+image_to_bytes+s  rcCs.t|dr|jdkr|jddSdt|dS)Nfilenamer0)hasattrrr)r[r*r*r+r<6s r<r8c Cs>tjrddl}ntd|j}|jdkr|jnt}|j}|j}d}|j ddrM|dvr7t d|d|dt d }||krLt d |d |d nB|tvrT|}n;|d kr||t|}t |tvryt |}t d |d |d n|d}|d ksX|durt d|dt|j||}dt|dS)Nrr/=r )uizUnsupported array dtype z for image encoding. Only z' is supported for multi-channel arrays.rzDowncasting array dtype z to z to be compatible with 'Pillow'rzCannot downcast dtype z- to a valid image dtype. Valid image dtypes: r0)r r2r3r4r byteorder_NATIVE_BYTEORDERkinditemsizeshape TypeErrorr7warningswarn_VALID_IMAGE_ARRAY_DTPYESr9r fromarrayastyper) r8rArdtype_byteorder dtype_kinddtype_itemsize dest_dtype dtype_strr[r*r*r+r;=sB   r;objscstjrddl}ntd|rIt|\}}t|tr dd|DSt|tjr3t t fdd|DSt||j j rGt t fdd|DS|S|S)zmEncode a list of objects into a format suitable for creating an extension array of type `ImageExtensionType`.rNr/cSs"g|] }|dur |ddndqS)Nr0r*rkobjr*r*r+rmts"z2objects_to_list_of_image_dicts..cg|]}|qSr*r*robj_to_image_dict_funcr*r+rmwcrr*r*rrr*r+rmzr) r r2r3r4rr5r9r7r:rr;rr<)rrA_rr*rr+objects_to_list_of_image_dictshs    r)4r>sysr dataclassesrriortypingrrrrr numpyr7pyarrowrnrr download.download_configr tabler utils.file_utilsrrutils.py_utilsrrrr3rArdrrr6r9rrrrrrrr!rrr<r:r;rr*r*r*r+sZ      q  +