o iH @s6dZddlmZddlmZmZddlZddlm Z m Z m Z ddl m Z mZmZddlmZmZmZmZmZmZmZmZmZmZmZdd lmZmZmZm Z m!Z!m"Z"dd l#m$Z$e r_ddl%Z%erfddl&Z&e!'e(Z)d d Z* dd ej+dee,dee,deeee,ffddZ-e$ddGddde Z.dgZ/dS)z%Image processor class for LayoutLMv3.)Iterable)OptionalUnionN)BaseImageProcessor BatchFeature get_size_dict)resizeto_channel_dimension_format to_pil_image) IMAGENET_STANDARD_MEANIMAGENET_STANDARD_STDChannelDimension ImageInputPILImageResamplinginfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_pytesseract_availableis_vision_availableloggingrequires_backends)requirescCsLtd|d|td|d|td|d|td|d|gS)Nirr)int)boxwidthheightr$n/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/layoutlmv3/image_processing_layoutlmv3.py normalize_box:s r&imagelangtesseract_configinput_data_formatcsRt||d}|j\}}tj||d|d}|d|d|d|d|df\}} } } } d d t|Dfd d t|D}fd d t| D} fd d t| D} fdd t| D} fdd t| D} g} t| | | | D]\}}}}||||||g}| |qsg}| D] }|t|||qt|t|ksJd||fS)zdApplies Tesseract OCR on a document image, and returns recognized words + normalized bounding boxes.r*dict)r( output_typeconfigtextlefttopr"r#cSsg|] \}}|s|qSr$)strip.0idxwordr$r$r% Rz#apply_tesseract..cg|] \}}|vr|qSr$r$r3irrelevant_indicesr$r%r7Sr8cr9r$r$r4r5coordr:r$r%r7Tr8cr9r$r$r<r:r$r%r7Ur8cr9r$r$r<r:r$r%r7Vr8cr9r$r$r<r:r$r%r7Wr8z-Not as many words as there are bounding boxes) r size pytesseract image_to_data enumeratezipappendr&len)r'r(r)r* pil_image image_width image_heightdatawordsr0r1r"r# actual_boxesxywh actual_boxnormalized_boxesr!r$r:r%apply_tesseractCs& , rQ)vision)backendsc!seZdZdZdgZddejddddddddf dedee e e fd ed ed e d ed ee e ee fdee e ee fdedee dee ddffdd Zejddfdejde e e fd edee e efdee e efdejf ddZeddddddddddddejdfdedeedee e e fd eedee d eed ee e ee fdee e ee fdeedee dee dee e efdedee e efdejjfddZZS)LayoutLMv3ImageProcessora Constructs a LayoutLMv3 image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to `(size["height"], size["width"])`. Can be overridden by `do_resize` in `preprocess`. size (`dict[str, int]` *optional*, defaults to `{"height": 224, "width": 224}`): Size of the image after resizing. Can be overridden by `size` in `preprocess`. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by `resample` in `preprocess`. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image's pixel values by the specified `rescale_value`. Can be overridden by `do_rescale` in `preprocess`. rescale_factor (`float`, *optional*, defaults to 1 / 255): Value by which the image's pixel values are rescaled. Can be overridden by `rescale_factor` in `preprocess`. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. image_mean (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`Iterable[float]` or `float`, *optional*, defaults to `IMAGENET_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. apply_ocr (`bool`, *optional*, defaults to `True`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. Can be overridden by the `apply_ocr` parameter in the `preprocess` method. ocr_lang (`str`, *optional*): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. Can be overridden by the `ocr_lang` parameter in the `preprocess` method. tesseract_config (`str`, *optional*): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. For example: '--psm 6'. Can be overridden by the `tesseract_config` parameter in the `preprocess` method. pixel_valuesTNgp? do_resizer>resample do_rescale rescale_value do_normalize image_mean image_std apply_ocrocr_langr)returnc  stjdi| |dur|nddd}t|}||_||_||_||_||_||_|dur0|nt |_ |dur9|nt |_ | |_ | |_| |_dS)N)r#r"r$)super__init__rrWr>rXrYrescale_factorr[r r\r r]r^r_r)) selfrWr>rXrYrZr[r\r]r^r_r)kwargs __class__r$r%rcs z!LayoutLMv3ImageProcessor.__init__r' data_formatr*cKsTt|}d|vs d|vrtd||d|df}t|f||||d|S)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. r#r"zFThe `size` dictionary must contain the keys `height` and `width`. Got )r>rXrir*)r ValueErrorkeysr )rer'r>rXrir*rf output_sizer$r$r%r s#zLayoutLMv3ImageProcessor.resizeimagesrdreturn_tensorsc s|dur|nj}durnjtdurnj|dur%|nj}dur.nj|dur7|nj}dur@njdurInj| durR| nj } | dur[| nj } | durd| nj } t |}t |sstdt|||ddd|D}|rt|drtddurt|d| rtdg}g}|D]}t|| | d \}}||||q|rχfd d|D}|r܇fd d|D}|rfd d|D}fd d|D}td|i| d}| r||d<||d<|S)a% Preprocess an image or batch of images. Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`dict[str, int]`, *optional*, defaults to `self.size`): Desired size of the output image after applying `resize`. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the `PILImageResampling` filters. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image pixel values between [0, 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to apply to the image pixel values. Only has an effect if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `Iterable[float]`, *optional*, defaults to `self.image_mean`): Mean values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`float` or `Iterable[float]`, *optional*, defaults to `self.image_std`): Standard deviation values to be used for normalization. Only has an effect if `do_normalize` is set to `True`. apply_ocr (`bool`, *optional*, defaults to `self.apply_ocr`): Whether to apply the Tesseract OCR engine to get words + normalized bounding boxes. ocr_lang (`str`, *optional*, defaults to `self.ocr_lang`): The language, specified by its ISO code, to be used by the Tesseract OCR engine. By default, English is used. tesseract_config (`str`, *optional*, defaults to `self.tesseract_config`): Any additional custom configuration flags that are forwarded to the `config` parameter when calling Tesseract. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)rYrdr[r\r]rWr>rXcSsg|]}t|qSr$)rr4r'r$r$r%r7Hsz7LayoutLMv3ImageProcessor.preprocess..rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.r?r+csg|] }j|dqS))r'r>rXr*)r ro)r*rXrer>r$r%r7_csg|] }j|dqS))r'scaler*)rescalero)r*rdrer$r%r7escsg|] }j|dqS))r'meanstdr*) normalizero)r\r]r*rer$r%r7krpcsg|] }t|dqS))input_channel_dim)r ro)rir*r$r%r7psrU)rH tensor_typerIboxes)rWr>rrXrYrdr[r\r]r^r_r)rrrjrrlogger warning_oncerrrQrCr)rermrWr>rXrYrdr[r\r]r^r_r)rnrir* words_batch boxes_batchr'rIrxrHr$)rir\r]r*rXrdrer>r% preprocesss|F      z#LayoutLMv3ImageProcessor.preprocess)__name__ __module__ __qualname____doc__model_input_namesrBILINEARboolrr,strr floatrrrcnpndarrayrr rFIRSTrrPILImager} __classcell__r$r$rgr%rTis&    $  0     rT)N)0rcollections.abcrtypingrrnumpyrimage_processing_utilsrrrimage_transformsr r r image_utilsr r rrrrrrrrrutilsrrrrrrutils.import_utilsrrr? get_loggerr~ryr&rrrQrT__all__r$r$r$r%s< 4    &