o iX @sdZddlmZddlmZmZddlZddlm Z m Z m Z ddl m Z mZmZmZmZddlmZmZmZmZmZmZmZmZmZmZdd lmZmZm Z m!Z!e rWddl"Z"e!#e$Z%d e&e&efd d Z' ddej(de)deee*efd e+e)e)ffddZ,Gddde Z-dgZ.dS)zImage processor class for TVP.)Iterable)OptionalUnionN)BaseImageProcessor BatchFeature get_size_dict) PaddingModeflip_channel_orderpadresizeto_channel_dimension_format) IMAGENET_STANDARD_MEANIMAGENET_STANDARD_STDChannelDimension ImageInputPILImageResamplingget_image_sizeis_valid_imageto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_vision_availableloggingreturncCsrt|ttfrt|dttfrt|ddr|St|ttfr*t|dr*|gSt|r2|ggStd|)Nrz"Could not make batched video from ) isinstancelisttupler ValueError)videosr"`/home/ubuntu/.local/lib/python3.10/site-packages/transformers/models/tvp/image_processing_tvp.py make_batched5s0r$ input_imagemax_sizeinput_data_formatc Cs\t||\}}||kr|d|}|}||}n |d|}|}||}t|t|f}|S)Ng?)rint) r&r'r(heightwidthratio new_height new_widthsizer"r"r#get_resize_output_image_sizeBs   r0c+seZdZdZdgZddejdddddddejddddfde de e e e fd ed e d e e e e fd e d ee efde de e e e fdeeeefdede de de eeeefde eeeefddf fdd Zejddfdejde e e fd ede ee efde ee efdejf ddZddejddfdejde e e e fdeeeefdede ee efde ee eff ddZdddddddddddddddejdfdede e de e e e fd e ed e e d e e e e fd e e d e ede de e e e fde eeeefde ede e de e de eeeefde eeeefde ede ee efdejf&d d!Zeddddddddddddddddejdfd"eeeeeeefde e de e e e fd e ed e e d e e e e fd e e d e ede e de e e e fde eeeefde ede e de e de eeeefde eeeefd#e ee efdede ee efdejjf(d$d%ZZ S)&TvpImageProcessora Constructs a Tvp image processor. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions to the specified `size`. Can be overridden by the `do_resize` parameter in the `preprocess` method. size (`dict[str, int]` *optional*, defaults to `{"longest_edge": 448}`): Size of the output image after resizing. The longest edge of the image will be resized to `size["longest_edge"]` while maintaining the aspect ratio of the original image. Can be overridden by `size` in the `preprocess` method. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR`): Resampling filter to use if resizing the image. Can be overridden by the `resample` parameter in the `preprocess` method. do_center_crop (`bool`, *optional*, defaults to `True`): Whether to center crop the image to the specified `crop_size`. Can be overridden by the `do_center_crop` parameter in the `preprocess` method. crop_size (`dict[str, int]`, *optional*, defaults to `{"height": 448, "width": 448}`): Size of the image after applying the center crop. Can be overridden by the `crop_size` parameter in the `preprocess` method. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. Can be overridden by the `do_rescale` parameter in the `preprocess` method. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Defines the scale factor to use if rescaling the image. Can be overridden by the `rescale_factor` parameter in the `preprocess` method. do_pad (`bool`, *optional*, defaults to `True`): Whether to pad the image. Can be overridden by the `do_pad` parameter in the `preprocess` method. pad_size (`dict[str, int]`, *optional*, defaults to `{"height": 448, "width": 448}`): Size of the image after applying the padding. Can be overridden by the `pad_size` parameter in the `preprocess` method. constant_values (`Union[float, Iterable[float]]`, *optional*, defaults to 0): The fill value to use when padding the image. pad_mode (`PaddingMode`, *optional*, defaults to `PaddingMode.CONSTANT`): Use what kind of mode in padding. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. do_flip_channel_order (`bool`, *optional*, defaults to `True`): Whether to flip the color channels from RGB to BGR. Can be overridden by the `do_flip_channel_order` parameter in the `preprocess` method. image_mean (`float` or `list[float]`, *optional*, defaults to `IMAGENET_STANDARD_MEAN`): Mean to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_mean` parameter in the `preprocess` method. image_std (`float` or `list[float]`, *optional*, defaults to `IMAGENET_STANDARD_STD`): Standard deviation to use if normalizing the image. This is a float or list of floats the length of the number of channels in the image. Can be overridden by the `image_std` parameter in the `preprocess` method. pixel_valuesTNgp?r do_resizer/resampledo_center_crop crop_size do_rescalerescale_factordo_padpad_sizeconstant_valuespad_mode do_normalizedo_flip_channel_order image_mean image_stdrc stjdi||dur|nddi}|dur|nddd}| dur$| nddd} ||_||_||_||_||_||_||_||_ | |_ | |_ | |_ | |_ | |_|durV|nt|_|durb||_dSt|_dS)N longest_edger%)r*r+r")super__init__r3r/r5r6r4r7r8r9r:r;r<r=r>rr?rr@)selfr3r/r4r5r6r7r8r9r:r;r<r=r>r?r@kwargs __class__r"r#rCs&zTvpImageProcessor.__init__image data_formatr(cKstt|dd}d|vrd|vr|d|df}nd|vr$t||d|}n td|t|f||||d|S)a Resize an image. Args: image (`np.ndarray`): Image to resize. size (`dict[str, int]`): Size of the output image. If `size` is of the form `{"height": h, "width": w}`, the output image will have the size `(h, w)`. If `size` is of the form `{"longest_edge": s}`, the output image will have its longest edge of length `s` while keeping the aspect ratio of the original image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): Resampling filter to use when resiizing the image. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format of the image. If not provided, it will be the same as the input image. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. Fdefault_to_squarer*r+rAzCSize must have 'height' and 'width' or 'longest_edge' as keys. Got )r/r4rIr()rr0r keysr )rDrHr/r4rIr(rE output_sizer"r"r#r s zTvpImageProcessor.resizecKszt||d\}} |d|} |d| } | | | |} } | dks%| dkr)tdd| fd| ff}t||||||d}|S)a+ Pad an image with zeros to the given size. Args: image (`np.ndarray`): Image to pad. pad_size (`dict[str, int]`) Size of the output image with pad. constant_values (`Union[float, Iterable[float]]`) The fill value to use when padding the image. pad_mode (`PaddingMode`) The pad mode, default to PaddingMode.CONSTANT data_format (`ChannelDimension` or `str`, *optional*) The channel dimension format of the image. If not provided, it will be the same as the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. ) channel_dimr*r+rz0The padding size must be greater than image size)moder;rIr()rgetr r )rDrHr:r;r<rIr(rEr*r+ max_height max_width pad_right pad_bottompadding padded_imager"r"r# pad_images    zTvpImageProcessor.pad_imagec Kst||| |||||||d t|}|r|j||||d}|r'|j|||d}|r1|j|||d}| r@|j|tj|||d}| rL|j || | | |d}|rTt ||d}t |||d}|S) zPreprocesses a single image.) r7r8r=r?r@r5r6r3r/r4)rHr/r4r()r/r()rHscaler()rHmeanstdr()rHr:r;r<r()rHr()input_channel_dim) rrr center_croprescale normalizeastypenpfloat32rWr r )rDrHr3r/r4r5r6r7r8r9r:r;r<r=r>r?r@rIr(rEr"r"r#_preprocess_image sF z#TvpImageProcessor._preprocess_imager!return_tensorscsdurnjdurnjdurnjdur!njdur*njdur3nj dur< nj durEnj rL nj durUnj dur^nj  durg nj durp nj durynj tdddurnjtddt|stdt|} fdd|D}d |i}t||d S) a9 Preprocess an image or batch of images. Args: videos (`ImageInput` or `list[ImageInput]` or `list[list[ImageInput]]`): Frames to preprocess. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`dict[str, int]`, *optional*, defaults to `self.size`): Size of the image after applying resize. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`, Only has an effect if `do_resize` is set to `True`. do_center_crop (`bool`, *optional*, defaults to `self.do_centre_crop`): Whether to centre crop the image. crop_size (`dict[str, int]`, *optional*, defaults to `self.crop_size`): Size of the image after applying the centre crop. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image values between [0 - 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_pad (`bool`, *optional*, defaults to `True`): Whether to pad the image. Can be overridden by the `do_pad` parameter in the `preprocess` method. pad_size (`dict[str, int]`, *optional*, defaults to `{"height": 448, "width": 448}`): Size of the image after applying the padding. Can be overridden by the `pad_size` parameter in the `preprocess` method. constant_values (`Union[float, Iterable[float]]`, *optional*, defaults to 0): The fill value to use when padding the image. pad_mode (`PaddingMode`, *optional*, defaults to "PaddingMode.CONSTANT"): Use what kind of mode in padding. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. do_flip_channel_order (`bool`, *optional*, defaults to `self.do_flip_channel_order`): Whether to flip the channel order of the image. image_mean (`float` or `list[float]`, *optional*, defaults to `self.image_mean`): Image mean. image_std (`float` or `list[float]`, *optional*, defaults to `self.image_std`): Image standard deviation. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the inferred channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. NFrJr6) param_namezkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.csFg|]}t fdd|DqS)csg|]>}jdid|ddddddddd d d d d d d dd qS)rHr3r/r4r5r6r7r8r9r:r;r<r=r>r?r@rIr(r")rb).0imgr;r6rIr5r>r=r9r7r3r?r@r(r<r:r4r8rDr/r"r# sP      z;TvpImageProcessor.preprocess...)r`array)revideorgr"r#rhs,z0TvpImageProcessor.preprocess..r2)data tensor_type)r3r4r5r7r8r9r:r;r<r=r>r?r@r/rr6rr r$r)rDr!r3r/r4r5r6r7r8r9r:r;r<r=r>r?r@rcrIr(rkr"rgr# preprocessRs8P  , zTvpImageProcessor.preprocess)!__name__ __module__ __qualname____doc__model_input_namesrBILINEARr CONSTANTboolrdictstrr)rfloatrrrCr`ndarrayrr rWFIRSTrrbrrPILImagerm __classcell__r"r"rFr#r1Us1      ,  . 2      G     r1)r%N)/rqcollections.abcrtypingrrnumpyr`image_processing_utilsrrrimage_transformsr r r r r image_utilsrrrrrrrrrrutilsrrrrr{ get_loggerrnloggerrr$ryr)rwrr0r1__all__r"r"r"r#s8 0