o ۷i`@sddlmZddlmZddlmZmZddlmZm Z m Z m Z ddl Z ddlmZmZmZddlmZmZmZmZmZdd lmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#dd l$m%Z%dd l&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,dd l-m.Z.e+r{dd lm/Z/e)rddl0Z0e*rddl1m2Z3ddlm4Z4ndZ4e,5e6Z7edddddddddddddej8f de e9de e:de e9de e e:e;e:fde e e:e;e:fde e9de ede e9de ede dde e ed$d"fd%d&Z?d'eed$e;efd(d)Z@d*e;d"d$eAe>d+ffd,d-ZBd.e e jCd"fd/e>d$e;e e jCd"ffd0d1ZDGd2d3d3e d4d5ZEe(Gd6d7d7eZFdS)9)Iterable)deepcopy) lru_cachepartial)AnyOptional TypedDictUnionN)BaseImageProcessor BatchFeature get_size_dict)convert_to_rgbget_resize_output_image_sizeget_size_with_aspect_ratiogroup_images_by_shapereorder_images) ChannelDimension ImageInput ImageTypeSizeDictget_image_size#get_image_size_for_max_height_widthget_image_typeinfer_channel_dimension_formatmake_flat_list_of_imagesvalidate_kwargsvalidate_preprocess_arguments)Unpack) TensorTypeauto_docstringis_torch_availableis_torchvision_availableis_vision_availablelogging)is_rocm_platform)PILImageResampling) functional)pil_torch_interpolation_mapping maxsize do_rescalerescale_factor do_normalize image_mean image_stddo_center_crop crop_size do_resizesize interpolationF.InterpolationModereturn_tensors data_formatc CsJt|||||||||| d | dur| dkrtd| tjkr#tddS)z Checks validity of typically used arguments in an `ImageProcessorFast` `preprocess` method. Raises `ValueError` if arguments incompatibility is caught. ) r,r-r.r/r0r1r2r3r4r5Nptz6Only returning PyTorch tensors is currently supported.z6Only channel first data format is currently supported.)r ValueErrorrFIRST) r,r-r.r/r0r1r2r3r4r5r7r8r<^/home/ubuntu/vllm_env/lib/python3.10/site-packages/transformers/image_processing_utils_fast.py"validate_fast_preprocess_argumentsIs"  r>tensor torch.TensoraxisreturncCs6|dur|Sz|j|dWSty|YSw)zF Squeezes a tensor, but only if the axis specified has dim 1. N)rA)squeezer:)r?rAr<r<r= safe_squeezeps rDvaluescCsddt|DS)zO Return the maximum value across all indices of an iterable of values. cSsg|]}t|qSr<)max).0values_ir<r<r= z&max_across_indices..)zip)rEr<r<r=max_across_indices}srLimages.cCs tdd|D\}}}||fS)zH Get the maximum height and width across all images in a batch. cSsg|]}|jqSr<)shaperGimgr<r<r=rIsz(get_max_height_width..)rL)rM_ max_height max_widthr<r<r=get_max_height_widthsrTimage patch_sizecCsjg}t|tjd\}}td||D]!}td||D]}|dd||||||f}||qq|S)a6 Divides an image into patches of a specified size. Args: image (`Union[np.array, "torch.Tensor"]`): The input image. patch_size (`int`): The size of each patch. Returns: list: A list of Union[np.array, "torch.Tensor"] representing the patches. ) channel_dimrN)rrr;rangeappend)rUrVpatchesheightwidthijpatchr<r<r=divide_to_patchess" r`c@s>eZdZUeeed<eeeefed<eeed<ee ded<eeed<eeeefed<eeed<ee ee fed <eeed <ee e e e fed <ee e e e fed <eeed <eeeefed<eeed<ee ee fed<ee ed<ee ee fed<eded<eeed<dS)DefaultFastImageProcessorKwargsr3r4default_to_square)r&r6resampler1r2r,r-r.r/r0do_padpad_sizedo_convert_rgbr7r8input_data_format torch.devicedevicedisable_groupingN)__name__ __module__ __qualname__rbool__annotations__dictstrintr floatlistrrr<r<r<r=ras(         raF)totalc!seZdZdZdZdZdZdZdZdZ dZ dZ dZ dZ dZdZdZdZejZdZdZdgZeZdZdeeffdd Zedefd d Z  d[ddde de!e"de!e#dede!eddfddZ$  d\ddde de!ddeddf ddZ%e&  d\ddde'e"e"fde!ddeddf dd Z(ddd!e)ddfd"d#Z*ddd$e+e)e,e)fd%e+e)e,e)fddfd&d'Z-e.d(d)      d]d*e!ed+e!e+e)e/e)fd,e!e+e)e/e)fd-e!ed.e!e)d/e!d0de'fd1d2Z0ddd-ed.e)d*ed+e+e)e/e)fd,e+e)e/e)fddfd3d4Z1ddde ddfd5d6Z2de3de3fd7d8Z4de5fd9d:Z6 ;d^de3dZ7   d_de3d?e!ed@e!e+e#efd/e!d0ddf dAdBZ8    ;d`de3d?e!ed@e!e+e#efd/e!d0de?de3deede=fdQdRZ@ddSde3d?ed@ed/e!e+e#d0fdeede=f dTdUZAde/ddJede de!ddKedEe d-ed.e)d*ed+e!e+e)e/e)fd,e!e+e)e/e)fdVe!ede!e de!edLe!e+e#e;fde=f dWdXZBfdYdZZCZDS)cBaseImageProcessorFastNTgp? pixel_valueskwargsc stjdi|||}|d|j}|dur$t||d|jdnd|_|d|j}|dur8t|ddnd|_|d|j}|durLt|ddnd|_|j j D]}||d}|durft |||qSt ||t t ||dqSt|j j |_dS) Nr4rbr4rbr2 param_namerer4r{r<)super__init__filter_out_unused_kwargspopr4r rbr2re valid_kwargsrosetattrrgetattrrtkeys_valid_kwargs_names)selfrxr4r2rekeykwarg __class__r<r=r~s"   zBaseImageProcessorFast.__init__rBcCsdS)zv `bool`: Whether or not this image processor is a fast processor (backed by PyTorch and TorchVision). Tr<)rr<r<r=is_fastszBaseImageProcessorFast.is_fastrconstantFrMr@re fill_value padding_mode return_maskrjcKsX|dur|jr |jstd|d|j|jf}nt|}t||d\}} i} i} |D]l\} } | jdd}|d|d}|d|d}|dksP|dkr[td|d |d||krndd||f}tj| |||d } | | | <|rt j | t j d d dddddf}d|d d|dd|df<|| | <q-t | | }|rt | | }||fS|S) a Pads images to `(pad_size["height"], pad_size["width"])` or to the largest size in the batch. Args: images (`torch.Tensor`): Images to pad. pad_size (`SizeDict`, *optional*): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. fill_value (`int`, *optional*, defaults to `0`): The constant value used to fill the padded area. padding_mode (`str`, *optional*, defaults to "constant"): The padding mode to use. Can be any of the modes supported by `torch.nn.functional.pad` (e.g. constant, reflection, replication). return_mask (`bool`, *optional*, defaults to `False`): Whether to return a pixel mask to denote padded regions. disable_grouping (`bool`, *optional*, defaults to `False`): Whether to disable grouping of images by size. Returns: `torch.Tensor`: The resized image. NzCPad size must contain 'height' and 'width' keys only. Got pad_size=.rjrr zrPadding dimensions are negative. Please make sure that the `pad_size` is larger than the image size. Got pad_size=z , image_size=)fillrdtype.) r[r\r:rTritemsrNFpadtorch zeros_likeint64r)rrMrerrrrjrxgrouped_imagesgrouped_images_indexprocessed_images_groupedprocessed_masks_groupedrNstacked_images image_sizepadding_height padding_widthpadding stacked_masksprocessed_imagesprocessed_masksr<r<r=rsD  $  zBaseImageProcessorFast.padrUr4r5r6 antialiascKs|dur|ntjj}|jr|jrt|dd|j|j}n8|jr-t||jdtj d}n*|j rB|j rBt |dd|j |j }n|j rO|jrO|j |jf}ntd|dtjrgtrg|||||Stj||||dS)a@ Resize an image to `(size["height"], size["width"])`. Args: image (`torch.Tensor`): Image to resize. size (`SizeDict`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. interpolation (`InterpolationMode`, *optional*, defaults to `InterpolationMode.BILINEAR`): `InterpolationMode` filter to use when resizing the image e.g. `InterpolationMode.BICUBIC`. Returns: `torch.Tensor`: The resized image. NrF)r4rbrgzjSize must contain 'height' and 'width' keys, or 'max_height' and 'max_width', or 'shortest_edge' key. Got rr5r)rInterpolationModeBILINEAR shortest_edge longest_edgerr4rrr;rRrSrr[r\r:rcompiler is_compilingr%compile_friendly_resizeresize)rrUr4r5rrxnew_sizer<r<r=r7s4   zBaseImageProcessorFast.resizercCs|jtjkr5|d}tj||||d}|d}t|dkd|}t|dkd|}|tj}|Stj||||d}|S)z{ A wrapper around `F.resize` so that it is compatible with torch.compile when the image is a uint8 tensor. rr) rruint8rsrrwhereroundto)rUrr5rr<r<r=rms z.BaseImageProcessorFast.compile_friendly_resizescalecKs||S)a? Rescale an image by a scale factor. image = image * scale. Args: image (`torch.Tensor`): Image to rescale. scale (`float`): The scaling factor to rescale pixel values by. Returns: `torch.Tensor`: The rescaled image. r<)rrUrrxr<r<r=rescaleszBaseImageProcessorFast.rescalemeanstdcKst|||S)a Normalize an image. image = (image - image_mean) / image_std. Args: image (`torch.Tensor`): Image to normalize. mean (`torch.Tensor`, `float` or `Iterable[float]`): Image mean to use for normalization. std (`torch.Tensor`, `float` or `Iterable[float]`): Image standard deviation to use for normalization. Returns: `torch.Tensor`: The normalized image. )r normalize)rrUrrrxr<r<r=rsz BaseImageProcessorFast.normalizer)r*r.r/r0r,r-rirhcCsB|r|rtj||dd|}tj||dd|}d}|||fS)Nrig?F)rr?)rr.r/r0r,r-rir<r<r=!_fuse_mean_std_and_rescale_factors   z8BaseImageProcessorFast._fuse_mean_std_and_rescale_factorcCsR|j||||||jd\}}}|r||jtjd||}|S|r'|||}|S)z/ Rescale and normalize images. )r.r/r0r,r-rir)rrirrrfloat32r)rrMr,r-r.r/r0r<r<r=rescale_and_normalizes    z,BaseImageProcessorFast.rescale_and_normalizec Ks"|jdus |jdurtd||jdd\}}|j|j}}||ks+||krx||kr5||dnd||kr@||dnd||krM||ddnd||krZ||ddndg}tj||dd}|jdd\}}||krx||krx|St||d} t||d} t|| | ||S) a Note: override torchvision's center_crop to have the same behavior as the slow processor. Center crop an image to `(size["height"], size["width"])`. If the input size is smaller than `crop_size` along any edge, the image is padded with 0's and then center cropped. Args: image (`"torch.Tensor"`): Image to center crop. size (`dict[str, int]`): Size of the output image. Returns: `torch.Tensor`: The center cropped image. Nz=The size dictionary must have keys 'height' and 'width'. Got rrr )rg@) r[r\r:rrNrrrrcrop) rrUr4rx image_height image_width crop_height crop_width padding_ltrbcrop_top crop_leftr<r<r= center_crops"z"BaseImageProcessorFast.center_cropcCst|S)a' Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image as is. Args: image (ImageInput): The image to convert. Returns: ImageInput: The converted image. )r)rrUr<r<r=rsz%BaseImageProcessorFast.convert_to_rgbcCsB|jdur|S|jD]}||vrtd|d||q |S)zJ Filter out the unused kwargs from the kwargs dictionary. Nz!This processor does not use the `z ` parameter. It will be ignored.) unused_kwargslogger warning_oncer)rrx kwarg_namer<r<r=rs   z/BaseImageProcessorFast.filter_out_unused_kwargsexpected_ndimscCs||}t||dS)z Prepare the images structure for processing. Args: images (`ImageInput`): The input images to process. Returns: `ImageInput`: The images with a valid nesting. r) fetch_imagesr)rrMrr<r<r=_prepare_images_structure$s  z0BaseImageProcessorFast._prepare_images_structurerfrgcCst|}|tjtjtjfvrtd||r||}|tjkr't|}n |tjkr3t | }|j dkr=| d}|durEt|}|tjkrS|ddd }|dur\||}|S)NzUnsupported input image type rrr )rrPILTORCHNUMPYr:rr pil_to_tensorr from_numpy contiguousndim unsqueezerrLASTpermuter)rrUrfrgri image_typer<r<r=_process_image7s$        z%BaseImageProcessorFast._process_imagecsn|j||d}t|j|||dt|dkot|dttf}|r,fdd|D}|Sfdd|D}|S)a Prepare image-like inputs for processing. Args: images (`ImageInput`): The image-like inputs to process. do_convert_rgb (`bool`, *optional*): Whether to convert the images to RGB. input_data_format (`str` or `ChannelDimension`, *optional*): The input data format of the images. device (`torch.device`, *optional*): The device to put the processed images on. expected_ndims (`int`, *optional*): The expected number of dimensions for the images. (can be 2 for segmentation maps etc.) Returns: List[`torch.Tensor`]: The processed images. rrfrgrircsg|] }fdd|DqS)cg|]}|qSr<r<rOprocess_image_partialr<r=rIrJzPBaseImageProcessorFast._prepare_image_like_inputs...r<)rG nested_listrr<r=rIszEBaseImageProcessorFast._prepare_image_like_inputs..crr<r<rOrr<r=rIrJ)rrrlen isinstancerttuple)rrMrfrgrirhas_nested_structurerr<rr=_prepare_image_like_inputs]s z1BaseImageProcessorFast._prepare_image_like_inputsr2rbr8c Ks|duri}|durtd it||d}|dur$td it|dd}|dur3td it|dd}t|tr)rr,r-r.r/r0r3r4r1r2r5r7r8rxr<r<r=_validate_preprocess_kwargss z2BaseImageProcessorFast._validate_preprocess_kwargscOs|j|g|Ri|SN) preprocess)rrMargsrxr<r<r=__call__szBaseImageProcessorFast.__call__cOst||jd|jD] }||t||dq |d}|d}|d}|jdi|}|jdi||d|j|g|R|||d|S)N)captured_kwargsvalid_processor_keysrfrgrir8rr<) rrr setdefaultrrrr_preprocess_image_like_inputs)rrMrrxrrfrgrir<r<r=rs$     z!BaseImageProcessorFast.preprocessrcOs*|j||||d}|j|g|Ri|S)z Preprocess image-like inputs. To be overridden by subclasses when image-like inputs other than images should be processed. It can be used for segmentation maps, depth maps, etc. )rMrfrgri)r _preprocess)rrMrfrgrirrxr<r<r=rsz4BaseImageProcessorFast._preprocess_image_like_inputsrdc Kst||d\}}i}|D]\}}|r|j|||d}|||<qt||}t||d\}}i}|D]\}}|r@|||}||||| | | }|||<q4t||}| r^|j|| |d}|rgtj|ddn|}t d|i|dS)Nr)rUr4r5)rerjr)dimrw)data tensor_type) rrrrrrrrstackr )rrMr3r4r5r1r2r,r-r.r/r0rdrerjr7rxrrresized_images_groupedrNrresized_imagesrrr<r<r=r s*      z"BaseImageProcessorFast._preprocesscs&t}|dd|dd|S)N_valid_processor_keysr)r}to_dictr)r encoder_dictrr<r=r=s   zBaseImageProcessorFast.to_dict)NrrFF)NT)NNNNNN)r)NNN)NNNr)NNNNNNN) NNNNNNNNNNNN)Erkrlrmrcr/r0r4rbr2r3r1rdrer,r-r.rfr7rr;r8rgrimodel_input_namesrarrrr~propertyrnrrrrrrqrr staticmethodrrrsrr rrrrtrrrrrrprrrrrrrr rr rrrr __classcell__r<r<rr=rvsB  H 6       )   ) . 2     "        0rvr)Gcollections.abcrcopyr functoolsrrtypingrrrr numpynpimage_processing_utilsr r r image_transformsrrrrr image_utilsrrrrrrrrrrrprocessing_utilsrutilsrr r!r"r#r$utils.import_utilsr%r&rtorchvision.transforms.v2r'rr( get_loggerrkrr;rnrsrtrqr>rrrDrLrrTndarrayr`rarvr<r<r<r=s  4         &