o i!@sddlZddlmZddlmZmZmZddlmZm Z m Z m Z m Z m Z ddlmZmZe r;ddlmZdd lmZe rHddlZdd lmZe rWdd lmZdd lmZe eZeed dGdddeZdS)N)UserDict)AnyUnionoverload)add_end_docstringsis_tf_availableis_torch_availableis_vision_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args)Image) load_image)6MODEL_FOR_ZERO_SHOT_IMAGE_CLASSIFICATION_MAPPING_NAMES)9TF_MODEL_FOR_ZERO_SHOT_IMAGE_CLASSIFICATION_MAPPING_NAMES)stable_softmaxT)has_image_processorcs.eZdZdZdZdZdZdZfddZe de e dfde e d e d e ee e ffd d Ze de e e e dfde e d e d e e ee e ffd d Zde e e e de dfde e d e d e e ee e fe e ee e ffffdd ZdddZ    dddZddZddZZS)#ZeroShotImageClassificationPipelineaL Zero shot image classification pipeline using `CLIPModel`. This pipeline predicts the class of an image when you provide an image and a set of `candidate_labels`. Example: ```python >>> from transformers import pipeline >>> classifier = pipeline(model="google/siglip-so400m-patch14-384") >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["animals", "humans", "landscape"], ... ) [{'score': 0.965, 'label': 'animals'}, {'score': 0.03, 'label': 'humans'}, {'score': 0.005, 'label': 'landscape'}] >>> classifier( ... "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png", ... candidate_labels=["black and white", "photorealist", "painting"], ... ) [{'score': 0.996, 'label': 'black and white'}, {'score': 0.003, 'label': 'photorealist'}, {'score': 0.0, 'label': 'painting'}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This image classification pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"zero-shot-image-classification"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=zero-shot-image-classification). FTc s>tjdi|t|d||jdkrtdStdS)Nvisiontf)super__init__r check_model_type frameworkrr)selfkwargs __class__ri/home/ubuntu/.local/lib/python3.10/site-packages/transformers/pipelines/zero_shot_image_classification.pyrHs  z,ZeroShotImageClassificationPipeline.__init__imagez Image.Imagecandidate_labelsrreturncKdSNrrr#r$rrrr"__call__Rz,ZeroShotImageClassificationPipeline.__call__cKr&r'rr(rrr"r)Wr*c s:d|vr |d}|durtdtj|fd|i|S)a Assign labels to the image(s) passed as inputs. Args: image (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`): The pipeline handles three types of images: - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly candidate_labels (`list[str]`): The candidate labels for this image. They will be formatted using *hypothesis_template*. hypothesis_template (`str`, *optional*, defaults to `"This is a photo of {}"`): The format used in conjunction with *candidate_labels* to attempt the image classification by replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are already formatted. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A list of dictionaries containing one entry per proposed label. Each dictionary contains the following keys: - **label** (`str`) -- One of the suggested *candidate_labels*. - **score** (`float`) -- The score attributed by the model to that label. It is a value between 0 and 1, computed as the `softmax` of `logits_per_image`. imagesNzSCannot call the zero-shot-image-classification pipeline without an images argument!r$)pop ValueErrorrr)r(r rr"r)\s % NcKsfi}d|vr |d|d<d|vr|d|d<d|vr |d|d<|dur.tdt||d<|iifS)Nr$timeouthypothesis_templatez^The `tokenizer_kwargs` argument is deprecated and will be removed in version 5 of Transformerstokenizer_kwargs)warningswarn FutureWarning)rr0rpreprocess_paramsrrr"_sanitize_parameterss    z8ZeroShotImageClassificationPipeline._sanitize_parametersThis is a photo of {}.c s|duri}t||d}|j|g|jd}|jdkr ||j}||d<fdd|D}ddi}d |jjjvr@|jd d dd |||j |fd |ji|} | g|d<|S)N)r.)r+return_tensorsptr$csg|]}|qSr)format).0xr/rr" szBZeroShotImageClassificationPipeline.preprocess..paddingTsiglip max_length@)r>r@ truncationr7 text_inputs) rimage_processorrtodtypemodelconfig model_typeupdate tokenizer) rr#r$r/r.r0inputs sequencestokenizer_default_kwargsrCrr<r" preprocesss     z.ZeroShotImageClassificationPipeline.preprocesscCs\|d}|d}t|dtr|d}n|dd}|jdi||}||jd}|S)Nr$rCr)r$logitsr)r, isinstancerrGlogits_per_image)r model_inputsr$rCoutputs model_outputsrrr"_forwards    z,ZeroShotImageClassificationPipeline._forwardcCs|d}|dd}|jdkr,d|jjjvr,t|d}|}t |t s+|g}n5|jdkrG|j ddd}|}t |t sF|g}n|jdkrYt |dd }| }ntd |jd d tt||d ddD}|S)Nr$rPrr8r?)dimr)axiszUnsupported framework: cSsg|] \}}||dqS))scorelabelr)r:rZcandidate_labelrrr"r=szCZeroShotImageClassificationPipeline.postprocess..cSs |d S)Nrr)r;rrr"s zAZeroShotImageClassificationPipeline.postprocess..)key)r,rrGrHrItorchsigmoidsqueezetolistrQlistsoftmaxrnumpyr-sortedzip)rrUr$rPprobsscoresresultrrr" postprocesss,       z/ZeroShotImageClassificationPipeline.postprocessr')Nr6NN)__name__ __module__ __qualname____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerrrrstrrcrdictr)r5rOrVrk __classcell__rrr r"r!sX      & + r) r1 collectionsrtypingrrrutilsrrr r r r baserrPILr image_utilsrr_models.auto.modeling_autormodels.auto.modeling_tf_autortf_utilsr get_loggerrlloggerrrrrr"s"