o wi @sddlZddlmZmZddlZddlmZddlm Z m Z ddl m Z m Z mZmZmZddlmZmZddlmZGd d d e ZGd d d ed dZGdddeZdgZdS)N)OptionalUnion) BatchFeature) ImageInputmake_nested_list_of_images) ImagesKwargsMultiModalDataProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput) to_py_objc@sJeZdZUeeed<eeed<eeed<eeed<eeed<dS)Gemma3ImagesKwargsdo_pan_and_scanpan_and_scan_min_crop_sizepan_and_scan_max_num_crops"pan_and_scan_min_ratio_to_activatedo_convert_rgbN)__name__ __module__ __qualname__rbool__annotations__intfloatrri/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/transformers/models/gemma3/processing_gemma3.pyrs    rc@s0eZdZUeed<ddddddddd Zd S) Gemma3ProcessorKwargs images_kwargsFT)paddingreturn_mm_token_type_idsg333333?)rrrr) text_kwargsr N)rrrrr _defaultsrrrrr$s  rF)totalc seZdZddgZdZdZ  ddeffdd Z    dd ed e e e e e e e fd e ed efddZdddZddZddZeddZZS)Gemma3Processorimage_processor tokenizerAutoImageProcessor AutoTokenizerNr#image_seq_lengthc sh||_|j|_|j|_|j|_d|jg|}d|j||jd|_tjd|||d|dS)Nz )r)r* chat_templater) r-image_token_id boi_token image_tokenjoin eoi_tokenfull_image_sequencesuper__init__)selfr)r*r/r-kwargsimage_tokens_expanded __class__rrr79s zGemma3Processor.__init__imagestextr9returnc sn|dur |dur tdjtfdjji|}t|tr"|g}nt|ts2t|dts2tdi}|durt|}j |fi|d}|sRfdd|D}t |t |kritdt |d t |d t | d fd d|D} t t||| D]a\} \} }d dtj| D} t |t | krtdt | dt |dttt| D]-\} }| rdjddjg| }| d||| |t jd} | || <qqfdd|D}|d dd}|d dd}jd d|i|d}j||dgd|r-t|d}t|}d||jk<||d<ti|||dS)!Nz+Provide at least one of `text` or `images`.tokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsr cs"g|] }djgt|qS) )r3r1len.0r=r8rr ls"z,Gemma3Processor.__call__..z1Received inconsistently sized batches of images (z ) and text (z). num_cropscs&g|]}fddtt|DqS)csg|]}dqS)r)pop)rD_rGrrrFusz7Gemma3Processor.__call__...)rangerBrCrJrrrFus&cSsg|]}|qSr)start)rDmrrrrFwszPrompt contained z image tokens but received z images.zHere is the original image z0 and here are some crops to help you see better rAcsg|] }|jjqSr)replacer1r5)rDpromptrErrrFsr%return_tensorsr"Fr>image) modalities input_idstoken_type_ids)data tensor_typer) ValueError _merge_kwargsrr* init_kwargs isinstancestrlistrr)rBrrH enumerateziprefinditerr1reversedr3_check_special_mm_tokensnparray zeros_liker0tolistr)r8r=r>videosaudior9 output_kwargs image_inputsbatched_imagesbatch_num_crops batch_idxrO image_indexesnumidxformatted_image_textrPr" text_inputs array_idsmm_token_type_idsr)rGr8r__call__Osh  &   zGemma3Processor.__call__cKsHi}|dur|jgt|}dgt|}|||dtdi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. NrT)num_image_tokensnum_image_patchesr)r-rBupdater )r8 image_sizesr9 vision_datarwrxrrr_get_num_multimodal_tokenss  z*Gemma3Processor._get_num_multimodal_tokenscO|jj|i|S)z This method forwards all its arguments to GemmaTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r* batch_decoder8argsr9rrrr~zGemma3Processor.batch_decodecOr})z This method forwards all its arguments to GemmaTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )r*decoderrrrrrzGemma3Processor.decodecCs(|jjdg}|jj}tt||S)NrU)r*model_input_namesr)r]dictfromkeys)r8tokenizer_input_namesimage_processor_input_namesrrrrsz!Gemma3Processor.model_input_names)Nr#)NNNN)N)rrr attributesimage_processor_classtokenizer_classrr7rrrr r]r rrrvr|r~rpropertyr __classcell__rrr;rr(4s6 Jr()r`typingrrnumpyrdfeature_extraction_utilsr image_utilsrrprocessing_utilsrr r r r tokenization_utils_baser rutilsrrrr(__all__rrrrs