o ei@sdZddlZddlmZddlmZddlmZm Z m Z m Z m Z ddl mZmZddlmZGd d d e d d ZGd dde d d ZeGddde ZdgZdS)z Processor class for Chameleon. N) BatchFeature) ImageInput)MultiModalDataProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringc@seZdZUdZeed<dS)ChameleonTextKwargsa return_for_text_completion (`bool`, *optional*, defaults to `False`): Whether the processed text is intended for text completion tasks. When `True`, the processor does not append the separator token (`sep_token`) to the end of the prompt, which is typically used for chat mode. When `False`, the separator token is appended for proper chat formatting. return_for_text_completionN)__name__ __module__ __qualname____doc__bool__annotations__rrp/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/chameleon/processing_chameleon.pyr !s  r F)totalc@s,eZdZUeed<ddddddidZdS)ChameleonProcessorKwargs text_kwargsF)paddingrreturn_mm_token_type_idsreturn_tensorspt)r common_kwargsN)rrrr r _defaultsrrrrr,s  rc sveZdZddedeffdd Ze  ddedBd ee Be eBe e BdBd e e d e fd d ZdddZZS)ChameleonProcessorimage_seq_length image_tokencs||_t|dr |jn||_||j|_t|dr|jnd|_t|dr(|jnd|_||j|_||j|_ ||j|_ |j|j |j g|_ t ||dS)a image_seq_length (`int`, *optional*, defaults to 1024): Sequence length of one image embedding. image_token (`str`, *optional*, defaults to `""`): The special token used to indicate image in the text. r$ boi_tokenz eoi_tokenzN)r#hasattrr$convert_tokens_to_idsimage_token_idr%image_start_tokenr&image_end_tokenimage_start_token_idimage_end_token_id image_idssuper__init__)selfimage_processor tokenizerr#r$ __class__rrr0<szChameleonProcessor.__init__NimagestextkwargsreturncKst|tr |g}nt|tst|dtstd|dur%|dur%td|jtfd|jji|}|d dd}g}|j |j |j |j }|D]}||j |}|s[||jj7}||qJi} |durr|j|fi|d } |d d d} |d d d} |j|fi|dd di} |j|| d gd | rt| d} t| d}d|t| |j<|| d<ti| | | dS)aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. rzAInvalid input text. Please provide a string, or a list of stringsNz&You must provide either text or imagestokenizer_init_kwargsrrF images_kwargsrrimage) modalities input_idsmm_token_type_ids)data tensor_type) isinstancestrlist TypeError ValueError _merge_kwargsrr3 init_kwargspopr*r$r#r+replace sep_tokenappendr2_check_special_mm_tokensnparray zeros_likeisinr.tolistr)r1r6r7r8 output_kwargsrprompt_stringsone_img_tokenssample image_inputsrr text_inputs array_idsr@rrr__call__QsD    zChameleonProcessor.__call__cKsLi}|dur|jdgt|}dgt|}|||dtdi|S)a Computes the number of placeholder tokens needed for multimodal inputs with the given sizes. Args: image_sizes (`list[list[int]]`, *optional*): The input sizes formatted as (height, width) per each image. Returns: `MultiModalData`: A `MultiModalData` object holding number of tokens per each of the provided input modalities, along with other useful data. Nr?)num_image_tokensnum_image_patchesr)r#lenupdater)r1 image_sizesr8 vision_datar]r^rrr_get_num_multimodal_tokenss  z-ChameleonProcessor._get_num_multimodal_tokens)r!r")NN)N)rrrintrDr0r rr r rEr rrr[rc __classcell__rrr4rr :s 9r )rnumpyrOfeature_extraction_utilsr image_utilsrprocessing_utilsrrrrr tokenization_utils_baser r utilsr r rr __all__rrrrs      h