o ei;@sdZddlmZddlmZddlmZmZmZm Z ddl m Z m Z ddl mZmZeeZdZGdd d ed d ZGd d d ed d ZeGdddeZdgZdS)z Processor class for Janus. ) BatchFeature) ImageInput)ProcessingKwargsProcessorMixin TextKwargsUnpack)PreTokenizedInput TextInput)auto_docstringloggingzYou are a helpful language and vision assistant. You are able to understand the visual content that the user provides, and assist the user with a variety of tasks using natural language. c@seZdZUdZeed<dS)JanusTextKwargsas generation_mode (`str`, *optional*, defaults to `"text"`): The generation mode indicating which modality to generate. Can be one of `"text"` or `"image"`. When set to `"text"`, the processor prepares inputs for text generation. When set to `"image"`, it prepares inputs for image generation by appending image start tokens to the prompt. generation_modeN)__name__ __module__ __qualname____doc__str__annotations__rrh/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/janus/processing_janus.pyr "s  r F)totalc@s*eZdZUeed<dddddidZdS) JanusProcessorKwargs text_kwargsFtext)paddingr return_tensorspt)r common_kwargsN)rrrr r _defaultsrrrrr-s  rc sxeZdZdfdd Ze  ddeeBeeBeeBdedBde e de fd d Z defd d Z dddZZS)JanusProcessorNFc s:d|_|j|_|j|_|j|_||_tj|||ddS)z use_default_system_prompt (`bool`, *optional*, defaults to `False`): Use default system prompt for Text Generation. i@) chat_templateN) num_image_tokens image_token boi_tokenimage_start_token eoi_tokenimage_end_tokenuse_default_system_promptsuper__init__)selfimage_processor tokenizerr r'kwargs __class__rrr)7s zJanusProcessor.__init__rimagesr-returnc Ks8|jtfd|jji|}|dur|durtd|dur:t|tr&|g}nt|ttfr6t dd|Ds:td|d d}g}|j |j |j |j}|D]"}||j |}|jre|d kret|}|d krn||j 7}||qQ|j|fi|d} |dur|d kr|jdd |i|d d | d <t| dS)aA Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. tokenizer_init_kwargsNz'You must specify either text or images.css|]}t|tVqdS)N) isinstancer).0trrr `sz*JanusProcessor.__call__..zAInvalid input text. Please provide a string, or a list of stringsrr rimager0 images_kwargs pixel_values)datar) _merge_kwargsrr, init_kwargs ValueErrorr3rlisttupleallpopr$r"r!r&replacer'DEFAULT_SYSTEM_PROMPTappendr+r) r*rr0r- output_kwargsr prompt_stringsone_img_tokenspromptr:rrr__call__Ds<     zJanusProcessor.__call__cKs|jj|fi|S)z Forwards all arguments to the image processor's `postprocess` method. Refer to the original method's docstring for more details. )r+ postprocess)r*r0r-rrrrJzszJanusProcessor.postprocessTcKsh|dus|dkr|j|fd|i|S|dkr(t|}|j|dd}|dSt|jjd|d ) a  Post-process the output of a multimodal model to return the requested modality output. If the model cannot generated the requested modality, an error will be raised. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. generation_mode (`str`, *optional*): Generation mode indicated which modality to output and can be one of `["text", "image", "audio"]`. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `list[Union[str, PIL.Image.Image]]`: The decoded text or generated image. Nrskip_special_tokensr7zPIL.Image.Image)rr9z# got an unexpected generation_mode=z.. Supported options are only `text` and `image)post_process_image_text_to_textr>floatrJr=r/r)r*generated_outputsrKr r-r0rrrpost_process_multimodal_outputs z-JanusProcessor.post_process_multimodal_output)NF)NN)TN)rrrr)r r rr>rrrrrIrJrO __classcell__rrr.rr5s"  5rN)rfeature_extraction_utilsr image_utilsrprocessing_utilsrrrrtokenization_utils_baserr utilsr r get_loggerrloggerrCr rr__all__rrrrs     p