o wi/@spdZddlmZddlmZddlmZmZee Z GdddeZ Gdd d eZ Gd d d eZ d gZd S) zIdefics2 model configuration)PretrainedConfig)logging)CONFIG_MAPPING AutoConfigcs@eZdZdZdZdZ       dfdd ZZS)Idefics2VisionConfiga This is the configuration class to store the configuration of a [`Idefics2VisionModel`]. It is used to instantiate a Idefics2 vision encoder according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the SigLIP checkpoint [google/siglip-base-patch16-224](https://huggingface.co/google/siglip-base-patch16-224) used in the Idefics2 model [HuggingFaceM4/idefics2-8b](https://huggingface.co/HuggingFaceM4/idefics2-8b). Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 768): Dimensionality of the encoder layers and the pooler layer. intermediate_size (`int`, *optional*, defaults to 3072): Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer encoder. num_channels (`int`, *optional*, defaults to 3): Number of channels in the input images. image_size (`int`, *optional*, defaults to 224): The size (resolution) of each image. patch_size (`int`, *optional*, defaults to 32): The size (resolution) of each patch. hidden_act (`str` or `function`, *optional*, defaults to `"gelu_pytorch_tanh"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"selu"` and `"gelu_new"` `"quick_gelu"` are supported. layer_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the layer normalization layers. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation for initializing all weight matrices in the model. Example: ```python >>> from transformers.models.idefics2.modeling_idefics2 import Idefics2VisionTransformer >>> from transformers.models.idefics2.configuration_idefics2 import Idefics2VisionConfig >>> # Initializing a Idefics2VisionConfig with google/siglip-base-patch16-224 style configuration >>> configuration = Idefics2VisionConfig() >>> # Initializing a Idefics2VisionTransformer (with random weights) from the google/siglip-base-patch16-224 style configuration >>> model = Idefics2VisionTransformer(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```idefics2_vision vision_config r gelu_pytorch_tanhư>{Gz?c  sXtjdi| ||_||_||_||_||_||_||_| |_ | |_ ||_ | |_ dS)N) super__init__ hidden_sizeintermediate_sizenum_hidden_layersnum_attention_heads num_channels patch_size image_sizeattention_dropoutlayer_norm_eps hidden_actinitializer_range) selfrrrrrrrrrrr kwargs __class__rp/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/transformers/models/idefics2/configuration_idefics2.pyrOs zIdefics2VisionConfig.__init__) r r r r rr rrrrr)__name__ __module__ __qualname____doc__ model_typebase_config_keyr __classcell__rrr#r%rs 3rcs:eZdZdZdZ       dfd d ZZS)Idefics2PerceiverConfigao Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the perceiver block. hidden_size (`int`, *optional*, defaults to 4096): Dimension of the hidden representations. rms_norm_eps (`float`, *optional*, defaults to 1e-06): The epsilon used by the rms normalization layers. resampler_n_latents (`int`, *optional*, defaults to 64): Number of latent embeddings to resample ("compress") the input sequence to (usually < 128). resampler_depth (`int`, *optional*, defaults to 3): Depth of the Perceiver Resampler (Transformer w/ cross attention). Should be shallow (<= 3). resampler_n_heads (`int`, *optional*, defaults to 16): Number of heads in each Transformer block (for multi-headed self-attention). resampler_head_dim (`int`, *optional*, defaults to 96): Dimensionality of each head projection in the Transformer block. num_key_value_heads (`int`, *optional*, defaults to 4): Number of key-value heads in the perceiver attention block. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation for initializing all weight matrices in the model. idefics2_perceiversilur@r`rrc  sv||_||_||_||_||_||_||_||_| |_| |_ |j|jkr0t d|jd|jt j di| dS)Nznum_key_value_heads=z1 must be less than or equal to resampler_n_heads=r) rr rms_norm_epsresampler_n_latentsresampler_depthresampler_n_headsnum_key_value_headsresampler_head_dimrr ValueErrorrr) r!rrr5r6r7r8r:r9rr r"r#rr%rs"  z Idefics2PerceiverConfig.__init__) r/r0rr1rr2r3r4rr)r&r'r(r)r*rr,rrr#r%r-msr-cs>eZdZdZdZeeedZ      d fdd Z Z S) Idefics2Configa This is the configuration class to store the configuration of a [`Idefics2Model`]. It is used to instantiate a Idefics2 model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the model of the Idefics2 [HuggingFaceM4/idefics2-8b](https://huggingface.co/HuggingFaceM4/idefics2-8b) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should cache the key/value pairs of the attention mechanism. image_token_id (`int`, *optional*, defaults to 32001): The id of the "image" token. tie_word_embeddings (`bool`, *optional*, defaults to `False`): Whether or not to tie the word embeddings with the token embeddings. vision_config (`IdeficsVisionConfig` or `dict`, *optional*): Custom vision config or dict perceiver_config (`IdeficsPerceiverConfig` or `dict`, *optional*): Custom perceiver config or dict text_config (`MistralConfig` or `dict`, *optional*): Custom text config or dict for the text model Example: ```python >>> from transformers import Idefics2Model, Idefics2Config >>> # Initializing configuration >>> configuration = Idefics2Config() >>> # Initializing a model from the configuration >>> model = Idefics2Model(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```idefics2) text_configperceiver_configr T}FNc sX||_||_||_|durt|_tdnt|tr%td i||_nt|tr-||_|dur;t |_ tdnt|trIt d i||_ nt|t rQ||_ t|trnd|vr^|dnd|d<t |dd i|}n|durtdt ddddd d }||_ |j j |jj kr|j j |j_ |j j|j_td tjd i|d |idS)Nz7perciver_config is None, using default perceiver configz2vision_config is None, using default vision configr*mistralz.text_config is None, using default text configigh㈵>F)max_position_embeddingsr5 pad_token_idtie_word_embeddingszPerceiver config has a different `hidden_size` than text config, which means default values were used. In your model's config on the hub, add `hidden_size` and `rms_norm_eps` keys under the `perceiver_config` dict. rEr)image_token_id use_cacherEr-r?loggerinfo isinstancedictrr rr>rr5 warning_oncerr)r!rGrFrEr r?r>r"r#rr%rsH           zIdefics2Config.__init__)Tr@FNNN) r&r'r(r)r*rr-r sub_configsrr,rrr#r%r<s"r<N)r)configuration_utilsrutilsrautorr get_loggerr&rHrr-r<__all__rrrr%s   U> a