o ¾e¦i¥>ã@sndZddlmZddlmZddlmZe e¡Z Gdd„deƒZ Gdd„deƒZ Gd d „d eƒZ gd ¢Z d S) zDia model configurationé)ÚPreTrainedConfig)ÚRopeParameters)ÚloggingcsteZdZdZdZ         dd ededededededededededed Bdef‡fdd„ Z ‡Z S)ÚDiaEncoderConfiga× This is the configuration class to store the configuration of a [`DiaEncoder`]. It is used to instantiate a Dia encoder according to the specified arguments, defining the encoder architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: max_position_embeddings (`int`, *optional*, defaults to 1024): The maximum sequence length that this model might ever be used with. num_hidden_layers (`int`, *optional*, defaults to 12): Number of hidden layers in the Transformer encoder. hidden_size (`int`, *optional*, defaults to 1024): Dimensionality of the encoder layers and the pooler layer. num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer encoder. num_key_value_heads (`int`, *optional*, defaults to 16): Number of key and value heads for each attention layer in the Transformer encoder. head_dim (`int`, *optional*, defaults to 128): Dimensionality of the attention head. intermediate_size (`int`, *optional*, defaults to 4096): Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer encoder. norm_eps (`float`, *optional*, defaults to 1e-05): The epsilon used by the normalization layers. vocab_size (`int`, *optional*, defaults to 256): Vocabulary size of the Dia model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`DiaModel`]. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"swish"` and `"gelu_new"` are supported. rope_parameters (`RopeParameters`, *optional*): Dictionary containing the configuration parameters for the RoPE embeddings. The dictionary should contain a value for `rope_theta` and optionally parameters used for scaling in case you want to use RoPE with longer `max_position_embeddings`. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. Ú dia_encoderéé éé€éçñh㈵øä>éÚsiluNç{®Gáz”?Úmax_position_embeddingsÚnum_hidden_layersÚ hidden_sizeÚnum_attention_headsÚnum_key_value_headsÚhead_dimÚintermediate_sizeÚnorm_epsÚ vocab_sizeÚ hidden_actÚrope_parametersÚinitializer_rangec  s^||_||_||_||_||_||_||_| |_||_| |_ | |_ | |_ t ƒj di| ¤ŽdS)N©)rrrrrrrrrrrrÚsuperÚ__init__)ÚselfrrrrrrrrrrrrÚkwargs©Ú __class__rúg/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/dia/configuration_dia.pyrAszDiaEncoderConfig.__init__) rrrr r r r r r rNr) Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú model_typeÚintÚfloatÚstrrrÚ __classcell__rrr!r#rsP&óþýüûúùø ÷ ö õ ô órc-s°eZdZdZdZ               d,dedededededededededededed ed!ed"ed#edBd$ed%e d&e d'ed(ed)ef,‡fd*d+„ Z ‡Z S)-ÚDiaDecoderConfiga• This is the configuration class to store the configuration of a [`DiaDecoder`]. It is used to instantiate a Dia decoder according to the specified arguments, defining the decoder architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: max_position_embeddings (`int`, *optional*, defaults to 3072): The maximum sequence length that this model might ever be used with. num_hidden_layers (`int`, *optional*, defaults to 18): Number of hidden layers in the Transformer decoder. hidden_size (`int`, *optional*, defaults to 2048): Dimensionality of the decoder layers and the pooler layer. intermediate_size (`int`, *optional*, defaults to 8192): Dimensionality of the "intermediate" (often named feed-forward) layer in the Transformer decoder. num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer decoder. num_key_value_heads (`int`, *optional*, defaults to 4): Number of key and value heads for each attention layer in the Transformer decoder. head_dim (`int`, *optional*, defaults to 128): Dimensionality of the attention head. cross_num_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each cross-attention layer in the Transformer decoder. cross_head_dim (`int`, *optional*, defaults to 128): Dimensionality of the cross-attention head. cross_num_key_value_heads (`int`, *optional*, defaults to 16): Number of key and value heads for each cross-attention layer in the Transformer decoder. cross_hidden_size (`int`, *optional*, defaults to 1024): Dimensionality of the cross-attention layers. norm_eps (`float`, *optional*, defaults to 1e-05): The epsilon used by the normalization layers. vocab_size (`int`, *optional*, defaults to 1028): Vocabulary size of the Dia model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`DiaModel`]. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the decoder. If string, `"gelu"`, `"relu"`, `"swish"` and `"gelu_new"` are supported. num_channels (`int`, *optional*, defaults to 9): Number of channels for the Dia decoder. rope_parameters (`RopeParameters`, *optional*): Dictionary containing the configuration parameters for the RoPE embeddings. The dictionary should contain a value for `rope_theta` and optionally parameters used for scaling in case you want to use RoPE with longer `max_position_embeddings`. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). is_encoder_decoder (`bool`, *optional*, defaults to `True`): Indicating that this model is part of an encoder-decoder architecture. pad_token_id (`int`, *optional*, defaults to 1025): The token id used for padding sequences to the same length within a batch. eos_token_id (`int`, *optional*, defaults to 1024): The token id representing the end-of-sequence token, indicating that generation should stop. bos_token_id (`int`, *optional*, defaults to 1026): The token id representing the beginning-of-sequence token, used to initialize decoding. Ú dia_decoderé ééé r ér rr éré NrTéérrrrrrrÚcross_num_attention_headsÚcross_head_dimÚcross_num_key_value_headsÚcross_hidden_sizerrrÚ num_channelsrrÚ use_cacheÚis_encoder_decoderÚ pad_token_idÚ eos_token_idÚ bos_token_idc s˜||_||_||_||_||_||_||_| |_||_| |_ | |_ | |_ | |_ ||_ ||_||_||_||_||_||_||_tƒjdd|i|¤ŽdS)Nr>r)rrrrrrrr:r8r9r;rrrr<rr=rr?r@rArr)rrrrrrrrr8r9r:r;rrrr<rrr=r>r?r@rAr r!rr#ržs,zDiaDecoderConfig.__init__)r/r0r1r2r r3r r r r rr r4rr5NrTTr6rr7) r$r%r&r'r(r)r*r+rÚboolrr,rrr!r#r-asŒ:éþýüûúùø ÷ ö õ ô óòñðïîíìëêér-cs˜eZdZdZdZdgZeedœZ          dd edBd edBd e d e d e dBde dBde dBde e dBde de f‡fdd„ Z dd„Z‡ZS)Ú DiaConfigav This is the configuration class to store the configuration of a [`DiaModel`]. It is used to instantiate a Dia model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the [nari-labs/Dia-1.6B](https://huggingface.co/nari-labs/Dia-1.6B) architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: encoder_config (`DiaEncoderConfig`, *optional*): Configuration for the encoder part of the model. If not provided, a default `DiaEncoderConfig` will be used. decoder_config (`DiaDecoderConfig`, *optional*): Configuration for the decoder part of the model. If not provided, a default `DiaDecoderConfig` will be used. norm_eps (`float`, *optional*, defaults to 1e-05): The epsilon used by the normalization layers. is_encoder_decoder (`bool`, *optional*, defaults to `True`): Indicating that this model uses an encoder-decoder architecture. pad_token_id (`int`, *optional*): Deprecated. Please set this on `DiaDecoderConfig` directly. If provided, it will be forwarded to `decoder_config`. eos_token_id (`int`, *optional*): Deprecated. Please set this on `DiaDecoderConfig` directly. If provided, it will be forwarded to `decoder_config`. bos_token_id (`int`, *optional*): Deprecated. Please set this on `DiaDecoderConfig` directly. If provided, it will be forwarded to `decoder_config`. delay_pattern (`list[int]`, *optional*, defaults to `[0, 8, 9, 10, 11, 12, 13, 14, 15]`): The delay pattern for the decoder. The length of this list must match `decoder_config.num_channels`. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). Example: ```python >>> from transformers import DiaConfig, DiaModel >>> # Initializing a DiaConfig with default values >>> configuration = DiaConfig() >>> # Initializing a DiaModel (with random weights) from the configuration >>> model = DiaModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` ÚdiaÚpast_key_values)Úencoder_configÚdecoder_configNr TrrFrGrr>r?r@rAÚ delay_patternrr=c  st|tƒr tdi|¤Ž}t|tƒrtdi|¤Ž}|dur|ntƒ|_|dur(|ntƒ|_||_|dur5|ngd¢|_| |_| |_ |durMt   d¡||j_ |durZt   d¡||j_ |durgt   d¡||j_|jjt|jƒkstJdƒ‚tƒjdd|i| ¤ŽdS)N) éér5é é ré éézjPassing `pad_token_id` to `DiaConfig` is deprecated. Please set it directly on `DiaDecoderConfig` instead.zjPassing `eos_token_id` to `DiaConfig` is deprecated. Please set it directly on `DiaDecoderConfig` instead.zjPassing `bos_token_id` to `DiaConfig` is deprecated. Please set it directly on `DiaDecoderConfig` instead.z3Number of channels must match delay pattern length.r>r)Ú isinstanceÚdictrr-rFrGrrHrr=ÚloggerÚ warning_oncer?r@rAr<Úlenrr) rrFrGrr>r?r@rArHrr=r r!rr#rs:  ÿÿÿÿzDiaConfig.__init__cOs|jS)z^Defaulting to audio config as it's the decoder in this case which is usually the text backbone)rG)rÚargsr rrr#Úget_text_config<szDiaConfig.get_text_config) NNr TNNNNrT)r$r%r&r'r(Úkeys_to_ignore_at_inferencerr-Ú sub_configsr*rBr)ÚlistrrVr,rrr!r#rCÑsJ2 õþýüûúùø ÷ ö õ4rC)rCrr-N)r'Úconfiguration_utilsrÚmodeling_rope_utilsrÚutilsrÚ get_loggerr$rRrr-rCÚ__all__rrrr#Ús    Ip p