o ¾e¦i÷'ã@sRdZddlmZddlmZe e¡ZGdd„deƒZGdd„deƒZ ddgZ dS) zParakeet model configuration.é)ÚPreTrainedConfig)ÚloggingcsVeZdZdZdZdgZ              d‡fdd„ Z‡ZS)ÚParakeetEncoderConfiga¢ This is the configuration class to store the configuration of a [`ParakeetEncoder`]. It is used to instantiate a `ParakeetEncoder` model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: hidden_size (`int`, *optional*, defaults to 1024): Dimension of the layers and the hidden states. num_hidden_layers (`int`, *optional*, defaults to 24): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 8): Number of attention heads for each attention layer in the Transformer encoder. intermediate_size (`int`, *optional*, defaults to 4096): Dimension of the "intermediate" (often named feed-forward) layer in the Transformer encoder. hidden_act (`str` or `function`, *optional*, defaults to `"silu"`): The non-linear activation function (function or string) in the encoder and pooler. attention_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in the attention layers. convolution_bias (`bool`, *optional*, defaults to `True`): Whether to use bias in convolutions of the conformer's convolution module. conv_kernel_size (`int`, *optional*, defaults to 9): The kernel size of the convolution layers in the Conformer block. subsampling_factor (`int`, *optional*, defaults to 8): The factor by which the input sequence is subsampled. subsampling_conv_channels (`int`, *optional*, defaults to 256): The number of channels in the subsampling convolution layers. num_mel_bins (`int`, *optional*, defaults to 80): Number of mel features. subsampling_conv_kernel_size (`int`, *optional*, defaults to 3): The kernel size of the subsampling convolution layers. subsampling_conv_stride (`int`, *optional*, defaults to 2): The stride of the subsampling convolution layers. dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for all fully connected layers in the embeddings, encoder, and pooler. dropout_positions (`float`, *optional*, defaults to 0.0): The dropout ratio for the positions in the input sequence. layerdrop (`float`, *optional*, defaults to 0.1): The dropout ratio for the layers in the encoder. activation_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for activations inside the fully connected layer. attention_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention layers. max_position_embeddings (`int`, *optional*, defaults to 5000): The maximum sequence length that this model might ever be used with. scale_input (`bool`, *optional*, defaults to `True`): Whether to scale the input embeddings. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. Example: ```python >>> from transformers import ParakeetEncoderModel, ParakeetEncoderConfig >>> # Initializing a `ParakeetEncoder` configuration >>> configuration = ParakeetEncoderConfig() >>> # Initializing a model from the configuration >>> model = ParakeetEncoderModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the ParakeetEncoder architecture from NVIDIA NeMo. You can find more details and pre-trained models at [nvidia/parakeet-ctc-1.1b](https://huggingface.co/nvidia/parakeet-ctc-1.1b). Úparakeet_encoderÚpast_key_valuesééééÚsiluTé ééPréçš™™™™™¹?çéˆç{®Gáz”?c sš||_||_||_||_||_||_||_||_||_| |_ | |_ | |_ | |_ | |_ ||_||_||_||_||_||_||_||_tƒjdi|¤ŽdS©N©)Ú hidden_sizeÚnum_hidden_layersÚnum_attention_headsÚnum_key_value_headsÚintermediate_sizeÚ hidden_actÚattention_biasÚconvolution_biasÚconv_kernel_sizeÚsubsampling_conv_kernel_sizeÚsubsampling_conv_strideÚsubsampling_factorÚsubsampling_conv_channelsÚ num_mel_binsÚdropoutÚdropout_positionsÚ layerdropÚactivation_dropoutÚattention_dropoutÚmax_position_embeddingsÚ scale_inputÚinitializer_rangeÚsuperÚ__init__)Úselfrrrrrrrrr!r"r#rr r$r%r&r'r(r)r*r+Úkwargs©Ú __class__rúq/home/ubuntu/transcripts/venv/lib/python3.10/site-packages/transformers/models/parakeet/configuration_parakeet.pyr-`s2  ÿzParakeetEncoderConfig.__init__)rrr r r TTr r r rrrrrrrrrTr)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú model_typeÚkeys_to_ignore_at_inferencer-Ú __classcell__rrr0r2rs4EêrcsTeZdZdZdZdeiZ     d deeBf‡fd d „ Ze defd d „ƒZ ‡Z S)ÚParakeetCTCConfigaã This is the configuration class to store the configuration of a [`ParakeetForCTC`]. It is used to instantiate a Parakeet CTC model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PreTrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PreTrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 1025): Vocabulary size of the model. ctc_loss_reduction (`str`, *optional*, defaults to `"mean"`): Specifies the reduction to apply to the output of `torch.nn.CTCLoss`. Only relevant when training an instance of [`ParakeetForCTC`]. ctc_zero_infinity (`bool`, *optional*, defaults to `True`): Whether to zero infinite losses and the associated gradients of `torch.nn.CTCLoss`. Infinite losses mainly occur when the inputs are too short to be aligned to the targets. Only relevant when training an instance of [`ParakeetForCTC`]. encoder_config (`Union[dict, ParakeetEncoderConfig]`, *optional*): The config object or dictionary of the encoder. pad_token_id (`int`, *optional*, defaults to 1024): Padding token id. Also used as blank token id. Example: ```python >>> from transformers import ParakeetForCTC, ParakeetCTCConfig >>> # Initializing a Parakeet configuration >>> configuration = ParakeetCTCConfig() >>> # Initializing a model from the configuration >>> model = ParakeetForCTC(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` This configuration class is based on the Parakeet CTC architecture from NVIDIA NeMo. You can find more details and pre-trained models at [nvidia/parakeet-ctc-1.1b](https://huggingface.co/nvidia/parakeet-ctc-1.1b). Ú parakeet_ctcÚencoder_configéÚmeanTNrc sl||_||_||_t|tƒrtdi|¤Ž|_n|durtƒ|_|j|_|jj|_||_t ƒj di|¤ŽdSr) Ú vocab_sizeÚctc_loss_reductionÚctc_zero_infinityÚ isinstanceÚdictrr<r+Ú pad_token_idr,r-)r.r?r@rAr<rDr/r0rr2r-Äs   zParakeetCTCConfig.__init__cKs|dd| ¡i|¤ŽS)zÐ Instantiate a [`ParakeetCTCConfig`] (or a derived class) from parakeet encoder model configuration. Returns: [`ParakeetCTCConfig`]: An instance of a configuration object r<Nr)Úto_dict)Úclsr<r/rrr2Úfrom_encoder_configÜs z%ParakeetCTCConfig.from_encoder_config)r=r>TNr) r3r4r5r6r7rÚ sub_configsrCr-Ú classmethodrGr9rrr0r2r:˜s(úûr:N) r6Úconfiguration_utilsrÚutilsrÚ get_loggerr3Úloggerrr:Ú__all__rrrr2Ús    P