o ߥi8@sBdZddlmZmZmZmZddlZeeZ Gddde Z dS)z BERT model configuration )absolute_importdivisionprint_functionunicode_literalsNc@s6eZdZdZ          dd dZdS) BertConfigaU :class:`~transformers.BertConfig` is the configuration class to store the configuration of a `BertModel`. Arguments: vocab_size_or_config_json_file: Vocabulary size of `inputs_ids` in `BertModel`. hidden_size: Size of the encoder layers and the pooler layer. num_hidden_layers: Number of hidden layers in the Transformer encoder. num_attention_heads: Number of attention heads for each attention layer in the Transformer encoder. intermediate_size: The size of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. hidden_act: The non-linear activation function (function or string) in the encoder and pooler. If string, "gelu", "relu", "swish" and "gelu_new" are supported. hidden_dropout_prob: The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. attention_probs_dropout_prob: The dropout ratio for the attention probabilities. max_position_embeddings: The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). type_vocab_size: The vocabulary size of the `token_type_ids` passed into `BertModel`. initializer_range: The stdev of the truncated_normal_initializer for initializing all weight matrices. layer_norm_eps: The epsilon used by LayerNorm. :w gelu皙?{Gz?-q=FcCsX||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ | |_ ||_ dS)N) vocab_size hidden_sizenum_hidden_layersnum_attention_heads hidden_actintermediate_sizehidden_dropout_probattention_probs_dropout_probmax_position_embeddingstype_vocab_sizeinitializer_rangelayer_norm_epsoutput_attentionsoutput_hidden_states)selfvocab_size_or_config_json_filerrrrrrrrrrrrrr!i/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/multi_modal/clip/configuration_bert.py__init__6s zBertConfig.__init__N)rrr r r r r r r rrrFF)__name__ __module__ __qualname____doc__r#r!r!r!r"rs"r) r' __future__rrrrlogging getLoggerr$loggerobjectrr!r!r!r"s