o ߥµi3ã@s8dZddlmZddlmZe ¡ZGdd„deƒZdS)z ChatGLM model configuration é)ÚPretrainedConfig)ÚloggercsHeZdZdZdZ           d‡fdd„ Z‡ZS)Ú ChatGLMConfigaÎ This is the configuration class to store the configuration of a [`~ChatGLMModel`]. It is used to instantiate an ChatGLM model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the ChatGLM-6B [THUDM/ChatGLM-6B](https://huggingface.co/THUDM/chatglm-6b) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 150528): Vocabulary size of the ChatGLM-6B model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`~ChatGLMModel`] or [`~TFChatGLMModel`]. hidden_size (`int`, *optional*, defaults to 4096): Dimension of the encoder layers and the pooler layer. num_hidden_layers (`int`, *optional*, defaults to 28): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 32): Number of attention heads for each attention layer in the Transformer encoder. inner_hidden_size (`int`, *optional*, defaults to 16384): Dimension of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder. max_sequence_length (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). layernorm_epsilon (`float`, *optional*, defaults to 1e-5): The epsilon used by the layer normalization layers. use_cache (`bool`, *optional*, defaults to `True`): Whether the model should return the last key/values attentions (not used by all models). Example: ```python >>> from modelscope.models.nlp.chatglm.configuration import ChatGLMConfig >>> from modelscope.models.nlp.chatglm.text_generation import ChatGLMModel >>> # Initializing a ChatGLM-6B THUDM/ChatGLM-6B style configuration >>> configuration = ChatGLMConfig() >>> # Initializing a model from the THUDM/ChatGLM-6B style configuration >>> model = ChatGLMModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` ÚchatglméLééé çñh㈵øä>FéôIéõIéðIéñIréé@TNc s„||_||_||_||_| |_||_| |_||_||_||_ | |_ | |_ | |_ ||_ ||_||_||_tƒjd| ||dœ|¤ŽdS)N)Ú pad_token_idÚ bos_token_idÚ eos_token_id©)Ú num_layersÚ vocab_sizeÚ hidden_sizeÚnum_attention_headsÚmax_sequence_lengthÚlayernorm_epsilonÚinner_hidden_sizeÚ use_cacherrrÚ mask_token_idÚgmask_token_idÚposition_encoding_2dÚquantization_bitÚ pre_seq_lenÚprefix_projectionÚsuperÚ__init__)Úselfrrrrrrrrrrrrrrr r!r"Úkwargs©Ú __class__rú_/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/nlp/chatglm/configuration.pyr$=s0ý üzChatGLMConfig.__init__)rrrr r Fr r r rrrrTrNF)Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú model_typer$Ú __classcell__rrr'r)r s*0ïrN)r-Ú transformers.configuration_utilsrÚmodelscope.utilsrÚloggingÚ get_loggerrrrrr)Ús