o Ni@sTddlmZmZddlmZmZddlmZddlm Z m Z eGdddeZ dS)) dataclassfield)LiteralOptional)PromptLearningConfig)PeftTypeTaskTypec@seZdZUdZedddidZeeee d<edddidZ eeee d<eddd idZ eeee d <ed dd idZ ee d e d<edddidZeee d<edddidZeee d<edddidZeee d<edddidZeee d<dZddZdS) CPTConfigaa CPT Configuration class extending PeftConfig for Context-aware Prompt Tuning (CPT). This class introduces additional parameters required for CPT, such as: - Token type masks - Prompt tuning initialization - Loss weighting - Projection settings For more details, see the paper: https://huggingface.co/papers/2410.17222 Nhelpz)Tensor of token IDs used for CPT prompts.)defaultmetadata cpt_token_idsz"Tensor mask applied to CPT tokens.cpt_maskz+Mask indicating the type of each CPT token.cpt_tokens_type_masknonez)Type of weighted loss: 'none' or 'decay'.)rdecayopt_weighted_loss_typeg?z/Factor for exponential decay in loss weighting.opt_loss_decay_factorg?z#Epsilon value for input projection.opt_projection_epsilonz$Epsilon value for format projection.opt_projection_format_epsilonz`The tokenizer to use for prompt tuning initialization. Only used if prompt_tuning_init is `TEXT`tokenizer_name_or_pathTcCsd|_d|_d|_d|_d|_tj|_|jt j kr't |j j dt j jd|jdur0dg|_t|j|_|jdurDdd|jD|_|jdurRd d|jD|_t|jt|jkrnt|jkrn|jksst d t d dS) zh Post-initialization hook to set additional attributes after the config is initialized. TNz only supports task_type = .rcSg|]}dqSr.0_rrJ/home/ubuntu/.local/lib/python3.10/site-packages/peft/tuners/cpt/config.py [z+CPTConfig.__post_init__..cSrrrrrrrr ^r!zKcpt_token_ids, cpt_mask and cpt_tokens_type_mask must have the same length.)is_prompt_learning num_layers token_dimnum_attention_headsnum_transformer_submodulesrCPT peft_type task_typer CAUSAL_LM ValueError __class____name__valuer lennum_virtual_tokensrr)selfrrr __post_init__Gs(     0zCPTConfig.__post_init__)r- __module__ __qualname____doc__rr rlistint__annotations__rrrrrfloatrrrstrr"r2rrrrr s8    r N) dataclassesrrtypingrr peft.configr peft.utilsrrr rrrrs