o 8wiq@s\ddlZddlmZmZddlmZmZddlmZddl m Z m Z eGdddeZ dS)N) dataclassfield)LiteralOptional)PromptLearningConfig)PeftTypeTaskTypec@seZdZUdZedddidZeeee d<edddidZ eeee d<eddd idZ eeee d <ed dd idZ ee d e d<edddidZeee d<edddidZeee d<edddidZeee d<edddidZeee d<dZddZdS) CPTConfigaa CPT Configuration class extending PeftConfig for Context-aware Prompt Tuning (CPT). This class introduces additional parameters required for CPT, such as: - Token type masks - Prompt tuning initialization - Loss weighting - Projection settings For more details, see the paper: https://huggingface.co/papers/2410.17222 Nhelpz)Tensor of token IDs used for CPT prompts.)defaultmetadata cpt_token_idsz"Tensor mask applied to CPT tokens.cpt_maskz+Mask indicating the type of each CPT token.cpt_tokens_type_masknonez)Type of weighted loss: 'none' or 'decay'.)rdecayopt_weighted_loss_typeg?z/Factor for exponential decay in loss weighting.opt_loss_decay_factorg?z#Epsilon value for input projection.opt_projection_epsilonz$Epsilon value for format projection.opt_projection_format_epsilonz`The tokenizer to use for prompt tuning initialization. Only used if prompt_tuning_init is `TEXT`tokenizer_name_or_pathTcCsd|_d|_d|_d|_d|_tj|_|jt j kr-t |j jdt j jdtt j |_|jdur6dg|_t|j|_|jdurJdd|jD|_|jdurXd d|jD|_t|jt|jkrtt|jkrt|jksytd td dS) zh Post-initialization hook to set additional attributes after the config is initialized. TNz only supports task_type = zP, setting it automatically. This will raise an error starting from PEFT v0.18.0.rcSg|]}dqSr.0_rrS/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/peft/tuners/cpt/config.py bz+CPTConfig.__post_init__..cSrrrrrrrrer zKcpt_token_ids, cpt_mask and cpt_tokens_type_mask must have the same length.)is_prompt_learning num_layers token_dimnum_attention_headsnum_transformer_submodulesrCPT peft_type task_typer CAUSAL_LMwarningswarn __class____name__value FutureWarningr lennum_virtual_tokensrr ValueError)selfrrr __post_init__Hs0     0zCPTConfig.__post_init__)r- __module__ __qualname____doc__rr rlistint__annotations__rrrrrfloatrrrstrr!r4rrrrr s8    r ) r* dataclassesrrtypingrr peft.configr peft.utilsrrr rrrrs