o Ni@s`ddlmZddlmZmZddlmZmZmZddl m Z ddl m Z eGddde Z dS) ) annotations) dataclassfield)LiteralOptionalUnion) PeftConfig)PeftTypecseZdZUdZedddidZded<eddd idZd ed <eddd idZd ed <edddidZ ded<edddidZ ded<edddidZ ded<edddidZ ded<edddidZ ded<edddidZded <fd!d"ZZS)# PolyConfiga This is the configuration class to store the configuration of a [`PolyModel`]. - [Polytropon (Poly)](https://huggingface.co/papers/2202.13914) - [Multi-Head Routing (MHR)](https://huggingface.co/papers/2211.03831) Args: r (`int`): Attention dimension of each Lora in Poly. target_modules (`Union[List[str],str]`): The names of the modules to apply Poly to. exclude_modules (`Optional[Union[List[str], str]]`): The names of the modules to not apply the adapter. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. modules_to_save (`List[str]`): List of modules apart from Poly layers to be set as trainable and saved in the final checkpoint. init_weights (bool): Whether to perform initialization of Poly weights. poly_type (`Literal["poly"]`): The variant of the Poly module to use. Currently, only "poly" is supported. n_tasks (`int`): The number of tasks in a multitasking scenario. n_skills (`int`): The number of skills (LoRA) in each Poly layer. n_splits (`int`): The number of splits within each LoRA of a Poly layer. A value greater than 1 indicates the use of Multi-Head Routing (MHR). helpzLora attention dimension)defaultmetadataintrNzList of module names or regex expression of the module names to replace with Poly.For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' zOptional[Union[list[str], str]]target_moduleszRList of module names or regex expression of the module names to exclude from Poly.exclude_modulesaList of modules apart from Poly layers to be set as trainable and saved in the final checkpoint. For example, in Sequence Classification or Token Classification tasks, the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved.zOptional[list[str]]modules_to_saveTzWhether to initialize the weights of the Poly layers with their default initialization. Don't change this setting, except if you know exactly what you're doing.bool init_weightspolyzDType of Poly modules to be used. Currently only "poly" is supported.zLiteral['poly'] poly_typez)Number of tasks in multitasking scenario.n_tasksz+Number of skills (LoRA) in each Poly layer.n_skillsz2Number of splits within each LoRA of a Poly layer.n_splitscsXttj|_t|jtrt|jn|j|_t|j tr&t|j |_ dS|j |_ dS)N) super __post_init__r POLY peft_type isinstancerlistsetr)self __class__K/home/ubuntu/.local/lib/python3.10/site-packages/peft/tuners/poly/config.pyr_s zPolyConfig.__post_init__)__name__ __module__ __qualname____doc__rr__annotations__rrrrrrrrr __classcell__r'r'r%r(r sP  r N) __future__r dataclassesrrtypingrrr peft.configr peft.utilsr r r'r'r'r(s