o 8wÖiã@s\ddlmZddlZddlmZddlmZddlmZddl m Z eGdd„deƒƒZ dS) é)Ú annotationsN)Ú dataclass)ÚOptional)Ú PeftConfig)ÚPeftTypecs¾eZdZUdZdZded<dZded<dZded <d Zded <d Z ded <d Z ded<dZ ded<dZ ded<d Z ded<dZded<dZded<dZded<dZded<‡fdd„Z‡ZS)Ú XLoraConfiga² This is the configuration class to store the configuration of a `XLoraModel`. When the config is reloaded, the paths of the `adapters` field is disregarded in favor of the saved adapters. As such, only the keys matter during loading. Args: hidden_size (`int`): Hidden size of the base model. adapters (`dict`): Mapping of adapter names to the LoRA adapter id, as per PeftModel.load_adapter. *They will be automatically loaded*, to use as LoRA experts. When using from_pretrained, pass the new adapters dict as a keyword argument. enable_softmax (`bool`, *optional*, defaults to `True`): Enable softmax application for the X-LoRA classifier. enable_softmax_topk (`bool`, *optional*, defaults to `False`): Enable softmax application for the top-k LoRA adapters. Mutually exclusive to `enable_softmax` and must only be set if `top_k_lora` is. softmax_temperature (`float`, *optional*, defaults to 1.0): Softmax temperature, lower yields sharper predictions layerwise_scalings (`bool`, *optional*, defaults to `False`): If True, generate scalings for each LoRA adapter (each layer). If this is False, then scalings will be broadcasted, the same, to each layer. top_k_lora (`int`, *optional*, defaults to None): Sparsely select the top_k LoRA experts instead of the default dense method. xlora_depth (`int`, *optional*, defaults to 1): Depth of the X-LoRA classifier. xlora_size (`int`, *optional*, defaults to 2048): Hidden size of the X-LoRA classifier, irrelevant if `xlora_depth=1`. xlora_dropout_p (`float`, *optional*, defaults to 0.2): Dropout probability of the X-LoRA classifier, irrelevant if `xlora_depth=1`. use_trainable_adapters (`bool`, *optional*, defaults to False): Make the adapters trainable. scaling_pass_value (`float`, *optional*, defaults to 0): Scaling pass value. global_scaling_weight (`float`, *optional*, defaults to 1): Weight to multiply output of each LoRA adapter by. NÚintÚ hidden_sizezdict[str, str]ÚadaptersTÚboolÚenable_softmaxFÚenable_softmax_topkÚlayerwise_scalingséÚ xlora_depthiÚ xlora_sizegš™™™™™É?ÚfloatÚxlora_dropout_pÚuse_trainable_adaptersgð?Úsoftmax_temperaturez Optional[int]Ú top_k_loragÚscaling_pass_valueÚglobal_scaling_weightcs tƒ ¡tj|_|jdurt d¡d|_|jdur#t d¡i|_|j r0|j dur0t d¡|j r;|j r;t d¡|j durL|j dkrNt d¡dSdSdS)NzqNo value was provided for `hidden_size`. This will be set to 4096 by default, please ensure that this is correct.izhNo value was provided for for `adapters`. This will be set to empty, please ensure that this is correct.z5`enable_softmax_topk` enabled `top_k_lora` is not setzc`enable_softmax_topk` and `enable_softmax` are both enabled. This will result in worse performance.rz&`top_k_lora` value must be at least 1.) ÚsuperÚ __post_init__rÚXLORAÚ peft_typer ÚwarningsÚwarnr r rr )Úself©Ú __class__©úU/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/peft/tuners/xlora/config.pyrNs*  ÿ ÿ  ÿÿzXLoraConfig.__post_init__)Ú__name__Ú __module__Ú __qualname__Ú__doc__r Ú__annotations__r r r rrrrrrrrrrÚ __classcell__r"r"r r#rs  &            r) Ú __future__rrÚ dataclassesrÚtypingrÚ peft.configrÚpeft.utils.peft_typesrrr"r"r"r#Ús