o pi%@s ddlZddlmZmZmZmZmZddlZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZddlmZddlmZmZerOdd lmZmZeeZ d Z!d Z"d Z#d Z$dZ%GdddeZ&GdddeZ'GdddeZ(GdddeZ)Gddde&Z*Gddde&Z+dS)N)CallableDictListOptionalUnion)validate_hf_hub_args) USE_PEFT_BACKENDconvert_state_dict_to_diffusersconvert_state_dict_to_peftconvert_unet_state_dict_to_peft deprecateget_adapter_nameget_peft_kwargsis_peft_versionis_transformers_availableloggingscale_lora_layers) LoraBaseMixin)(_convert_non_diffusers_lora_to_diffusers"_maybe_map_sgm_blocks_to_diffusers)text_encoder_attn_modulestext_encoder_mlp_modules text_encoderunet transformerzpytorch_lora_weights.binz pytorch_lora_weights.safetensorscsNeZdZdZddgZeZeZ d"de e e e e j fffddZeede e e e e j fffdd Zed#d d Ze   d$d dZe      d%de e ejfde e e e jje j ffde e e jjfdede dedefddZddgd ddfdee dededeee ffdd Zddgfdee ffd d! ZZ S)&StableDiffusionLoraLoaderMixinz Load LoRA layers into Stable Diffusion [`UNet2DConditionModel`] and [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel). rrN%pretrained_model_name_or_path_or_dictcKststdt|tr|}|j|fi|\}}tdd|D}|s+td|j||t |ds:t ||j n|j ||d|j ||t |dsPt ||jn|j|j||dd S) a Load LoRA weights specified in `pretrained_model_name_or_path_or_dict` into `self.unet` and `self.text_encoder`. All kwargs are forwarded to `self.lora_state_dict`. See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`] for more details on how the state dict is loaded. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_unet`] for more details on how the state dict is loaded into `self.unet`. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_text_encoder`] for more details on how the state dict is loaded into `self.text_encoder`. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. kwargs (`dict`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. )PEFT backend is required for this method.cs |] }d|vp d|vVqdSlora dora_scaleN.0keyr$r$]/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/loaders/lora_pipeline.py azCStableDiffusionLoraLoaderMixin.load_lora_weights..Invalid LoRA checkpoint.rnetwork_alphasr adapter_name _pipeliner)r-r lora_scaler.r/N)r ValueError isinstancedictcopylora_state_dictallkeysload_lora_into_unethasattrgetattr unet_namerload_lora_into_text_encodertext_encoder_namerr0)selfrr.kwargs state_dictr-is_correct_formatr$r$r(load_lora_weights<s2  z0StableDiffusionLoraLoaderMixin.load_lora_weightscK|dd}|dd}|dd}|dd}|dd}|dd}|d d} |d d} |d d} |d d} d} | durFd } d } ddd}|j|| | ||||||| || d }d}tdd|Drx| durrt|| }t|\}}||fS) Return state dict for lora weights and the network alphas. We support loading A1111 formatted LoRA checkpoints in a limited capacity. This function is experimental and might change in the future. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`ModelMixin.save_pretrained`]. - A [torch state dict](https://pytorch.org/tutorials/beginner/saving_loading_models.html#what-is-a-state-dict). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only (`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. subfolder (`str`, *optional*, defaults to `""`): The subfolder location of a model file within a larger model repository on the Hub or locally. weight_name (`str`, *optional*, defaults to None): Name of the serialized state dict file. cache_dirNforce_downloadFproxieslocal_files_onlytokenrevision subfolder weight_name unet_configuse_safetensorsTattn_procs_weightspytorch file_type framework rrLrNrHrErFrGrIrJrK user_agent allow_picklecs8|]}|dp|dp|dp|dVqdSlora_te_ lora_unet_ lora_te1_ lora_te2_N startswithr&kr$r$r(r)  zAStableDiffusionLoraLoaderMixin.lora_state_dict..pop_fetch_state_dictr6r7rrclsrr?rErFrGrHrIrJrKrLrMrNrVrUr@r-r$r$r(r5wsN 7           z.StableDiffusionLoraLoaderMixin.lora_state_dictc`tstdt|}tfdd|D}|s.tdjd|j||||ddSdS) This will load the LoRA layers specified in `state_dict` into `unet`. Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The keys can either be indexed directly into the unet or prefixed with an additional `unet` which can be used to distinguish between text encoder lora layers. network_alphas (`Dict[str, float]`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. Refer to [this link](https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning). unet (`UNet2DConditionModel`): The UNet model to load the LoRA layers into. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rc3|] }|jVqdSNr^r=r%rfr$r(r)zEStableDiffusionLoraLoaderMixin.load_lora_into_unet..Loading .r-r.r/N r r1listr7r6loggerinfor;load_attn_procsrfr@r-rr.r/r7only_text_encoderr$rlr(r8s  z2StableDiffusionLoraLoaderMixin.load_lora_into_unet?cDtstdddlm}t|} durjntfdd| Drfdd| Dfd d |D} t | dkrt d d i} t | } t | } t|D]\} } d D]}| d |d}|| vroq`| |jd| |<q`qZt|D]\} } dD]}| d |d}|| vrq| |jd| |<qq~|durfdd|Dfdd |D}t| || dd}d|vr|drtddrtdn tddr|d|di|}|durt|}|\}}|j|| |dt||d|j|j|jd|r|dS|r |dSdSdSdS)? This will load the LoRA layers specified in `state_dict` into `text_encoder` Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The key should be prefixed with an additional `text_encoder` to distinguish between unet lora layers. network_alphas (`Dict[str, float]`): See `LoRALinearLayer` for more details. text_encoder (`CLIPTextModel`): The text encoder model to load the LoRA layers into. prefix (`str`): Expected prefix of the `text_encoder` in the `state_dict`. lora_scale (`float`): How much to scale the output of the lora linear layer before it is added with the output of the regular lora layer. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rr LoraConfigNc3|]}j|vVqdSrjr=r%rlr$r(r)3zMStableDiffusionLoraLoaderMixin.load_lora_into_text_encoder..c,g|]}|r|ddkr|qSrorr^splitr_prefixr$r( 5,zNStableDiffusionLoraLoaderMixin.load_lora_into_text_encoder..c,i|]\}}|vr|dd|qSroreplacer&r`vrtext_encoder_keysr$r( 6&zNStableDiffusionLoraLoaderMixin.load_lora_into_text_encoder..rnroout_projq_projk_projv_proj.lora_B.weightrfc1fc2c,g|]}|r|ddkr|qSrrr_rr$r(rQ c,i|]\}}|vr|dd|qSrrr alpha_keysrr$r(rTrFis_unetuse_dora<0.9.0eYou need `peft` 0.9.0 at least to use DoRA-enabled LoRAs. Please upgrade your installation of `peft`.r.adapter_state_dict peft_configweightdevicedtyper$r r1peftr|rrr7r=anyitemslenrsrtr r rshaperrrrcr_optionally_disable_offloading load_adapterrtorrenable_model_cpu_offloadenable_sequential_cpu_offloadrfr@r-rrr0r.r/r|r7text_encoder_lora_state_dictrankname_modulerank_keylora_config_kwargs lora_configis_model_cpu_offloadis_sequential_cpu_offloadr$rrfrrr(r<s           @z:StableDiffusionLoraLoaderMixin.load_lora_into_text_encoderTsave_directoryunet_lora_layerstext_encoder_lora_layersis_main_processrL save_functionsafe_serializationc C^i}|s |s td|r||||j|r"||||j|j||||||ddS) Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. unet_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `unet`. text_encoder_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. zPYou must pass at least one of `unet_lora_layers` and `text_encoder_lora_layers`.r@rrrLrrN)r1update pack_weightsr;r=write_lora_layers) rfrrrrrLrrr@r$r$r(save_lora_weights~! z0StableDiffusionLoraLoaderMixin.save_lora_weightsF componentsr0 safe_fusing adapter_namesc tj||||ddSa Fuses the LoRA parameters into the original parameters of the corresponding blocks. This is an experimental API. Args: components: (`List[str]`): List of LoRA-injectable components to fuse the LoRAs into. lora_scale (`float`, defaults to 1.0): Controls how much to influence the outputs with the LoRA parameters. safe_fusing (`bool`, defaults to `False`): Whether to check fused weights for NaN values before fusing and if values are NaN not fusing them. adapter_names (`List[str]`, *optional*): Adapter names to be used for fusing. If nothing is passed, all active adapters will be fused. Example: ```py from diffusers import DiffusionPipeline import torch pipeline = DiffusionPipeline.from_pretrained( "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16 ).to("cuda") pipeline.load_lora_weights("nerijs/pixel-art-xl", weight_name="pixel-art-xl.safetensors", adapter_name="pixel") pipeline.fuse_lora(lora_scale=0.7) ``` )rr0rrNsuper fuse_lorar>rr0rrr? __class__r$r(r' z(StableDiffusionLoraLoaderMixin.fuse_lorac tj|ddSa Reverses the effect of [`pipe.fuse_lora()`](https://huggingface.co/docs/diffusers/main/en/api/loaders#diffusers.loaders.LoraBaseMixin.fuse_lora). This is an experimental API. Args: components (`List[str]`): List of LoRA-injectable components to unfuse LoRA from. unfuse_unet (`bool`, defaults to `True`): Whether to unfuse the UNet LoRA parameters. unfuse_text_encoder (`bool`, defaults to `True`): Whether to unfuse the text encoder LoRA parameters. If the text encoder wasn't monkey-patched with the LoRA parameters then it won't have any effect. rNr unfuse_lorar>rr?rr$r(rz*StableDiffusionLoraLoaderMixin.unfuse_lorarjNNNrxNNNNTNNT)!__name__ __module__ __qualname____doc___lora_loadable_modules UNET_NAMEr;TEXT_ENCODER_NAMEr=rstrrtorchTensorrB classmethodrr5r8r<osPathLikennModuleboolrrrfloatrrr __classcell__r$r$rr(r2st ;l " u  7 &+rcs|eZdZdZgdZeZeZ d#de e e e e j ffdee fddZeede e e e e j fffdd Zed$d d Ze   d%d dZe       d&de e ejfde e e e jje j ffde e e e jje j ffde e e e jje j ffdede dedefddZgdd ddfdee dededeee ffdd Zgdfdee ffd!d" ZZ S)' StableDiffusionXLLoraLoaderMixinaC Load LoRA layers into Stable Diffusion XL [`UNet2DConditionModel`], [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel), and [`CLIPTextModelWithProjection`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModelWithProjection). )rrtext_encoder_2Nrr.c Kststdt|tr|}|j|fd|jji|\}}tdd| D}|s/td|j |||j||ddd| D}t |d krW|j |||jd |j||d d d| D}t |d krv|j |||jd |j||d dSdS)a Load LoRA weights specified in `pretrained_model_name_or_path_or_dict` into `self.unet` and `self.text_encoder`. All kwargs are forwarded to `self.lora_state_dict`. See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`] for more details on how the state dict is loaded. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_unet`] for more details on how the state dict is loaded into `self.unet`. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_text_encoder`] for more details on how the state dict is loaded into `self.text_encoder`. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. kwargs (`dict`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. rrMcsr r!r$r%r$r$r(r).r*zEStableDiffusionXLLoraLoaderMixin.load_lora_weights..r+r,cSi|] \}}d|vr||qSz text_encoder.r$rr$r$r(r5zFStableDiffusionXLLoraLoaderMixin.load_lora_weights..rrr-rrr0r.r/cSrztext_encoder_2.r$rr$r$r(rArrN)r r1r2r3r4r5rconfigr6r7r8rrr<rr0r) r>rr.r?r@r-rAtext_encoder_state_dicttext_encoder_2_state_dictr$r$r(rBsP      z2StableDiffusionXLLoraLoaderMixin.load_lora_weightscKrC)rDrENrFFrGrHrIrJrKrLrMrNTrOrPrQrTcsrWrXr]r_r$r$r(r)razCStableDiffusionXLLoraLoaderMixin.lora_state_dict..rbrer$r$r(r5MsN 8           z0StableDiffusionXLLoraLoaderMixin.lora_state_dictcrg)rhrc3rirjrkr%rlr$r(r)rmzGStableDiffusionXLLoraLoaderMixin.load_lora_into_unet..rnrorpNrqrvr$rlr(r8s  z4StableDiffusionXLLoraLoaderMixin.load_lora_into_unetrxcry)rzrrr{Nc3r}rjr~r%rlr$r(r) rzOStableDiffusionXLLoraLoaderMixin.load_lora_into_text_encoder..crrrr_rr$r(rrzPStableDiffusionXLLoraLoaderMixin.load_lora_into_text_encoder..crrrrrr$r(rrzPStableDiffusionXLLoraLoaderMixin.load_lora_into_text_encoder..rnrorrrrcrrrr_rr$r(r*rcrrrrrr$r(r-rFrrrrrrrrr$rrr$rr(r<            @z We support loading A1111 formatted LoRA checkpoints in a limited capacity. This function is experimental and might change in the future. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`ModelMixin.save_pretrained`]. - A [torch state dict](https://pytorch.org/tutorials/beginner/saving_loading_models.html#what-is-a-state-dict). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. local_files_only (`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. subfolder (`str`, *optional*, defaults to `""`): The subfolder location of a model file within a larger model repository on the Hub or locally. rENrFFrGrHrIrJrKrLrNTrOrPrQrTrcrdrfrr?rErFrGrHrIrJrKrLrNrVrUr@r$r$r(r5s> 6        z"SD3LoraLoaderMixin.lora_state_dictNc Kststdt|tr|}|j|fi|}tdd|D}|s)td|j|t |ds7t ||j n|j ||ddd| D}t|d kr[|j|d |jd |j||d d d| D}t|d krz|j|d |jd|j||d d Sd S)a Load LoRA weights specified in `pretrained_model_name_or_path_or_dict` into `self.unet` and `self.text_encoder`. All kwargs are forwarded to `self.lora_state_dict`. See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`] for more details on how the state dict is loaded. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_transformer`] for more details on how the state dict is loaded into `self.transformer`. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. kwargs (`dict`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rcsr r!r$r%r$r$r(r)_r*z7SD3LoraLoaderMixin.load_lora_weights..r+rrr.r/cSrrr$rr$r$r(rjrz8SD3LoraLoaderMixin.load_lora_weights..rNrrcSrrr$rr$r$r(rvrr)r r1r2r3r4r5r6r7load_lora_into_transformerr9r:transformer_namerrrr<rr0r)r>rr.r?r@rArrr$r$r(rB=sJ    z$SD3LoraLoaderMixin.load_lora_weightscddlm}m}m}t|}fdd|Dfdd|D}t|dkrtt |} d| vr=t |}|t |divrMt d |d i} |D]\} } d | vrb| j d | | <qSt| d |d} d| vr| dr{tddr{t d| d|di| }|d urt|}|\}}||||d||||}|d urt |dd }|rtd|d|r|d S|r|d Sd Sd S) This will load the LoRA layers specified in `state_dict` into `transformer`. Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The keys can either be indexed directly into the unet or prefixed with an additional `unet` which can be used to distinguish between text encoder lora layers. transformer (`SD3Transformer2DModel`): The Transformer model to load the LoRA layers into. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rr|inject_adapter_in_modelset_peft_model_state_dictcg|] }|jr|qSr$r^rr_rlr$r(rzASD3LoraLoaderMixin.load_lora_into_transformer..c.i|]\}}|vr|jdd|qSrrrrrftransformer_keysr$r(r(zASD3LoraLoaderMixin.load_lora_into_transformer..lora_Ar Adapter name F already in use in the transformer - please select a new adapter name.lora_BrNnetwork_alpha_dictpeft_state_dictrrrrr.unexpected_keysXLoading adapter weights from state_dict led to unexpected keys not found in the model: . r$rr|rrrrr7rrnextiterr r:r1rrrrcrrrswarningrrrfr@rr.r/r|rrr7 first_keyrr'valrrrrincompatible_keysr&r$rr(rs\        1z-SD3LoraLoaderMixin.load_lora_into_transformerrxcry)rzrrr{Nc3r}rjr~r%rlr$r(r)rzASD3LoraLoaderMixin.load_lora_into_text_encoder..crrrr_rr$r(rrzBSD3LoraLoaderMixin.load_lora_into_text_encoder..crrrrrr$r(rrzBSD3LoraLoaderMixin.load_lora_into_text_encoder..rnrorrrrcrrrr_rr$r(rrcrrrrrr$r(rrFrrrrrrrrr$rrr$rr(r<rz.SD3LoraLoaderMixin.load_lora_into_text_encoderTrtransformer_lora_layersrrrrLrrc Csvi} |s |s |s td|r| |||j|r#| ||d|r.| ||d|j| |||||ddS)uD Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. transformer_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `transformer`. text_encoder_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. text_encoder_2_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder_2`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. zrYou must pass at least one of `transformer_lora_layers`, `text_encoder_lora_layers`, `text_encoder_2_lora_layers`.rrrN)r1rrrr) rfrr1rrrrLrrr@r$r$r(rGs&%  z$SD3LoraLoaderMixin.save_lora_weightsFrr0rrc rrrrrr$r(rrzSD3LoraLoaderMixin.fuse_lorac rrrrrr$r(rrzSD3LoraLoaderMixin.unfuse_lorarjrrr!rrrrrTRANSFORMER_NAMErrr=rrrrrrrr5rBrr<rrrrrrrrrrrrrr$r$rr(r szZ E M u  @ &+r csNeZdZdZddgZeZeZe e de e e e ejfffddZ d"de e e e ejfffdd Ze d#d d Ze   d$d dZe      d%de e ejfde e e ejjejffde e ejjfdede dedefddZddgd ddfdee dededeee ffdd Zddgfdee ffd d! ZZ S)&FluxLoraLoaderMixinz Load LoRA layers into [`FluxTransformer2DModel`], [`CLIPTextModel`](https://huggingface.co/docs/transformers/model_doc/clip#transformers.CLIPTextModel). Specific to [`StableDiffusion3Pipeline`]. rrrcKr r r r r$r$r(r5s> 7        z#FluxLoraLoaderMixin.lora_state_dictNc Kststdt|tr|}|j|fi|}tdd|D}|s)td|j|t |ds7t ||j n|j ||ddd| D}t|d kr]|j|d |jd |j||d d Sd S) a Load LoRA weights specified in `pretrained_model_name_or_path_or_dict` into `self.transformer` and `self.text_encoder`. All kwargs are forwarded to `self.lora_state_dict`. See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`] for more details on how the state dict is loaded. See [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_into_transformer`] for more details on how the state dict is loaded into `self.transformer`. Parameters: pretrained_model_name_or_path_or_dict (`str` or `os.PathLike` or `dict`): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. kwargs (`dict`, *optional*): See [`~loaders.StableDiffusionLoraLoaderMixin.lora_state_dict`]. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rcsr r!r$r%r$r$r(r)Pr*z8FluxLoraLoaderMixin.load_lora_weights..r+rrcSrrr$rr$r$r(r[rz9FluxLoraLoaderMixin.load_lora_weights..rNrr)r r1r2r3r4r5r6r7rr9r:rrrrr<rr0)r>rr.r?r@rArr$r$r(rB.s4   z%FluxLoraLoaderMixin.load_lora_weightscr)rrrcrr$rr_rlr$r(r|rzBFluxLoraLoaderMixin.load_lora_into_transformer..crrrrrr$r(r}rzBFluxLoraLoaderMixin.load_lora_into_transformer..rrrr r!rNr"rrrrr%r&r'r(r$r)r-r$rr(rgs\        1z.FluxLoraLoaderMixin.load_lora_into_transformerrxcry)rzrrr{Nc3r}rjr~r%rlr$r(r)rzBFluxLoraLoaderMixin.load_lora_into_text_encoder..crrrr_rr$r(rrzCFluxLoraLoaderMixin.load_lora_into_text_encoder..crrrrrr$r(rrzCFluxLoraLoaderMixin.load_lora_into_text_encoder..rnrorrrrcrrrr_rr$r(rrcrrrrrr$r(rrFrrrrrrrrr$rrr$rr(r<rz/FluxLoraLoaderMixin.load_lora_into_text_encoderTrr1rrrLrrc Cr)u Save the LoRA parameters corresponding to the UNet and text encoder. Arguments: save_directory (`str` or `os.PathLike`): Directory to save LoRA parameters to. Will be created if it doesn't exist. transformer_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `transformer`. text_encoder_lora_layers (`Dict[str, torch.nn.Module]` or `Dict[str, torch.Tensor]`): State dict of the LoRA layers corresponding to the `text_encoder`. Must explicitly pass the text encoder LoRA state dict because it comes from 🤗 Transformers. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. zWYou must pass at least one of `transformer_lora_layers` and `text_encoder_lora_layers`.rNr1rrrr=r) rfrr1rrrLrrr@r$r$r(r-s" z%FluxLoraLoaderMixin.save_lora_weightsFrr0rrc rrrrrr$r(rerzFluxLoraLoaderMixin.fuse_lorac r)al Reverses the effect of [`pipe.fuse_lora()`](https://huggingface.co/docs/diffusers/main/en/api/loaders#diffusers.loaders.LoraBaseMixin.fuse_lora). This is an experimental API. Args: components (`List[str]`): List of LoRA-injectable components to unfuse LoRA from. rNrrrr$r(rszFluxLoraLoaderMixin.unfuse_lorarjrrrr2r$r$rr(r4stZ 9 M u  8 &+r4c@seZdZddgZeZeZedddZ e    dddZ e     dd e e e jfd ee ejjfd ee ejjfd ede dedefddZdS)AmusedLoraLoaderMixinrrNcststdddlm}m}m}t|} fdd| Dfdd|D}|durFfd d|Dfd d|D}t |dkr|t |d ivr^td |d i} |D]\} } d| vrs| j d| | <qdt | ||} d| vr| drt ddrtd| d|di| }|durt|}|\}}||||d||||}|durt |dd}|rtd|d|r|dS|r|dSdSdS)ab This will load the LoRA layers specified in `state_dict` into `transformer`. Parameters: state_dict (`dict`): A standard state dict containing the lora layer parameters. The keys can either be indexed directly into the unet or prefixed with an additional `unet` which can be used to distinguish between text encoder lora layers. network_alphas (`Dict[str, float]`): See `LoRALinearLayer` for more details. unet (`UNet2DConditionModel`): The UNet model to load the LoRA layers into. adapter_name (`str`, *optional*): Adapter name to be used for referencing the loaded adapter model. If not specified, it will use `default_{i}` where i is the total number of adapters being loaded. rrrcrr$rr_rlr$r(rrzDAmusedLoraLoaderMixin.load_lora_into_transformer..crrrrrr$r(rrzDAmusedLoraLoaderMixin.load_lora_into_transformer..Ncrr$rr_rlr$r(rrcs.i|]\}}|vr|jdd|qSrrr)rrfr$r(rrrrr r!rrrrrr%r&r'r(r$)r r1rr|rrrrr7rrr:rrrrcrrrsr,rr)rfr@r-rr.r/r|rrr7rr'r/rrrrr0r&r$)rrfrr(rsd          ,z0AmusedLoraLoaderMixin.load_lora_into_transformerrxcry)rzrrr{Nc3r}rjr~r%rlr$r(r)(rzDAmusedLoraLoaderMixin.load_lora_into_text_encoder..crrrr_rr$r(r*rzEAmusedLoraLoaderMixin.load_lora_into_text_encoder..crrrrrr$r(r+rzEAmusedLoraLoaderMixin.load_lora_into_text_encoder..rnrorrrrcrrrr_rr$r(rFrcrrrrrr$r(rIrFrrrrrrrrr$rrr$rr(r<rz1AmusedLoraLoaderMixin.load_lora_into_text_encoderTrrr1rrLrrc Cs^i}|s |s td|r||||j|r"||||j|j||||||ddS)rzVYou must pass at least one of `transformer_lora_layers` or `text_encoder_lora_layers`.rNr5) rfrrr1rrLrrr@r$r$r(rsrz'AmusedLoraLoaderMixin.save_lora_weightsrrr)rrrrr3rrr=rrr<rrrrrrrrrrrr$r$r$r(r6sD S u r6cseZdZfddZZS)LoraLoaderMixincs&d}tdd|tj|i|dS)NzLoraLoaderMixin is deprecated and this will be removed in a future version. Please use `StableDiffusionLoraLoaderMixin`, instead.r7z1.0.0)r r__init__)r>argsr?deprecation_messagerr$r(r8s zLoraLoaderMixin.__init__)rrrr8rr$r$rr(r7sr7),rtypingrrrrrrhuggingface_hub.utilsrutilsr r r r r rrrrrr lora_baserlora_conversion_utilsrr models.lorarr get_loggerrrsrrr3LORA_WEIGHT_NAMELORA_WEIGHT_NAME_SAFErrr r4r6r7r$r$r$r(sF  4  Edt`