o ۷i}F@sddlmZddlZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZmZmZer:dd lmZd ZeeZGd d d e eZGd ddeZdddZdS)) annotationsN) TYPE_CHECKINGAny)validate_hf_hub_args)Self) ConfigMixin) BaseOutputPushToHubMixin get_logger BlockStatezguider_config.jsoncseZdZdZeZdZdZdQdRd d ZddZ ddZ ddZ dSddZ dTddZ dUfd!d" ZdVd%d&ZdVd'd(ZdWd,d-ZdXd0d1ZdYd3d4ZdZd5d6Zed[d7d8Zed[d9d:Zed\d;d<Zed]d@dAZed^dBdCZee   Dd_d`dJdKZdadbdOdPZZS)c BaseGuidancezGBase class providing the skeleton for implementing guidance techniques.N__guidance_identifier__?TstartfloatstopenabledboolcCstd||_||_d|_d|_d|_d|_d|_||_ d|kr'dks0nt d|d||kr:dksFnt d|d|d|j dusQt |j t sUt d dS) NzGuiders are currently an experimental feature under active development. The API is subject to breaking changes in future releases.rrrz4Expected `start` to be between 0.0 and 1.0, but got .zExpected `stop` to be between z and 1.0, but got z\`_input_predictions` must be a list of required prediction names for the guidance technique.)loggerwarning_start_stop_step_num_inference_steps _timestep_count_prepared _input_fields_enabled ValueError_input_predictions isinstancelist)selfrrrr'T/home/ubuntu/vllm_env/lib/python3.10/site-packages/diffusers/guiders/guider_utils.py__init__-s(zBaseGuidance.__init__cKs|jj|jfi|S)a Creates a copy of this guider instance, optionally with modified configuration parameters. Args: **kwargs: Configuration parameters to override in the new instance. If no kwargs are provided, returns an exact copy with the same configuration. Returns: A new guider instance with the same (or updated) configuration. Example: ```python # Create a CFG guider guider = ClassifierFreeGuidance(guidance_scale=3.5) # Create an exact copy same_guider = guider.new() # Create a copy with different start step, keeping other config the same new_guider = guider.new(guidance_scale=5) ``` ) __class__ from_configconfig)r&kwargsr'r'r(newEszBaseGuidance.newcC d|_dS)NFr!r&r'r'r(disable^ zBaseGuidance.disablecCr/)NTr0r1r'r'r(enablear3zBaseGuidance.enablestepintnum_inference_stepstimesteptorch.LongTensorreturnNonecCs||_||_||_d|_dS)Nr)rrrr)r&r5r7r8r'r'r( set_stateds zBaseGuidance.set_statedict[str, Any]cCs"|j|j|j|j|j|jd}|S)an Returns the current state of the guidance technique as a dictionary. The state variables will be included in the __repr__ method. Returns: `dict[str, Any]`: A dictionary containing the current state variables including: - step: Current inference step - num_inference_steps: Total number of inference steps - timestep: Current timestep tensor - count_prepared: Number of times prepare_models has been called - enabled: Whether the guidance is enabled - num_conditions: Number of conditions )r5r7r8count_preparedrnum_conditions)rrrrr!r?)r&stater'r'r( get_statejs zBaseGuidance.get_statestrc st}|}g}|D]0\}}t|}d|vr4|d}|ddddd|ddD}|d|d|qd|}|d |S) zq Returns a string representation of the guidance object including both config and current state.  rcSsg|]}d|qS)z r').0liner'r'r( sz)BaseGuidance.__repr__..Nz z: z State: )super__repr__rAitemsrBsplitjoinappend) r&str_reprr@ state_lineskvv_strv_lines state_strr*r'r(rIs  ( zBaseGuidance.__repr__denoisertorch.nn.ModulecCs|jd7_dS)z Prepares the models for the guidance technique on a given batch of data. This method should be overridden in subclasses to implement specific model preparation logic. rGN)rr&rVr'r'r(prepare_modelsszBaseGuidance.prepare_modelscCsdS)a* Cleans up the models for the guidance technique after a given batch of data. This method should be overridden in subclasses to implement specific model cleanup logic. It is useful for removing any hooks or other stateful modifications made during `prepare_models`. Nr'rXr'r'r(cleanup_modelsszBaseGuidance.cleanup_modelsdata 'BlockState'list['BlockState']cCtd)Nz?BaseGuidance::prepare_inputs must be implemented in subclasses.NotImplementedError)r&r[r'r'r(prepare_inputszBaseGuidance.prepare_inputs input_fields dict[str, str | tuple[str, str]]cCr^)NzPBaseGuidance::prepare_inputs_from_block_state must be implemented in subclasses.r_)r&r[rcr'r'r(prepare_inputs_from_block_statesz,BaseGuidance.prepare_inputs_from_block_statercsftdd|Ds tdt|jkr"tdjdt|dfdd|D}jd i|S) Ncss|]}t|dVqdS) noise_predN)hasattrrDdr'r'r( sz(BaseGuidance.__call__..z1Expected all data to have `noise_pred` attribute.z Expected z data items, but got z. Please check the input data.csi|] }t|j|jqSr')getattr_identifier_keyrfrhr1r'r( sz)BaseGuidance.__call__..r')allr"lenr?forward)r&r[forward_inputsr'r1r(__call__szBaseGuidance.__call__cOr^)Nz8BaseGuidance::forward must be implemented in subclasses.r_)r&argsr-r'r'r(rprbzBaseGuidance.forwardcCr^)Nz?BaseGuidance::is_conditional must be implemented in subclasses.r_r1r'r'r(is_conditionalzBaseGuidance.is_conditionalcCs|j S)N)rtr1r'r'r(is_unconditionalruzBaseGuidance.is_unconditionalcCr^)Nz?BaseGuidance::num_conditions must be implemented in subclasses.r_r1r'r'r(r?ruzBaseGuidance.num_conditions,dict[str, tuple[torch.Tensor, torch.Tensor]] tuple_index identifierc Csddlm}i}|D]8\}}z"t|tjr|||<nt|tr(||||<n tdt|Wq tyDt d|dYq w|||j <|di|S) Prepares a batch of data for the guidance technique. This method is used in the `prepare_inputs` method of the `BaseGuidance` class. It prepares the batch based on the provided tuple index. Args: input_fields (`dict[str, str | tuple[str, str]]`): A dictionary where the keys are the names of the fields that will be used to store the data once it is prepared with `prepare_inputs`. The values can be either a string or a tuple of length 2, which is used to look up the required data provided for preparation. If a string is provided, it will be used as the conditional data (or unconditional if used with a guidance method that requires it). If a tuple of length 2 is provided, the first element must be the conditional data identifier and the second element must be the unconditional data identifier or None. data (`BlockState`): The input data to be prepared. tuple_index (`int`): The index to use when accessing input fields that are tuples. Returns: `BlockState`: The prepared batch of data. rr zInvalid value type: "`data` does not have attribute(s) , skipping.Nr') "modular_pipelines.modular_pipeliner rJr$torchTensortupler"typerdebugrl)clsr[rxryr data_batchkeyvaluer'r'r(_prepare_batchs      zBaseGuidance._prepare_batchc Csddlm}i}|D]5\}}zt|trt||||<nt|tr-t|||||<n Wq tyAt d|dYq w|||j <|di|S)rzrr r{r|Nr') r}r rJr$rBrkrAttributeErrorrrrl) rrcr[rxryr rrrr'r'r(_prepare_batch_from_block_states     z,BaseGuidance._prepare_batch_from_block_stateFpretrained_model_name_or_pathstr | os.PathLike | None subfolder str | NonercKs6|jd||ddd|\}}}|j|fd|i|S)a Instantiate a guider from a pre-defined JSON configuration file in a local directory or Hub repository. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the guider configuration saved with [`~BaseGuidance.save_pretrained`]. subfolder (`str`, *optional*): The subfolder location of a model file within a larger model repository on the Hub or locally. return_unused_kwargs (`bool`, *optional*, defaults to `False`): Whether kwargs that are not consumed by the Python class should be returned or not. cache_dir (`str | os.PathLike`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. output_loading_info(`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. > [!TIP] > To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with `hf > auth login`. You can also activate the special > ["offline-mode"](https://huggingface.co/diffusers/installation.html#offline-mode) to use this method in a > firewalled environment. T)rrreturn_unused_kwargsreturn_commit_hashrNr') load_configr+)rrrrr-r, commit_hashr'r'r(from_pretrained!s4 zBaseGuidance.from_pretrainedsave_directorystr | os.PathLike push_to_hubcKs|jd||d|dS)a: Save a guider configuration object to a directory so that it can be reloaded using the [`~BaseGuidance.from_pretrained`] class method. Args: save_directory (`str` or `os.PathLike`): Directory where the configuration JSON file will be saved (will be created if it does not exist). push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face Hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. )rrNr') save_config)r&rrr-r'r'r(save_pretrained^szBaseGuidance.save_pretrained)rrT)rrrrrr)r5r6r7r6r8r9r:r;)r:r=)r:rB)rVrWr:r;)r[r\r:r])r[r\rcrdr:r])r[r]r:r)r:r)r:r)r:r6)r[rwrxr6ryrBr:r\) rcrdr[r\rxr6ryrBr:r\)NNF)rrrrr:r)F)rrrr) __name__ __module__ __qualname____doc__GUIDER_CONFIG_NAME config_namer#rlr)r.r2r4r<rArIrYrZrarerrrppropertyrtrvr? classmethodrrrrr __classcell__r'r'rUr(r&sF             * ,;rc@s&eZdZUded<ded<ded<dS) GuiderOutputz torch.Tensorpredztorch.Tensor | None pred_cond pred_uncondN)rrr__annotations__r'r'r'r(rps  rrcCsX|jttd|jdd}|jttd|jdd}|||}||d||}|S)a Rescales `noise_cfg` tensor based on `guidance_rescale` to improve image quality and fix overexposure. Based on Section 3.4 from [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891). Args: noise_cfg (`torch.Tensor`): The predicted noise tensor for the guided diffusion process. noise_pred_text (`torch.Tensor`): The predicted noise tensor for the text-guided diffusion process. guidance_rescale (`float`, *optional*, defaults to 0.0): A rescale factor applied to the noise predictions. Returns: noise_cfg (`torch.Tensor`): The rescaled noise prediction tensor. rGT)dimkeepdim)stdr%rangendim) noise_cfgnoise_pred_textguidance_rescalestd_textstd_cfgnoise_pred_rescaledr'r'r(rescale_noise_cfgvs  r)r) __future__rostypingrrr~huggingface_hub.utilsrtyping_extensionsrconfiguration_utilsrutilsr r r r}r rrrrrrr'r'r'r(s"     L