o Gi:u @sddlZddlmZddlmZddlZddlZddlm Z m Z ddl m Z m Z ddlmZmZe r7ddlZeGd d d e Z dd edededdejfddZGdddee ZdS)N) dataclass)Literal) ConfigMixinregister_to_config) BaseOutputis_scipy_available)KarrasDiffusionSchedulersSchedulerMixinc@s.eZdZUdZejed<dZejdBed<dS)KDPM2DiscreteSchedulerOutputaq Output class for the scheduler's `step` function output. Args: prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop. pred_original_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): The predicted denoised sample `(x_{0})` based on the model output from the current timestep. `pred_original_sample` can be used to preview progress or for guidance. prev_sampleNpred_original_sample)__name__ __module__ __qualname____doc__torchTensor__annotations__rrrd/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/schedulers/scheduling_k_dpm_2_discrete.pyr s  r +?cosinenum_diffusion_timestepsmax_betaalpha_transform_type)rexplaplacereturncCs|dkr dd}n|dkrdd}n|dkrdd}ntd|g}t|D]}||}|d |}|td |||||q(tj|tjd S) a> Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`str`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine`, `exp`, or `laplace`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. rcSs t|ddtjddS)NgMb?gT㥛 ?r)mathcospitrrr alpha_bar_fnMs z)betas_for_alpha_bar..alpha_bar_fnrc SsPdtdd|tddtd|d}t|}t|d|S)Ngr ?rgư>)r copysignlogfabsrsqrt)r$lmbsnrrrrr%Rs4 rcSst|dS)Ng()r rr#rrrr%Ysz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendminrtensorfloat32)rrrr%betasit1t2rrrbetas_for_alpha_bar3s     "r9c@seZdZdZddeDZdZe     dLdede de de de j e e Bd Bdededede de defddZeddZeddZedd ZdMd!efd"d#Zd$ejd%e ejBd&ejfd'd(Z dNd)ed*e ejBdefd+d,Zed-d.Z dOd%e ejBd/ejd Bd&efd0d1Zd%e ejBd&d fd2d3Zd4d5Zd6ejd&ejfd7d8Zd6ejd)ed&ejfd9d:Z ;dPd6ejd)edd?Z @dQdAeje j Bd%e ejBd$eje j BdBed&e!e"Bf dCdDZ#dEejdFejdGejd&ejfdHdIZ$dJdKZ%d S)RKDPM2DiscreteScheduleru/ KDPM2DiscreteScheduler is inspired by the DPMSolver2 and Algorithm 2 from the [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364) paper. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.00085): The starting `beta` value of inference. beta_end (`float`, defaults to 0.012): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear` or `scaled_linear`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://huggingface.co/papers/2210.02303) paper). timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. cCsg|]}|jqSr)name).0errr sz!KDPM2DiscreteScheduler.r_QK?~jt?linearNFepsilonlinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betasuse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasprediction_typetimestep_spacing steps_offsetc Cs|jjr ts tdt|jj|jj|jjgdkrtd|dur,tj |tj d|_ n:|dkr)z8KDPM2DiscreteScheduler.set_timesteps..crzrr{r}r~rrr>,rcrzrr{r}r~rrr>/r)rvgrcrzrr{)r<sigma_interpolr~rrr>CrrRrT),rurUrErNnprDr4copyarangeroundastyperOr/arrayr\r(interplenrJ_convert_to_karrasrK_convert_to_exponentialrL_convert_to_betar from_numpyrar concatenatelerprollrcatrepeat_interleaver`rrrTr.stackflatten timestepsrnr^r_) rbrurvrEr step_ratior`rrtimesteps_interpolinterleaved_timestepsrr~rr]s`  (  *       0& .z$KDPM2DiscreteScheduler.set_timestepscCs |jduSN)rnrfrrrrqPs z+KDPM2DiscreteScheduler.state_in_first_orderschedule_timestepscCs:|dur|j}||k}t|dkrdnd}||S)ak Find the index of a given timestep in the timestep schedule. Args: timestep (`float` or `torch.Tensor`): The timestep value to find in the schedule. schedule_timesteps (`torch.Tensor`, *optional*): The timestep schedule to search in. If `None`, uses `self.timesteps`. Returns: `int`: The index of the timestep in the schedule. For the very first step, returns the second index if multiple matches exist to avoid skipping a sigma when starting mid-schedule (e.g., for image-to-image). Nr r)rnonzeroritem)rbrorindicesposrrrindex_for_timestepUs   z)KDPM2DiscreteScheduler.index_for_timestepcCs@|jdurt|tjr||jj}|||_dS|j |_dS)z Initialize the step index for the scheduler based on the given timestep. Args: timestep (`float` or `torch.Tensor`): The current timestep to initialize the step index from. N) rl isinstancerrrarrvrr^r_)rbrorrrrpts   z'KDPM2DiscreteScheduler._init_step_indexc Cstt|d}||ddtjf}tj|dkddjddj|jddd}|d}||}||}||||} t| dd} d| || |} | |j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr)rer ) rr(maximumnewaxiscumsumargmaxclipshapereshape) rbrsr log_sigmadistslow_idxhigh_idxlowhighwr$rrrr|s, z"KDPM2DiscreteScheduler._sigma_to_tryc Cst|jdr |jj}nd}t|jdr|jj}nd}|dur |n|d}|dur,|n|d}d}tdd|}|d|}|d|}|||||} | S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. sigma_minN sigma_maxrwrg@r )hasattrrUrrrrrD) rbryrurrrhoramp min_inv_rho max_inv_rhor`rrrrs      z)KDPM2DiscreteScheduler._convert_to_karrascCst|jdr |jj}nd}t|jdr|jj}nd}|dur |n|d}|dur,|n|d}ttt |t ||}|S)a Construct an exponential noise schedule. Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following an exponential schedule. rNrrwr) rrUrrrrrrDr r()rbryrurrr`rrrrs     z.KDPM2DiscreteScheduler._convert_to_exponential333333?alphabetac st|jdr |jjndt|jdr|jjnddur n|ddur,n|dtfddfddd tdd |DD}|S) a Construct a beta noise schedule as proposed in [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. alpha (`float`, *optional*, defaults to `0.6`): The alpha parameter for the beta distribution. beta (`float`, *optional*, defaults to `0.6`): The beta parameter for the beta distribution. Returns: `torch.Tensor`: The converted sigma values following a beta distribution schedule. rNrrwrcsg|] }|qSrr)r<ppf)rrrrr>sz;KDPM2DiscreteScheduler._convert_to_beta..csg|] }tjj|qSr)scipystatsrr)r<ro)rrrrr>sr )rrUrrrrrrD)rbryrurrr`r)rrrrrrs       z'KDPM2DiscreteScheduler._convert_to_betaT model_output return_dictcCs|jdur |||jr$|j|j}|j|jd}|j|jd}n|j|jd}|j|j}|j|j}d}||d} |jjdkrT|jrK| n|} || |} n7|jjdkrw|jr_| n|} || | ddd|| dd} n|jjdkrtd td |jjd |jr|| | } || } ||_ n|| |} || } |j }d|_ |j d7_ || | }|s|| fSt || d S) a" Predict the sample from the previous timestep by reversing the SDE. This function propagates the diffusion process from the learned model outputs (most often the predicted noise). Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`float`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_k_dpm_2_discrete.KDPM2DiscreteSchedulerOutput`] or tuple. Returns: [`~schedulers.scheduling_k_dpm_2_discrete.KDPM2DiscreteSchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_k_dpm_2_discrete.KDPM2DiscreteSchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. Nr rrC v_predictionrr&rnz+prediction_type not implemented yet: samplezprediction_type given as z, must be one of `epsilon`, or `v_prediction`)r r) rirprqr`rrrUrMrXr/rnr^r )rbrrornrrsr sigma_nextgamma sigma_hat sigma_inputr derivativedtr rrrstep"sN             zKDPM2DiscreteScheduler.steporiginal_samplesnoisercs jj|j|jd}|jjdkr)t|r)jj|jtjd|j|jtjd}n j|j||j}j durFfdd|D}nj durUj g|j d}n j g|j d}|| }t |j t |j kr}|d}t |j t |j ksn|||}|S) am Add noise to the original samples according to the noise schedule at the specified timesteps. Args: original_samples (`torch.Tensor`): The original samples to which noise will be added. noise (`torch.Tensor`): The noise tensor to add to the original samples. timesteps (`torch.Tensor`): The timesteps at which to add noise, determining the noise level from the schedule. Returns: `torch.Tensor`: The noisy samples with added noise scaled according to the timestep schedule. )rvr.mpsr-Ncrzr)r)r<r$rrbrrr>rz4KDPM2DiscreteScheduler.add_noise..rrw)r`rarvr.typeris_floating_pointrr4rlrirrr unsqueeze)rbrrrr` step_indicesrs noisy_samplesrrr add_noises"      z KDPM2DiscreteScheduler.add_noisecCs|jjSr)rUrErfrrr__len__szKDPM2DiscreteScheduler.__len__) r?r@rArBNFFFrCrDr)r)NNr)rr)T)&rrrrr _compatiblesorderrintfloatstrrndarraylistboolrcpropertyrgrirlrmrrrtrvr]rqrrpr|rrrr tuplerrrrrrrr:gs&    +     ! T  %'# 5   _ 0r:)rr)r dataclassesrtypingrnumpyrrconfiguration_utilsrrutilsrrscheduling_utilsr r scipy.statsrr rrrr9r:rrrrs0   4