o piP@sddlZddlZddlmZddlmZmZmZmZddl Z ddl Z ddl m Z ddlmZmZddlmZdd lmZmZeGd d d eZ dddZGdddeeZdS)N) dataclass)ListOptionalTupleUnion) integrate) ConfigMixinregister_to_config) BaseOutput)KarrasDiffusionSchedulersSchedulerMixinc@s.eZdZUdZejed<dZeejed<dS)LMSDiscreteSchedulerOutputaq Output class for the scheduler's `step` function output. Args: prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop. pred_original_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images): The predicted denoised sample `(x_{0})` based on the model output from the current timestep. `pred_original_sample` can be used to preview progress or for guidance. prev_sampleNpred_original_sample) __name__ __module__ __qualname____doc__torchTensor__annotations__rrrrj/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/schedulers/scheduling_lms_discrete.pyrs  r+?cosinecCs|dkr dd}n|dkrdd}ntd|g}t|D]}||}|d|}|td|||||qtj|tjdS) a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs rcSs t|ddtjddS)NgMb?gT㥛 ?r)mathcospitrrr alpha_bar_fnIs z)betas_for_alpha_bar..alpha_bar_fnexpcSst|dS)Ng()rr#r rrrr"Nsz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendminrtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typer"betasit1t2rrrbetas_for_alpha_bar0s    "r3c@seZdZdZddeDZdZe     dDdede de de de e e jee fde ede de defddZeddZeddZeddZdEdefd d!Zd"ejd#e e ejfd$ejfd%d&Zd'd(ZdFd)ed*e e ejffd+d,ZdFd-d.Zd/d0Zd1d2Zd3ejd$ejfd4d5Z 6 7dGd8ejd#e e ejfd"ejd9ed:ed$e e!e"ff d;d<Z#d=ejd>ejd?ejd$ejfd@dAZ$dBdCZ%d S)HLMSDiscreteScheduleru A linear multistep scheduler for discrete beta schedules. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear` or `scaled_linear`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. cCsg|]}|jqSr)name).0errr }szLMSDiscreteScheduler.r -C6?{Gz?linearNFepsilonlinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betasuse_karras_sigmasprediction_typetimestep_spacing steps_offsetc Cs8|durtj|tjd|_n:|dkrtj|||tjd|_n*|dkr4tj|d|d|tjdd|_n|dkr>t||_n t|d|jd|j|_tj |jd d |_ t d |j |j d} t | ddd d ggt j} t| |_d|_||_||dg|_d|_d|_d|_|jd|_dS)Nr$r< scaled_linear?rsquaredcos_cap_v2z is not implemented for ?r)dimr Fcpu)rr*r+r/r>r3NotImplementedError __class__alphascumprodalphas_cumprodnparray concatenateastype from_numpysigmasnum_inference_stepsrD set_timesteps derivativesis_scale_input_called _step_index _begin_indexto) selfr?r@rArBrCrDrErFrGrZrrr__init__s, $  "  zLMSDiscreteScheduler.__init__cCs,|jjdvr |jS|jdddS)N)r>trailingrr rI)configrFrZmaxrbrrrinit_noise_sigmas  z%LMSDiscreteScheduler.init_noise_sigmacC|jS)zg The index counter for current timestep. It will increase 1 after each scheduler step. )r_rgrrr step_indexzLMSDiscreteScheduler.step_indexcCri)zq The index for the first timestep. It should be set from pipeline with `set_begin_index` method. r`rgrrr begin_indexrkz LMSDiscreteScheduler.begin_indexrmcCs ||_dS)z Sets the begin index for the scheduler. This function should be run from pipeline before the inference. Args: begin_index (`int`): The begin index for the scheduler. Nrl)rbrmrrrset_begin_indexs z$LMSDiscreteScheduler.set_begin_indexsampletimestepreturncCs>|jdur |||j|j}||ddd}d|_|S)a Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. timestep (`float` or `torch.Tensor`): The current timestep in the diffusion chain. Returns: `torch.Tensor`: A scaled input sample. Nrr rIT)rj_init_step_indexrZr^)rbrorpsigmarrrscale_model_inputs   z&LMSDiscreteScheduler.scale_model_inputcs<fdd}tj|jjdddd}|S)z Compute the linear multistep coefficient. Args: order (): t (): current_order (): csRd}tD] }|kr q||j|jj|9}q|S)NrK)r'rZ)tauprodk current_orderorderrbr!rrlms_derivatives  4z@LMSDiscreteScheduler.get_lms_coefficient..lms_derivativer r:)epsrelr)rquadrZ)rbrzr!ryr{integrated_coeffrrxrget_lms_coefficients &z(LMSDiscreteScheduler.get_lms_coefficientr[devicecs|_jjdkrtjdjjd|tjdddd}nVjjdkrHjjj}td|| ddd tj}|jj 7}n-jjdkrljjj}tjjd|   tj}|d8}n t jjd t djjd }t|t|tdt||}jjrj|d }t fd d |D}t|dgg tj}t|j|d_t|j|d_d_d_jd_g_dS)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. r>rr r$NrMleadingrdzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.rI) in_sigmascg|]}|qSr) _sigma_to_t)r6rs log_sigmasrbrrr8#z6LMSDiscreteScheduler.set_timesteps..rN)rrO)r[rerFrUr>r?r+copyarangeroundrXrGr&rVrTloginterplenrD_convert_to_karrasrWrrYrarZ timestepsr_r`r])rbr[rr step_ratiorZrrrr\s>  * $     z"LMSDiscreteScheduler.set_timestepscCs:|dur|j}||k}t|dkrdnd}||S)Nr r)rnonzeroritem)rbrpschedule_timestepsindicesposrrrindex_for_timestep0s   z'LMSDiscreteScheduler.index_for_timestepcCs@|jdurt|tjr||jj}|||_dS|j |_dSN) rm isinstancerrrarrrr_r`)rbrprrrrr?s   z%LMSDiscreteScheduler._init_step_indexc Cstt|d}||ddtjf}tj|dkddjddj|jddd}|d}||}||}||||} t| dd} d| || |} | |j} | S)Ng|=r)axisr)rfr ) rUrmaximumnewaxiscumsumargmaxclipshapereshape) rbrsr log_sigmadistslow_idxhigh_idxlowhighwr!rrrrHs, z LMSDiscreteScheduler._sigma_to_trc Cs\|d}|d}d}tdd|j}|d|}|d|}|||||}|S)z6Constructs the noise schedule of Karras et al. (2022).rMrg@r )rrUr>r[) rbr sigma_min sigma_maxrhoramp min_inv_rho max_inv_rhorZrrrr`s    z'LMSDiscreteScheduler._convert_to_karrasT model_outputrz return_dictc sHjstdjdur|jj}jjdkr%|||}n/jjdkrA|| |ddd||dd}njjdkrJ|}n td jjd |||}j |t j krmj d t jdfd d tD} |tddt| tj D} jd7_|s| fSt| |dS)aa Predict the sample from the previous timestep by reversing the SDE. This function propagates the diffusion process from the learned model outputs (most often the predicted noise). Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`float` or `torch.Tensor`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. order (`int`, defaults to 4): The order of the linear multistep method. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or tuple. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. zThe `scale_model_input` function should be called before `step` to ensure correct denoising. See `StableDiffusionPipeline` for a usage example.Nr= v_predictionrr rIrozprediction_type given as z, must be one of `epsilon`, or `v_prediction`rcsg|] }j|qSr)rrj)r6 curr_orderrzrbrrr8sz-LMSDiscreteScheduler.step..css|] \}}||VqdSrr)r6coeff derivativerrr s  z,LMSDiscreteScheduler.step..)rr)r^warningswarnrjrrrZrerEr&r]r(rpopr)r'sumzipreversedr_r) rbrrprorzrrsrr lms_coeffsrrrrstepms:     ,       zLMSDiscreteScheduler.steporiginal_samplesnoisercs jj|j|jd}|jjdkr)t|r)jj|jtjd|j|jtjd}n j|j||j}j durFfdd|D}nj durUj g|j d}n j g|j d}|| }t |j t |j kr}|d}t |j t |j ksn|||}|S)N)rr%mpsr$crr)r)r6r!rrbrrr8rz2LMSDiscreteScheduler.add_noise..rrM)rZrarr%typeris_floating_pointrr+rmrjrflattenr unsqueeze)rbrrrrZ step_indicesrs noisy_samplesrrr add_noises"      zLMSDiscreteScheduler.add_noisecCs|jjSr)rer?rgrrr__len__szLMSDiscreteScheduler.__len__) r9r:r;r<NFr=r>r)rr)rT)&rrrrr _compatiblesrzr intfloatstrrrrUndarrayrboolrcpropertyrhrjrmrnrrrtrrr\rrrrrrrrrrrrrrr4\s   +   $  5    N !r4)rr)rr dataclassesrtypingrrrrnumpyrUrscipyrconfiguration_utilsr r utilsr scheduling_utilsr rrr3r4rrrrs      ,