o pisW@sxddlZddlmZmZmZmZddlZddlZddl m Z m Z ddl m Z mZmZ  d d d ZGd d d ee ZdS)N)ListOptionalTupleUnion) ConfigMixinregister_to_config)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutput+?cosinecCs|dkr dd}n|dkrdd}ntd|g}t|D]}||}|d|}|td|||||qtj|tjdS) a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs rcSs t|ddtjddS)NgMb?gT㥛 ?r)mathcospitrk/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/schedulers/scheduling_heun_discrete.py alpha_bar_fn3s z)betas_for_alpha_bar..alpha_bar_fnexpcSst|dS)Ng()rrrrrrr8sz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typerbetasit1t2rrrbetas_for_alpha_bars    "r(c@seZdZdZddeDZdZe     dEdede de de de e e jee fde de ede ede de defddZdFddZeddZed d!Zed"d#ZdGd$efd%d&Zd'ejd(e e ejfd)ejfd*d+Z dHd,e ed-e e ejfde ed.e eefd/d0Zd1d2Zd3ejd)ejfd4d5Zed6d7Zd8d9Z :dId;e eje jfd(e e ejfd'e eje jfdZ#d?ejd@ejd.ejd)ejfdAdBZ$dCdDZ%d S)JHeunDiscreteScheduleru Scheduler with Heun steps for discrete beta schedules. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear` or `scaled_linear`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). clip_sample (`bool`, defaults to `True`): Clip the predicted sample for numerical stability. clip_sample_range (`float`, defaults to 1.0): The maximum magnitude for sample clipping. Valid only when `clip_sample=True`. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. cCsg|]}|jqSr)name).0errr ksz HeunDiscreteScheduler.r_QK?~jt?linearNepsilonF?linspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betasprediction_typeuse_karras_sigmas clip_sampleclip_sample_rangetimestep_spacing steps_offsetc Cs|durtj|tjd|_nH|dkrtj|||tjd|_n8|dkr4tj|d|d|tjdd|_n"|dkr@t|dd|_n|d krLt|d d|_n t|d |jd |j|_tj |jd d |_ | |d|||_ d|_ d|_|jd|_dS)Nrr1 scaled_linear?rsquaredcos_cap_v2r)r#rz is not implemented for r3r)dimcpu)rrr r$r4r(NotImplementedError __class__alphascumprodalphas_cumprod set_timestepsr; _step_index _begin_indexsigmasto) selfr5r6r7r8r9r:r;r<r=r>r?rrr__init__ns$$ zHeunDiscreteScheduler.__init__cCs:|dur|j}||k}t|dkrdnd}||S)Nr r) timestepsnonzerolenitem)rOtimestepschedule_timestepsindicesposrrrindex_for_timesteps   z(HeunDiscreteScheduler.index_for_timestepcCs,|jjdvr |jS|jdddS)N)r4trailingrr rA)configr>rMmaxrOrrrinit_noise_sigmas  z&HeunDiscreteScheduler.init_noise_sigmacC|jS)zg The index counter for current timestep. It will increase 1 after each scheduler step. )rKr]rrr step_indexz HeunDiscreteScheduler.step_indexcCr_)zq The index for the first timestep. It should be set from pipeline with `set_begin_index` method. rLr]rrr begin_indexraz!HeunDiscreteScheduler.begin_indexrccCs ||_dS)z Sets the begin index for the scheduler. This function should be run from pipeline before the inference. Args: begin_index (`int`): The begin index for the scheduler. Nrb)rOrcrrrset_begin_indexs z%HeunDiscreteScheduler.set_begin_indexsamplerUreturncCs8|jdur |||j|j}||ddd}|S)a Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. timestep (`int`, *optional*): The current timestep in the diffusion chain. Returns: `torch.Tensor`: A scaled input sample. Nrr rA)r`_init_step_indexrM)rOrerUsigmarrrscale_model_inputs   z'HeunDiscreteScheduler.scale_model_inputnum_inference_stepsdevicerQcs|dur |dur td|dur|durtd|dur$jjr$td|p)t|}|_|p2jj}|dur@tj|tjd}njjj dkrZtj d|d|tjdddd  }nPjj d kr|j}t d|| ddd  tj}|jj7}n)jj d kr|j}t |d|  tj}|d8}n tjj d tdjjd }t|t|t dt||}jjrj|jd}tfdd|D}t|dggtj}t|j|d}t|dd|dd d|d dg_t|}t|dd|dddg}|j|d_d_d_d_d_jd_dS)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. num_train_timesteps (`int`, *optional*): The number of diffusion steps used when training the model. If `None`, the default `num_train_timesteps` attribute is used. timesteps (`List[int]`, *optional*): Custom timesteps used to support arbitrary spacing between timesteps. If `None`, timesteps will be generated based on the `timestep_spacing` attribute. If `timesteps` is passed, `num_inference_steps` must be `None`, and `timestep_spacing` attribute will be ignored. NzEMust pass exactly one of `num_inference_steps` or `custom_timesteps`.zACan only pass one of `num_inference_steps` or `custom_timesteps`.z=Cannot use `timesteps` with `config.use_karras_sigmas = True`rr4rr leadingrZzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.rA) in_sigmasrjcg|]}|qSr) _sigma_to_t)r+rh log_sigmasrOrrr- z7HeunDiscreteScheduler.set_timesteps..g)rkrrD) rr[r;rSrjr5nparrayr r>r4copyarangeroundastyper?rIloginterp_convert_to_karras concatenater from_numpyrNcatrepeat_interleaverMrQprev_derivativedtrKrL)rOrjrkr5rQ step_ratiorMrrqrrJsR   (  *      0 $z#HeunDiscreteScheduler.set_timestepsc Cstt|d}||ddtjf}tj|dkddjddj|jddd}|d}||}||}||||} t| dd} d| || |} | |j} | S)Ng|=r)axisr)r\r ) rtrzmaximumnewaxiscumsumargmaxclipshapereshape) rOrhrr log_sigmadistslow_idxhigh_idxlowhighwrrrrrp4s, z!HeunDiscreteScheduler._sigma_to_trnc Cst|jdr |jj}nd}t|jdr|jj}nd}|dur |n|d}|dur,|n|d}d}tdd|}|d|}|d|}|||||} | S)z6Constructs the noise schedule of Karras et al. (2022). sigma_minN sigma_maxrlrg@r )hasattrr[rrrTrtr4) rOrnrjrrrhoramp min_inv_rho max_inv_rhorMrrrr|Ls      z(HeunDiscreteScheduler._convert_to_karrascCs |jduSN)rr]rrrstate_in_first_orderes z*HeunDiscreteScheduler.state_in_first_ordercCs@|jdurt|tjr||jj}|||_dS|j |_dSr) rc isinstancerTensorrNrQrkrYrKrL)rOrUrrrrgjs   z&HeunDiscreteScheduler._init_step_indexT model_output return_dictcCs|jdur |||jr|j|j}|j|jd}n|j|jd}|j|j}d}||d}|jjdkrF|jr=|n|} || |} n6|jjdkri|jrQ|n|} || | ddd|| dd} n|jjdkrr|} n td |jjd |jjr| |jj |jj } |jr|| |} ||} | |_ | |_ ||_ n|| |} |j | d} |j } |j }d|_ d|_ d|_ || | } |j d7_ |s| fSt| d S) a Predict the sample from the previous timestep by reversing the SDE. This function propagates the diffusion process from the learned model outputs (most often the predicted noise). Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`float`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or tuple. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. Nr rr2 v_predictionrrArezprediction_type given as z, must be one of `epsilon`, or `v_prediction`) prev_sample)r`rgrrMr[r:rr<clampr=rrrerKr )rOrrUrerrh sigma_nextgamma sigma_hat sigma_inputpred_original_sample derivativerrrrrsteprsX            zHeunDiscreteScheduler.steporiginal_samplesnoisecs jj|j|jd}|jjdkr)t|r)jj|jtjd|j|jtjd}n j|j||j}j durFfdd|D}nj durUj g|j d}n j g|j d}|| }t |j t |j kr}|d}t |j t |j ksn|||}|S)N)rkrmpsrcror)rY)r+rrVrOrrr-rsz3HeunDiscreteScheduler.add_noise..rrl)rMrNrkrtyperis_floating_pointrQr rcr`rflattenrS unsqueeze)rOrrrQrM step_indicesrh noisy_samplesrrr add_noises"      zHeunDiscreteScheduler.add_noisecCs|jjSr)r[r5r]rrr__len__szHeunDiscreteScheduler.__len__) r.r/r0r1Nr2FFr3r4rr)r)NNNN)T)&__name__ __module__ __qualname____doc__r _compatiblesorderrintfloatstrrrrtndarrayrboolrPrYpropertyr^r`rcrdrrrirkrJrpr|rrgr rrrrrrrrr)Fs$    )         S     d !r))r r)rtypingrrrrnumpyrtrconfiguration_utilsrrscheduling_utilsr r r r(r)rrrrs ,