o pi^@sddlZddlmZmZmZmZddlZddlZddl Z ddl m Z m Z ddl mZmZmZGdddZGd d d Z dd dZGdddee ZdS)N)ListOptionalTupleUnion) ConfigMixinregister_to_config)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputc@s.eZdZdZd ddZeddZddZdS) BatchedBrownianTreezGA wrapper around torchsde.BrownianTree that enables batches of entropy.Nc s|\|_dt||dur tddg}d|_zt||j dks/JdWnt yC|g}d|_Ynwfdd|D|_ dS)Nw0rlTFcs&g|]}tjfd|iqS)entropy)torchsde BrownianTree).0skwargst0t1rk/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/schedulers/scheduling_dpmsolver_sde.py )s&z0BatchedBrownianTree.__init__..) sortsigngettorch zeros_likerandintitembatchedlenshape TypeErrortrees)selfxrrseedrrrr__init__s   zBatchedBrownianTree.__init__cCs||kr ||dfS||dfS)Nr r)abrrrr+szBatchedBrownianTree.sortcsJ|\}tfdd|jD|j|}|jr!|S|dS)Ncsg|]}|qSrr)rtreerrrrr1sz0BatchedBrownianTree.__call__..r)rrstackr&rr")r'rrrwrr/r__call__/s&zBatchedBrownianTree.__call__N)__name__ __module__ __qualname____doc__r* staticmethodrr2rrrrr s    r c@s*eZdZdZdddfddZddZdS) BrownianTreeNoiseSamplerasA noise sampler backed by a torchsde.BrownianTree. Args: x (Tensor): The tensor whose shape, device and dtype to use to generate random samples. sigma_min (float): The low end of the valid interval. sigma_max (float): The high end of the valid interval. seed (int or List[int]): The random seed. If a list of seeds is supplied instead of a single integer, then the noise sampler will use one BrownianTree per batch item, each with its own seed. transform (callable): A function that maps sigma to the sampler's internal timestep. NcCs|Sr3r)r(rrrDsz!BrownianTreeNoiseSampler.cCs<||_|t||t|}}t|||||_dSr3) transformr as_tensorr r.)r'r( sigma_min sigma_maxr)r;rrrrrr*Ds"z!BrownianTreeNoiseSampler.__init__cCs>|t||t|}}|||||Sr3)r;rr<r.abssqrt)r'sigma sigma_nextrrrrrr2Is"z!BrownianTreeNoiseSampler.__call__)r4r5r6r7r*r2rrrrr95s r9+?cosinecCs|dkr dd}n|dkrdd}ntd|g}t|D]}||}|d|}|td|||||qtj|tjdS) a Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): the number of betas to produce. max_beta (`float`): the maximum beta to use; use values lower than 1 to prevent singularities. alpha_transform_type (`str`, *optional*, default to `cosine`): the type of noise schedule for alpha_bar. Choose from `cosine` or `exp` Returns: betas (`np.ndarray`): the betas used by the scheduler to step the model outputs rDcSs t|ddtjddS)NgMb?gT㥛 ?r)mathcospitrrr alpha_bar_fnhs z)betas_for_alpha_bar..alpha_bar_fnexpcSst|dS)Ng()rErKrHrrrrJmsz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendminrtensorfloat32)num_diffusion_timestepsmax_betaalpha_transform_typerJbetasirt2rrrbetas_for_alpha_barOs    "rZc@seZdZdZddeDZdZe     dGdede de de de e e jee fde de ede ede defddZdHddZddZeddZed d!Zed"d#ZdId$efd%d&Zd'ejd(e e ejfd)ejfd*d+Z dJd,ed-e e ejfde efd.d/Zd0d1Zd2d3Zd4ejd)ejfd5d6Z ed7d8Z! 9 :dKd;e eje jfd(e e ejfd'e eje jfdd?Z$d@ejdAejdBejd)ejfdCdDZ%dEdFZ&d S)LDPMSolverSDEScheduleru DPMSolverSDEScheduler implements the stochastic sampler from the [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364) paper. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.00085): The starting `beta` value of inference. beta_end (`float`, defaults to 0.012): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear` or `scaled_linear`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://imagen.research.google/video/paper.pdf) paper). use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. noise_sampler_seed (`int`, *optional*, defaults to `None`): The random seed to use for the noise sampler. If `None`, a random seed is generated. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. cCsg|]}|jqSr)name)rerrrrsz DPMSolverSDEScheduler.r_QK?~jt?linearNepsilonFlinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betasprediction_typeuse_karras_sigmasnoise_sampler_seedtimestep_spacing steps_offsetc Cs|durtj|tjd|_n:|dkrtj|||tjd|_n*|dkr4tj|d|d|tjdd|_n|dkr>t||_n t|d|jd|j|_tj |jd d |_ | |d|||_ d|_ ||_d|_d|_|jd |_dS) NrLra scaled_linear?rsquaredcos_cap_v2z is not implemented for ?r)dimcpu)rrRrSrWrcrZNotImplementedError __class__alphascumprodalphas_cumprod set_timestepsrj noise_samplerrk _step_index _begin_indexsigmasto) r'rdrerfrgrhrirjrkrlrmrrrr*s$$  zDPMSolverSDEScheduler.__init__cCs:|dur|j}||k}t|dkrdnd}||S)Nr r) timestepsnonzeror#r!)r'timestepschedule_timestepsindicesposrrrindex_for_timesteps   z(DPMSolverSDEScheduler.index_for_timestepcCs@|jdurt|tjr||jj}|||_dS|j |_dSr3) begin_index isinstancerTensorr~rdevicerr{r|)r'rrrr_init_step_indexs   z&DPMSolverSDEScheduler._init_step_indexcCs,|jjdvr |jS|jdddS)N)rctrailingrr ro)configrlr}maxr'rrrinit_noise_sigmas  z&DPMSolverSDEScheduler.init_noise_sigmacC|jS)zg The index counter for current timestep. It will increase 1 after each scheduler step. )r{rrrr step_indexz DPMSolverSDEScheduler.step_indexcCr)zq The index for the first timestep. It should be set from pipeline with `set_begin_index` method. r|rrrrrrz!DPMSolverSDEScheduler.begin_indexrcCs ||_dS)z Sets the begin index for the scheduler. This function should be run from pipeline before the inference. Args: begin_index (`int`): The begin index for the scheduler. Nr)r'rrrrset_begin_indexs z%DPMSolverSDEScheduler.set_begin_indexsamplerreturncCsH|jdur |||j|j}|jr|n|j}||ddd}|S)a Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. timestep (`int`, *optional*): The current timestep in the diffusion chain. Returns: `torch.Tensor`: A scaled input sample. Nrr ro)rrr}state_in_first_ordermid_point_sigma)r'rrrA sigma_inputrrrscale_model_inputs   z'DPMSolverSDEScheduler.scale_model_inputnum_inference_stepsrcsZ|_|pjj}jjdkr"tjd|d|tdddd}nNjjdkrH|j}td|| ddd t}|jj 7}n(jjdkrg|j}t|d|   t}|d8}n t jjd t djjd }t|t|tdt||}jjrj|d }t fd d |D}|}t|dgg tj}t|j|d}t|dd|ddd|ddg_t|}t|}t|dd|dddg}||ddd<t|dr|j|tjd_n|j|d_d_ d_!d_"d_#jd_d_$dS)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. rcrr rLNr+leadingrzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.ro) in_sigmascg|]}|qSr _sigma_to_trrA log_sigmasr'rrrJz7DPMSolverSDEScheduler.set_timesteps..g)rrmpsrs)%rrrdrlnprcfloatcopyarangeroundastypermrNarrayrxloginterpr#rj_convert_to_karras_second_order_timesteps concatenaterSr from_numpyr~catrepeat_interleaver}str startswithrrrr{r|rz)r'rrrdr step_ratior}second_order_timestepsrrrrysL  &  (       0  $ z#DPMSolverSDEScheduler.set_timestepsc s`dd}dd}d}||}t|}|dd||}||} tfdd| D} | S) NcSs t| Sr3)rrK_trrrsigma_fng z?DPMSolverSDEScheduler._second_order_timesteps..sigma_fncSs t| Sr3)rr_sigmarrrt_fnjrz;DPMSolverSDEScheduler._second_order_timesteps..t_fnror+crrrrrrrrrrzADPMSolverSDEScheduler._second_order_timesteps..)rdiffr) r'r}rrrmidpoint_ratiorI delta_time t_proposed sig_proposedrrrrrfs z-DPMSolverSDEScheduler._second_order_timestepsc Cstt|d}||ddtjf}tj|dkddjddj|jddd}|d}||}||}||||} t| dd} d| || |} | |j} | S)Ng|=r)axisr)rr ) rrmaximumnewaxiscumsumargmaxclipr$reshape) r'rAr log_sigmadistslow_idxhigh_idxlowhighr1rIrrrrvs, z!DPMSolverSDEScheduler._sigma_to_trc Cs\|d}|d}d}tdd|j}|d|}|d|}|||||}|S)z6Constructs the noise schedule of Karras et al. (2022).r+rg@r )r!rrcr) r'rr=r>rhoramp min_inv_rho max_inv_rhor}rrrrs    z(DPMSolverSDEScheduler._convert_to_karrascCs |jduSr3)rrrrrrs z*DPMSolverSDEScheduler.state_in_first_orderTrq model_output return_dicts_noisecCs|jdur |||jdur(|j|jdk|j}}t||||j|_dtj dtj fdd}dtj dtj fdd } |j rP|j|j} |j|jd } n|j|jd } |j|j} d } | | | | } }|| }| || }|j j d kr|j r~| n||}|||}n9|j j d kr|j r| n||}|| |dd d ||dd }n|j j dkrt dtd|j j d| dkr||| }| | }|||}ni|j r|}n|j}|| }||}t||d|d|d|dd }|d|dd }| |}|||| || ||}|||| ||||}|j r:||_|||_nd|_d|_|jd 7_|sM|fSt|dS)a Predict the sample from the previous timestep by reversing the SDE. This function propagates the diffusion process from the learned model outputs (most often the predicted noise). Args: model_output (`torch.Tensor` or `np.ndarray`): The direct output from learned diffusion model. timestep (`float` or `torch.Tensor`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor` or `np.ndarray`): A current instance of a sample created by the diffusion process. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or tuple. s_noise (`float`, *optional*, defaults to 1.0): Scaling factor for noise added to the sample. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. NrrrcS |Sr3)negrKrrrrrrz,DPMSolverSDEScheduler.step..sigma_fnrcSrr3)rrrrrrrrz(DPMSolverSDEScheduler.step..t_fnr rorb v_predictionrrz+prediction_type not implemented yet: samplezprediction_type given as z, must be one of `epsilon`, or `v_prediction`) prev_sample)rrrzr}rQrr9rkrrrrrirtrNrexpm1rr{r )r'rrrrr min_sigma max_sigmarrrArBrrIt_nextrrrpred_original_sample derivativedtr sigma_fromsigma_tosigma_up sigma_down ancestral_trrrstepsn           *   zDPMSolverSDEScheduler.steporiginal_samplesnoisercs jj|j|jd}|jjdkr)t|r)jj|jtjd|j|jtjd}n j|j||j}j durFfdd|D}nj durUj g|j d}n j g|j d}|| }t |j t |j kr}|d}t |j t |j ksn|||}|S)N)rrMrrLcrr)r)rrIrr'rrr!rz3DPMSolverSDEScheduler.add_noise..rr+)r}r~rrMtyperis_floating_pointrrSrrr$flattenr# unsqueeze)r'rrrr} step_indicesrA noisy_samplesrrr add_noises"      zDPMSolverSDEScheduler.add_noisecCs|jjSr3)rrdrrrr__len__0szDPMSolverSDEScheduler.__len__) r^r_r`raNrbFNrcrr3)r)NN)Trq)'r4r5r6r7r _compatiblesorderrintrrrrrndarrayrboolr*rrpropertyrrrrrrrrryrrrrr rrrrrrrrr[{s#   '        I    p !r[)rCrD)rEtypingrrrrnumpyrrrconfiguration_utilsrrscheduling_utilsr r r r r9rZr[rrrrs ,