o Gi7@sddlmZddlZddlmZmZddlmZddlm Z m Z m Z m Z ddl mZdd lmZmZe rAddlmmZd Znd Ze eZd ZdddZGdddeZdS))CallableN)UNet2DConditionModelVQModel) DDPMScheduler) deprecateis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)DiffusionPipelineImagePipelineOutputTFae Examples: ```py >>> from diffusers import KandinskyV22Pipeline, KandinskyV22PriorPipeline >>> import torch >>> pipe_prior = KandinskyV22PriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-prior") >>> pipe_prior.to("cuda") >>> prompt = "red cat, 4k photo" >>> out = pipe_prior(prompt) >>> image_emb = out.image_embeds >>> zero_image_emb = out.negative_image_embeds >>> pipe = KandinskyV22Pipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder") >>> pipe.to("cuda") >>> image = pipe( ... image_embeds=image_emb, ... negative_image_embeds=zero_image_emb, ... height=768, ... width=768, ... num_inference_steps=50, ... ).images >>> image[0].save("cat.png") ``` cCsX||d}||ddkr|d7}||d}||ddkr$|d7}||||fS)Nr r)heightwidth scale_factor new_height new_widthrrj/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/pipelines/kandinsky2_2/pipeline_kandinsky2_2.pydownscale_height_and_width>s  rcseZdZdZdZgdZdededeffdd Z d d Z e d d Z e d dZ e ddZeeedddddddddddgf dejeejBdejeejBdededededed ejeejBdBdejdBd!edBd"ed#eeegdfdBd$eefd%d&ZZS)'KandinskyV22Pipelinea Pipeline for text-to-image generation using Kandinsky This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: scheduler (`DDIMScheduler` | `DDPMScheduler`): A scheduler to be used in combination with `unet` to generate image latents. unet ([`UNet2DConditionModel`]): Conditional U-Net architecture to denoise the image embedding. movq ([`VQModel`]): MoVQ Decoder to generate the image from the latents. z unet->movq)latents image_embedsnegative_image_embedsunet schedulermovqcs6t|j|||ddt|jjjd|_dS)N)rrrr r)super__init__register_moduleslenrconfigblock_out_channelsmovq_scale_factor)selfrrr __class__rrr![s zKandinskyV22Pipeline.__init__cCsR|dur t||||d}n|j|krtd|jd|||}||j}|S)N) generatordevicedtypezUnexpected latents shape, got z , expected )r shape ValueErrortoinit_noise_sigma)r'r-r,r+r*rrrrrprepare_latentsks   z$KandinskyV22Pipeline.prepare_latentscC|jSN_guidance_scaler'rrrguidance_scalevz#KandinskyV22Pipeline.guidance_scalecCs |jdkS)Nrr4r6rrrdo_classifier_free_guidancezs z0KandinskyV22Pipeline.do_classifier_free_guidancecCr2r3)_num_timestepsr6rrr num_timesteps~r8z"KandinskyV22Pipeline.num_timestepsidg@rNpilTrrrrrnum_inference_stepsr7num_images_per_promptr* output_type return_dictcallback_on_step_end"callback_on_step_end_tensor_inputsc$ s|dd}|dd}|durtddd|dur tddd| durAtfdd| DsAtd jd fd d | Dj}|_t|trSt j |d d}|j d |}t|trft j |d d}j r|j |d d}|j |d d}t j ||gd djjj|d}jj||djj}jjj}t||j\}}||||f|j||| j} t|_t|D]\}}j rt | gdn| }d|i}j||d|ddd }j r|j| j ddd\}}|d\}}|d\}}|j||}t j ||gdd}t jjdrjjj!dvs*|j| j ddd\}}jj"||| |dd } | durci}| D] } t#| || <q?| |||}!|!d| } |!d|}|!d|}|dur~||d kr~|t$jdd}"||"|| t%rt&'q| dvrtd| | dksˈj(j)| dd d!}#| d"vr|#d#d#}#|#*d d}#|#+,d dd$d-.}#| d%krʈ/|#}#n| }#0| s|#fSt1|#d&S)'a Function invoked when calling the pipeline for generation. Args: image_embeds (`torch.Tensor` or `list[torch.Tensor]`): The clip image embeddings for text prompt, that will be used to condition the image generation. negative_image_embeds (`torch.Tensor` or `list[torch.Tensor]`): The clip image embeddings for negative text prompt, will be used to condition the image generation. height (`int`, *optional*, defaults to 512): The height in pixels of the generated image. width (`int`, *optional*, defaults to 512): The width in pixels of the generated image. num_inference_steps (`int`, *optional*, defaults to 100): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*, defaults to 4.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. generator (`torch.Generator` or `list[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will be generated by sampling using the supplied random `generator`. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generate image. Choose between: `"pil"` (`PIL.Image.Image`), `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`list`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` callbackNcallback_stepsz1.0.0zhPassing `callback` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`znPassing `callback_steps` as an input argument to `__call__` is deprecated, consider use `callback_on_step_end`c3s|]}|jvVqdSr3_callback_tensor_inputs.0kr6rr s  z0KandinskyV22Pipeline.__call__..z2`callback_on_step_end_tensor_inputs` has to be in z , but found csg|] }|jvr|qSrrFrHr6rr sz1KandinskyV22Pipeline.__call__..r)dim)r,r+)r+r rF)sampletimestepencoder_hidden_statesadded_cond_kwargsrAr variance_type)learned learned_range)r*rrorder)ptnpr=latentzIOnly the output types `pt`, `pil` and `np` are supported not output_type=rXT)force_not_quantizerN)rWr=g?rr=)images)2poprallr.rG_execution_devicer5 isinstancelisttorchcatr-r9repeat_interleaver/rr,r set_timesteps timestepsr$ in_channelsrr&r1r#r: enumerate progress_barsplitchunkr7hasattrrRsteplocalsgetattr XLA_AVAILABLExm mark_steprdecodeclampcpupermutefloatnumpy numpy_to_pilmaybe_free_model_hooksr)$r'rrrrr>r7r?r*rr@rArBrCkwargsrDrEr+ batch_sizerdnum_channels_latentsitlatent_model_inputrQ noise_pred variance_prednoise_pred_uncondnoise_pred_text_variance_pred_textcallback_kwargsrJcallback_outputsstep_idximagerr6r__call__s F                    zKandinskyV22Pipeline.__call__)__name__ __module__ __qualname____doc__model_cpu_offload_seqrGrrrr!r1propertyr7r9r;r`no_gradr EXAMPLE_DOC_STRINGTensorr_intru Generatorstrboolrr __classcell__rrr(rrHst        r)r)typingrr`modelsrr schedulersrutilsrrr r utils.torch_utilsr pipeline_utilsr rtorch_xla.core.xla_modelcore xla_modelrorn get_loggerrloggerrrrrrrrs