o pi&@spddlmZmZmZmZmZddlZddlmZm Z ddl m Z ddl m Z ddlmZmZGd d d eZdS) )DictListOptionalTupleUnionN) AutoencoderKLDiTTransformer2DModel)KarrasDiffusionSchedulers) randn_tensor)DiffusionPipelineImagePipelineOutputcseZdZdZdZ ddedededee e e fffdd Z d e e ee fd ee fd d Ze     ddee dedee ejeejfde dee ded e eeffddZZS) DiTPipelinea Pipeline for image generation based on a Transformer backbone instead of a UNet. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). Parameters: transformer ([`DiTTransformer2DModel`]): A class conditioned `DiTTransformer2DModel` to denoise the encoded image latents. vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) model to encode and decode images to and from latent representations. scheduler ([`DDIMScheduler`]): A scheduler to be used in combination with `transformer` to denoise the encoded image latents. ztransformer->vaeN transformervae schedulerid2labelcs|t|j|||di|_|dur<|D]\}}|dD] }t||j|<q!qt t |j|_dSdS)N)rrr,) super__init__register_moduleslabelsitemssplitintlstriprstripdictsorted)selfrrrrkeyvaluelabel __class__b/home/ubuntu/SoloSpeech/.venv/lib/python3.10/site-packages/diffusers/pipelines/dit/pipeline_dit.pyr1s zDiTPipeline.__init__r#returncsNt|ts t|}|D]}|jvrt|djdq fdd|DS)a0 Map label strings from ImageNet to corresponding class ids. Parameters: label (`str` or `dict` of `str`): Label strings to be mapped to class ids. Returns: `list` of `int`: Class ids to be processed by pipeline. zK does not exist. Please make sure to select one of the following labels: .csg|]}j|qSr&)r).0lr r&r' Zsz-DiTPipeline.get_label_ids..) isinstancelistr ValueError)r r#r+r&r,r' get_label_idsCs  zDiTPipeline.get_label_ids@2pilT class_labelsguidance_scale generatornum_inference_steps output_type return_dictcCst|}|jjj}|jjj} t|| ||f||j|jjd} |dkr)t | gdn| } tj ||jd d}tj dg||jd} |dkrMt || gdn|} |j |||j jD]}|dkru| dt| d}tj ||gdd } |j | |} |}t|s| jjd k}t|tr|rtjntj}n|rtjntj}tj |g|| jd }nt|jdkr|d| j}|| jd}|j| || d j}|dkr |ddd| f|dd| df}}tj|t|ddd \}}||||}tj ||gdd }tj ||gdd }|jjjd| kr!tj|| dd \}}n|}|j ||| j!} q\|dkr<| j"ddd \} }n| } d|j#jj$| } |j#%| j}|dd &dd}|'(dddd)}|dkrp|*|}|+|sz|fSt,|dS)a> The call function to the pipeline for generation. Args: class_labels (List[int]): List of ImageNet class labels for the images to be generated. guidance_scale (`float`, *optional*, defaults to 4.0): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. generator (`torch.Generator`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. num_inference_steps (`int`, *optional*, defaults to 250): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. Choose between `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`ImagePipelineOutput`] instead of a plain tuple. Examples: ```py >>> from diffusers import DiTPipeline, DPMSolverMultistepScheduler >>> import torch >>> pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", torch_dtype=torch.float16) >>> pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) >>> pipe = pipe.to("cuda") >>> # pick words from Imagenet class labels >>> pipe.labels # to print all available words >>> # pick words that exist in ImageNet >>> words = ["white shark", "umbrella"] >>> class_ids = pipe.get_label_ids(words) >>> generator = torch.manual_seed(33) >>> output = pipe(class_labels=class_ids, num_inference_steps=25, generator=generator) >>> image = output.images[0] # label 'white shark' ``` Returns: [`~pipelines.ImagePipelineOutput`] or `tuple`: If `return_dict` is `True`, [`~pipelines.ImagePipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated images )shaper7devicedtyper )r<irN)dimmps)r=r<)timestepr5g?rr4)images)-lenrconfig sample_size in_channelsr _execution_devicer=torchcattensorreshaper set_timesteps progress_bar timestepsscale_model_input is_tensorr<typer.floatfloat32float64int32int64r;toexpandsampler out_channelsstep prev_samplechunkrscaling_factordecodeclampcpupermutenumpy numpy_to_pilmaybe_free_model_hooksr)r r5r6r7r8r9r: batch_size latent_sizelatent_channelslatentslatent_model_input class_nullclass_labels_inputthalfrOis_mpsr= noise_predepsrestcond_eps uncond_epshalf_eps model_output_samplesr&r&r'__call__\sr<        *    zDiTPipeline.__call__)N)r2Nr3r4T)__name__ __module__ __qualname____doc__model_cpu_offload_seqr rr rrrstrrrrr1rIno_gradrS Generatorboolrrrz __classcell__r&r&r$r'rsF" r)typingrrrrrrImodelsrr schedulersr utils.torch_utilsr pipeline_utilsr rrr&r&r&r's