o ۷i(@sddlZddlmZmZddlmZddlmZddlm Z ddl m Z m Z er5ddl mmZd Znd ZGd d d e ZdS) N) AutoencoderKLDiTTransformer2DModel)KarrasDiffusionSchedulers)is_torch_xla_available) randn_tensor)DiffusionPipelineImagePipelineOutputTFcseZdZdZdZ ddedededee e fdBffdd Z d e e e Bd e e fd d Z e     dde e dedeje ejBdBde de dBded eeBfddZZS) DiTPipelinea Pipeline for image generation based on a Transformer backbone instead of a UNet. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). Parameters: transformer ([`DiTTransformer2DModel`]): A class conditioned `DiTTransformer2DModel` to denoise the encoded image latents. Initially published as [`Transformer2DModel`](https://huggingface.co/facebook/DiT-XL-2-256/blob/main/transformer/config.json#L2) in the config, but the mismatch can be ignored. vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) model to encode and decode images to and from latent representations. scheduler ([`DDIMScheduler`]): A scheduler to be used in combination with `transformer` to denoise the encoded image latents. ztransformer->vaeN transformervae schedulerid2labelcs|t|j|||di|_|dur<|D]\}}|dD] }t||j|<q!qt t |j|_dSdS)N)r r r,) super__init__register_moduleslabelsitemssplitintlstriprstripdictsorted)selfr r rrkeyvaluelabel __class__Z/home/ubuntu/vllm_env/lib/python3.10/site-packages/diffusers/pipelines/dit/pipeline_dit.pyr:s zDiTPipeline.__init__rreturncsNt|ts t|}|D]}|jvrt|djdq fdd|DS)a0 Map label strings from ImageNet to corresponding class ids. Parameters: label (`str` or `dict` of `str`): Label strings to be mapped to class ids. Returns: `list` of `int`: Class ids to be processed by pipeline. zK does not exist. Please make sure to select one of the following labels: .csg|]}j|qSr")r).0lrr"r# csz-DiTPipeline.get_label_ids..) isinstancelistr ValueError)rrr'r"r(r# get_label_idsLs  zDiTPipeline.get_label_ids@2pilT class_labelsguidance_scale generatornum_inference_steps output_type return_dictcCs t|}|jjj}|jjj} t|| ||f||j|jjd} |dkr)t | gdn| } tj ||jd d}tj dg||jd} |dkrMt || gdn|} |j |||j jD]}|dkru| dt| d}tj ||gdd } |j | |} |}t|s| jjd k}| jjd k}t|tr|s|rtjntj}n |s|rtjntj}tj |g|| jd }nt|jdkr|d| j}|| jd}|j| || d j}|dkr|ddd| f|dd| df}}tj|t|ddd \}}||||}tj ||gdd }tj ||gdd }|jjjd| kr+tj|| dd \}}n|}|j ||| j!} t"r=t#$q\|dkrM| j%ddd \} }n| } d|j&jj'| } |j&(| j}|dd)dd}|*+dddd,}|dkr|-|}|.|s|fSt/|dS)a> The call function to the pipeline for generation. Args: class_labels (list[int]): list of ImageNet class labels for the images to be generated. guidance_scale (`float`, *optional*, defaults to 4.0): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. generator (`torch.Generator`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. num_inference_steps (`int`, *optional*, defaults to 250): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. Choose between `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`ImagePipelineOutput`] instead of a plain tuple. Examples: ```py >>> from diffusers import DiTPipeline, DPMSolverMultistepScheduler >>> import torch >>> pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256", torch_dtype=torch.float16) >>> pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config) >>> pipe = pipe.to("cuda") >>> # pick words from Imagenet class labels >>> pipe.labels # to print all available words >>> # pick words that exist in ImageNet >>> words = ["white shark", "umbrella"] >>> class_ids = pipe.get_label_ids(words) >>> generator = torch.manual_seed(33) >>> output = pipe(class_labels=class_ids, num_inference_steps=25, generator=generator) >>> image = output.images[0] # label 'white shark' ``` Returns: [`~pipelines.ImagePipelineOutput`] or `tuple`: If `return_dict` is `True`, [`~pipelines.ImagePipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated images )shaper3devicedtyper)r8irN)dimmpsnpu)r9r8)timestepr1g?rr0)images)0lenr config sample_size in_channelsr_execution_devicer9torchcattensorreshaper set_timesteps progress_bar timestepsscale_model_input is_tensorr8typer*floatfloat32float64int32int64r7toexpandsampler out_channelsstep prev_sample XLA_AVAILABLExm mark_stepchunkr scaling_factordecodeclampcpupermutenumpy numpy_to_pilmaybe_free_model_hooksr )rr1r2r3r4r5r6 batch_size latent_sizelatent_channelslatentslatent_model_input class_nullclass_labels_inputthalfrLis_mpsis_npur9 noise_predepsrestcond_eps uncond_epshalf_eps model_output_samplesr"r"r#__call__esz<         *    zDiTPipeline.__call__)N)r.Nr/r0T)__name__ __module__ __qualname____doc__model_cpu_offload_seqrrrrrstrrr+r-rFno_gradrP Generatorboolr tupler{ __classcell__r"r"r r#r &sFr )rFmodelsrr schedulersrutilsrutils.torch_utilsrpipeline_utilsr r torch_xla.core.xla_modelcore xla_modelr\r[r r"r"r"r#s