o GÆÏiXã@sJddlmZddlmZmZddlmZmZddlm Z m Z m Z m Z m Z ddlmZddlmZdd lmZmZmZdd lmZmZe e¡ZGd d „d eƒZGd d„deƒZGdd„deƒZGdd„deƒZGdd„deƒZ Gdd„deƒZ!Gdd„deƒZ"Gdd„deƒZ#edeƒfdeƒfde#ƒfdeƒfgƒZ$Gdd „d eƒZ%d!S)"é)Úloggingé)ÚAutoPipelineBlocksÚSequentialPipelineBlocks)ÚInsertableDictÚ OutputParamé)ÚFluxImg2ImgPrepareLatentsStepÚFluxImg2ImgSetTimestepsStepÚFluxPrepareLatentsStepÚFluxRoPEInputsStepÚFluxSetTimestepsStep)ÚFluxDecodeStep)ÚFluxDenoiseStep)ÚFluxProcessImagesInputStepÚFluxTextEncoderStepÚFluxVaeEncoderStep)ÚFluxAdditionalInputsStepÚFluxTextInputStepc@s:eZdZdZdZeƒeƒgZddgZe de fdd„ƒZ dS) ÚFluxImg2ImgVaeEncoderStepa  Vae encoder step that preprocess andencode the image inputs into their latent representations. Components: image_processor (`VaeImageProcessor`) vae (`AutoencoderKL`) Inputs: resized_image (`None`, *optional*): TODO: Add description. image (`None`, *optional*): TODO: Add description. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. generator (`None`, *optional*): TODO: Add description. Outputs: processed_image (`None`): TODO: Add description. image_latents (`Tensor`): The latents representing the reference image ÚfluxÚ preprocessÚencodeÚreturncCódS)Nz^Vae encoder step that preprocess andencode the image inputs into their latent representations.©©Úselfrrúh/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/modular_pipelines/flux/modular_blocks_flux.pyÚ descriptionLóz%FluxImg2ImgVaeEncoderStep.descriptionN) Ú__name__Ú __module__Ú __qualname__Ú__doc__Ú model_namerrÚ block_classesÚ block_namesÚpropertyÚstrrrrrrr-s rc@s2eZdZdZdZegZdgZdgZe dd„ƒZ dS)ÚFluxAutoVaeEncoderStepaÔ Vae encoder step that encode the image inputs into their latent representations. This is an auto pipeline block that works for img2img tasks. - `FluxImg2ImgVaeEncoderStep` (img2img) is used when only `image` is provided. - if `image` is not provided, step will be skipped. Components: image_processor (`VaeImageProcessor`) vae (`AutoencoderKL`) Inputs: resized_image (`None`, *optional*): TODO: Add description. image (`None`, *optional*): TODO: Add description. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. generator (`None`, *optional*): TODO: Add description. Outputs: processed_image (`None`): TODO: Add description. image_latents (`Tensor`): The latents representing the reference image rÚimg2imgÚimagecCó dS)NaVae encoder step that encode the image inputs into their latent representations. This is an auto pipeline block that works for img2img tasks. - `FluxImg2ImgVaeEncoderStep` (img2img) is used when only `image` is provided. - if `image` is not provided, step will be skipped.rrrrrrtóÿz"FluxAutoVaeEncoderStep.descriptionN) r!r"r#r$r%rr&r'Úblock_trigger_inputsr(rrrrrr*Rsr*c@s8eZdZdZdZeƒeƒeƒgZgd¢Z e dd„ƒZ dS)ÚFluxBeforeDenoiseStepa— Before denoise step that prepares the inputs for the denoise step in text-to-image generation. Components: scheduler (`FlowMatchEulerDiscreteScheduler`) Inputs: height (`int`, *optional*): TODO: Add description. width (`int`, *optional*): TODO: Add description. latents (`Tensor | NoneType`, *optional*): TODO: Add description. num_images_per_prompt (`int`, *optional*, defaults to 1): TODO: Add description. generator (`None`, *optional*): TODO: Add description. batch_size (`int`): Number of prompts, the final batch size of model inputs should be `batch_size * num_images_per_prompt`. Can be generated in input step. dtype (`dtype`, *optional*): The dtype of the model inputs num_inference_steps (`None`, *optional*, defaults to 50): TODO: Add description. timesteps (`None`, *optional*): TODO: Add description. sigmas (`None`, *optional*): TODO: Add description. guidance_scale (`None`, *optional*, defaults to 3.5): TODO: Add description. prompt_embeds (`None`, *optional*): TODO: Add description. Outputs: latents (`Tensor`): The initial latents to use for the denoising process timesteps (`Tensor`): The timesteps to use for inference num_inference_steps (`int`): The number of denoising steps to perform at inference time guidance (`Tensor`): Optional guidance to be used. txt_ids (`list`): The sequence lengths of the prompt embeds, used for RoPE calculation. img_ids (`list`): The sequence lengths of the image latents, used for RoPE calculation. r)Úprepare_latentsÚ set_timestepsÚprepare_rope_inputscCr)Nz^Before denoise step that prepares the inputs for the denoise step in text-to-image generation.rrrrrrµr z!FluxBeforeDenoiseStep.descriptionN) r!r"r#r$r%r r r r&r'r(rrrrrr0€s0r0c@s<eZdZdZdZeƒeƒeƒeƒgZ gd¢Z e dd„ƒZ dS)ÚFluxImg2ImgBeforeDenoiseStepa Before denoise step that prepare the inputs for the denoise step for img2img task. Components: scheduler (`FlowMatchEulerDiscreteScheduler`) Inputs: height (`int`, *optional*): TODO: Add description. width (`int`, *optional*): TODO: Add description. latents (`Tensor | NoneType`, *optional*): TODO: Add description. num_images_per_prompt (`int`, *optional*, defaults to 1): TODO: Add description. generator (`None`, *optional*): TODO: Add description. batch_size (`int`): Number of prompts, the final batch size of model inputs should be `batch_size * num_images_per_prompt`. Can be generated in input step. dtype (`dtype`, *optional*): The dtype of the model inputs num_inference_steps (`None`, *optional*, defaults to 50): TODO: Add description. timesteps (`None`, *optional*): TODO: Add description. sigmas (`None`, *optional*): TODO: Add description. strength (`None`, *optional*, defaults to 0.6): TODO: Add description. guidance_scale (`None`, *optional*, defaults to 3.5): TODO: Add description. image_latents (`Tensor`): The image latents to use for the denoising process. Can be generated in vae encoder and packed in input step. prompt_embeds (`None`, *optional*): TODO: Add description. Outputs: latents (`Tensor`): The initial latents to use for the denoising process timesteps (`Tensor`): The timesteps to use for inference num_inference_steps (`int`): The number of denoising steps to perform at inference time guidance (`Tensor`): Optional guidance to be used. initial_noise (`Tensor`): The initial random noised used for inpainting denoising. txt_ids (`list`): The sequence lengths of the prompt embeds, used for RoPE calculation. img_ids (`list`): The sequence lengths of the image latents, used for RoPE calculation. r)r1r2Úprepare_img2img_latentsr3cCr)NzRBefore denoise step that prepare the inputs for the denoise step for img2img task.rrrrrrýr z(FluxImg2ImgBeforeDenoiseStep.descriptionN) r!r"r#r$r%r r r r r&r'r(rrrrrr4¼s7ür4c@ó8eZdZdZdZeegZddgZddgZ e dd„ƒZ dS) ÚFluxAutoBeforeDenoiseStepa¾ Before denoise step that prepare the inputs for the denoise step. This is an auto pipeline block that works for text2image. - `FluxBeforeDenoiseStep` (text2image) is used. - `FluxImg2ImgBeforeDenoiseStep` (img2img) is used when only `image_latents` is provided. Components: scheduler (`FlowMatchEulerDiscreteScheduler`) Inputs: height (`int`): TODO: Add description. width (`int`): TODO: Add description. latents (`Tensor | NoneType`, *optional*): TODO: Add description. num_images_per_prompt (`int`, *optional*, defaults to 1): TODO: Add description. generator (`None`, *optional*): TODO: Add description. batch_size (`int`): Number of prompts, the final batch size of model inputs should be `batch_size * num_images_per_prompt`. Can be generated in input step. dtype (`dtype`, *optional*): The dtype of the model inputs num_inference_steps (`None`, *optional*, defaults to 50): TODO: Add description. timesteps (`None`, *optional*): TODO: Add description. sigmas (`None`, *optional*): TODO: Add description. strength (`None`, *optional*, defaults to 0.6): TODO: Add description. guidance_scale (`None`, *optional*, defaults to 3.5): TODO: Add description. image_latents (`Tensor`, *optional*): The image latents to use for the denoising process. Can be generated in vae encoder and packed in input step. prompt_embeds (`None`, *optional*): TODO: Add description. Outputs: latents (`Tensor`): The initial latents to use for the denoising process timesteps (`Tensor`): The timesteps to use for inference num_inference_steps (`int`): The number of denoising steps to perform at inference time guidance (`Tensor`): Optional guidance to be used. initial_noise (`Tensor`): The initial random noised used for inpainting denoising. txt_ids (`list`): The sequence lengths of the prompt embeds, used for RoPE calculation. img_ids (`list`): The sequence lengths of the image latents, used for RoPE calculation. rr+Ú text2imageÚ image_latentsNcCr-)NaBefore denoise step that prepare the inputs for the denoise step. This is an auto pipeline block that works for text2image. - `FluxBeforeDenoiseStep` (text2image) is used. - `FluxImg2ImgBeforeDenoiseStep` (img2img) is used when only `image_latents` is provided. rrrrrrDr.z%FluxAutoBeforeDenoiseStep.description) r!r"r#r$r%r4r0r&r'r/r(rrrrrr7s:r7c@s4eZdZdZdZeƒeƒgZddgZe dd„ƒZ dS)ÚFluxImg2ImgInputStepa? Input step that prepares the inputs for the img2img denoising step. It: Inputs: num_images_per_prompt (`None`, *optional*, defaults to 1): TODO: Add description. prompt_embeds (`Tensor`): Pre-generated text embeddings. Can be generated from text_encoder step. pooled_prompt_embeds (`Tensor`, *optional*): Pre-generated pooled text embeddings. Can be generated from text_encoder step. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. image_latents (`None`, *optional*): TODO: Add description. Outputs: batch_size (`int`): Number of prompts, the final batch size of model inputs should be batch_size * num_images_per_prompt dtype (`dtype`): Data type of model tensor inputs (determined by `prompt_embeds`) prompt_embeds (`Tensor`): text embeddings used to guide the image generation pooled_prompt_embeds (`Tensor`): pooled text embeddings used to guide the image generation image_height (`int`): The height of the image latents image_width (`int`): The width of the image latents rÚ text_inputsÚadditional_inputscCr)NzHInput step that prepares the inputs for the img2img denoising step. It: rrrrrrwr z FluxImg2ImgInputStep.descriptionN) r!r"r#r$r%rrr&r'r(rrrrrr:Rs r:c@r6) ÚFluxAutoInputStepax Input step that standardize the inputs for the denoising step, e.g. make sure inputs have consistent batch size, and patchified. This is an auto pipeline block that works for text2image/img2img tasks. - `FluxImg2ImgInputStep` (img2img) is used when `image_latents` is provided. - `FluxTextInputStep` (text2image) is used when `image_latents` are not provided. Inputs: num_images_per_prompt (`None`, *optional*, defaults to 1): TODO: Add description. prompt_embeds (`Tensor`): Pre-generated text embeddings. Can be generated from text_encoder step. pooled_prompt_embeds (`Tensor`, *optional*): Pre-generated pooled text embeddings. Can be generated from text_encoder step. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. image_latents (`None`, *optional*): TODO: Add description. Outputs: batch_size (`int`): Number of prompts, the final batch size of model inputs should be batch_size * num_images_per_prompt dtype (`dtype`): Data type of model tensor inputs (determined by `prompt_embeds`) prompt_embeds (`Tensor`): text embeddings used to guide the image generation pooled_prompt_embeds (`Tensor`): pooled text embeddings used to guide the image generation image_height (`int`): The height of the image latents image_width (`int`): The width of the image latents rr+r8r9NcCr-)NalInput step that standardize the inputs for the denoising step, e.g. make sure inputs have consistent batch size, and patchified. This is an auto pipeline block that works for text2image/img2img tasks. - `FluxImg2ImgInputStep` (img2img) is used when `image_latents` is provided. - `FluxTextInputStep` (text2image) is used when `image_latents` are not provided. rrrrrrªr.zFluxAutoInputStep.description) r!r"r#r$r%r:rr&r'r/r(rrrrrr=s$r=c@s>eZdZdZdZeeegZgd¢Z e dd„ƒZ e dd„ƒZ dS) ÚFluxCoreDenoiseStepa* Core step that performs the denoising process for Flux. This step supports text-to-image and image-to-image tasks for Flux: - for image-to-image generation, you need to provide `image_latents` - for text-to-image generation, all you need to provide is prompt embeddings. Components: scheduler (`FlowMatchEulerDiscreteScheduler`) transformer (`FluxTransformer2DModel`) Inputs: num_images_per_prompt (`None`, *optional*, defaults to 1): TODO: Add description. prompt_embeds (`Tensor`): Pre-generated text embeddings. Can be generated from text_encoder step. pooled_prompt_embeds (`Tensor`, *optional*): Pre-generated pooled text embeddings. Can be generated from text_encoder step. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. image_latents (`None`, *optional*): TODO: Add description. latents (`Tensor | NoneType`, *optional*): TODO: Add description. generator (`None`, *optional*): TODO: Add description. num_inference_steps (`None`, *optional*, defaults to 50): TODO: Add description. timesteps (`None`, *optional*): TODO: Add description. sigmas (`None`, *optional*): TODO: Add description. strength (`None`, *optional*, defaults to 0.6): TODO: Add description. guidance_scale (`None`, *optional*, defaults to 3.5): TODO: Add description. joint_attention_kwargs (`None`, *optional*): TODO: Add description. Outputs: latents (`Tensor`): Denoised latents. r)ÚinputÚbefore_denoiseÚdenoisecCr-)NaCore step that performs the denoising process for Flux. This step supports text-to-image and image-to-image tasks for Flux: - for image-to-image generation, you need to provide `image_latents` - for text-to-image generation, all you need to provide is prompt embeddings.rrrrrrær.zFluxCoreDenoiseStep.descriptioncCó t d¡gS)NÚlatents©rÚtemplaterrrrÚoutputsïsÿzFluxCoreDenoiseStep.outputsN) r!r"r#r$r%r=r7rr&r'r(rrFrrrrr>µs,  r>Ú text_encoderÚ vae_encoderrAÚdecodec@sPeZdZdZdZe ¡Ze ¡Z ddidddœdœZ e dd„ƒZ e d d „ƒZ d S) ÚFluxAutoBlocksa‡ Auto Modular pipeline for text-to-image and image-to-image using Flux. Supported workflows: - `text2image`: requires `prompt` - `image2image`: requires `image`, `prompt` Components: text_encoder (`CLIPTextModel`) tokenizer (`CLIPTokenizer`) text_encoder_2 (`T5EncoderModel`) tokenizer_2 (`T5TokenizerFast`) image_processor (`VaeImageProcessor`) vae (`AutoencoderKL`) scheduler (`FlowMatchEulerDiscreteScheduler`) transformer (`FluxTransformer2DModel`) Inputs: prompt (`None`, *optional*): TODO: Add description. prompt_2 (`None`, *optional*): TODO: Add description. max_sequence_length (`int`, *optional*, defaults to 512): TODO: Add description. joint_attention_kwargs (`None`, *optional*): TODO: Add description. resized_image (`None`, *optional*): TODO: Add description. image (`None`, *optional*): TODO: Add description. height (`None`, *optional*): TODO: Add description. width (`None`, *optional*): TODO: Add description. generator (`None`, *optional*): TODO: Add description. num_images_per_prompt (`None`, *optional*, defaults to 1): TODO: Add description. image_latents (`None`, *optional*): TODO: Add description. latents (`Tensor | NoneType`, *optional*): TODO: Add description. num_inference_steps (`None`, *optional*, defaults to 50): TODO: Add description. timesteps (`None`, *optional*): TODO: Add description. sigmas (`None`, *optional*): TODO: Add description. strength (`None`, *optional*, defaults to 0.6): TODO: Add description. guidance_scale (`None`, *optional*, defaults to 3.5): TODO: Add description. output_type (`None`, *optional*, defaults to pil): TODO: Add description. Outputs: images (`list`): Generated images. rÚpromptT)r,rK)r8Ú image2imagecCr)NzFAuto Modular pipeline for text-to-image and image-to-image using Flux.rrrrrrDr zFluxAutoBlocks.descriptioncCrB)NÚimagesrDrrrrrFHs zFluxAutoBlocks.outputsN)r!r"r#r$r%Ú AUTO_BLOCKSÚvaluesr&Úkeysr'Ú _workflow_mapr(rrFrrrrrJs7þ rJN)&ÚutilsrÚmodular_pipelinerrÚmodular_pipeline_utilsrrr@r r r r r ÚdecodersrrArÚencodersrrrÚinputsrrÚ get_loggerr!Úloggerrr*r0r4r7r:r=r>rNrJrrrrÚs2    %.<HN-6Büÿ