o ۷i/@sddlmZddlZddlmZddlmZddlmZddl m Z m Z m Z m Z dd lmZmZmZd d l mZeeZGd d d e ZGddde ZGddde ZGddde ZGdddeZGdddeZdS))AnyN)FluxTransformer2DModel)FlowMatchEulerDiscreteScheduler)logging) BlockStateLoopSequentialPipelineBlocksModularPipelineBlocks PipelineState) ComponentSpec InputParam OutputParam)FluxModularPipelinec @~eZdZdZedeefddZedefddZ edee ee ffddZ e d ed ed ed e jdef d dZdS)FluxLoopDenoiserfluxreturncC tdtgSN transformerr rselfr^/home/ubuntu/vllm_env/lib/python3.10/site-packages/diffusers/modular_pipelines/flux/denoise.pyexpected_components& z$FluxLoopDenoiser.expected_componentscC dS)NzStep within the denoising loop that denoise the latents. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `FluxDenoiseLoopWrapper`)rrrrr description*zFluxLoopDenoiser.descriptionc Csjtdtddtjddtddtjddtd dtjd dtd dtjd dtd dtjddtddtjddgS)Njoint_attention_kwargslatentsT^The initial latents to use for the denoising process. Can be generated in prepare_latent step.required type_hintr guidanceFGuidance scale as a tensor prompt_embedsPrompt embeddingspooled_prompt_embedsPooled prompt embeddingstxt_ids/IDs computed from text sequence needed for RoPEimg_idsz0IDs computed from image sequence needed for RoPEr torchTensorrrrrinputs2sLzFluxLoopDenoiser.inputs components block_stateitc CsD|j|j|d|j|j|j|j|j|jdd d}||_ ||fS)NF hidden_statestimestepr(encoder_hidden_statespooled_projectionsr"r.r0 return_dictr) rr#flattenr(r*r,r"r.r0 noise_pred)rr5r6r7r8rArrr__call__\s   zFluxLoopDenoiser.__call__N__name__ __module__ __qualname__ model_namepropertylistr rstrr tuplerr4r2no_gradrrintr3r rBrrrrr#s()rc @r)FluxKontextLoopDenoiser flux-kontextrcCrrrrrrrrsrz+FluxKontextLoopDenoiser.expected_componentscCr)NzStep within the denoising loop that denoise the latents for Flux Kontext. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `FluxDenoiseLoopWrapper`)rrrrrr wr!z#FluxKontextLoopDenoiser.descriptionc Csxtdtddtjddtdtjddtd d tjd dtd dtjd dtddtjddtddtjddtddtjddgS)Nr"r#Tr$r% image_latentszXImage latents to use for the denoising process. Can be generated in prepare_latent step.r'r r(Fr)r*r+r,r-r.r/r0z1IDs computed from latent sequence needed for RoPEr1rrrrr4sVzFluxKontextLoopDenoiser.inputsr5r6r7r8c Cs|j}|}|j}|durtj||gdd}||jd|j}|j||d|j |j |j |j |j |jdd d} | ddd|df} | |_||fS)Nr)dimrr9Fr:)r#rPr2catexpandshapetodtyperr(r*r,r"r.r0sizerA) rr5r6r7r8r#latent_model_inputrPr<rArrrrBs,  z FluxKontextLoopDenoiser.__call__NrCrrrrrNps(.rNc @seZdZdZedeefddZedefddZ edee ee ffddZ edeefd d Z edeefd d Zed edededejfddZdS)FluxLoopAfterDenoiserrrcCr)N scheduler)r rrrrrrrz)FluxLoopAfterDenoiser.expected_componentscCr)Nzstep within the denoising loop that update the latents. This block should be used to compose the `sub_blocks` attribute of a `LoopSequentialPipelineBlocks` object (e.g. `FluxDenoiseLoopWrapper`)rrrrrr r!z!FluxLoopAfterDenoiser.descriptioncCsgS)Nrrrrrr4szFluxLoopAfterDenoiser.inputscCs tdgS)N generator)r rrrrintermediate_inputss z)FluxLoopAfterDenoiser.intermediate_inputscCstdtjddgS)Nr#zThe denoised latentsrQ)rr2r3rrrrintermediate_outputssz*FluxLoopAfterDenoiser.intermediate_outputsr5r6r7r8cCsH|jj}|jj|j||jddd|_|jj|kr |j||_||fS)NF)r?r)r#rWr[steprArV)rr5r6r7r8 latents_dtyperrrrBs zFluxLoopAfterDenoiser.__call__N)rDrErFrGrHrIr rrJr rKrr4r]rr^r2rLrrrMr3rBrrrrrZs"rZc@sleZdZdZedefddZedeefddZ edee fddZ e d ed edefd d Zd S)FluxDenoiseLoopWrapperrrcCr)NzPipeline block that iteratively denoise the latents over `timesteps`. The specific steps with each iteration can be customized with `sub_blocks` attributesrrrrrr r!z"FluxDenoiseLoopWrapper.descriptioncCstdttdtgS)Nr[r)r rrrrrrloop_expected_componentssz/FluxDenoiseLoopWrapper.loop_expected_componentscCs"tddtjddtddtddgS)N timestepsTzWThe timesteps to use for the denoising process. Can be generated in set_timesteps step.r%num_inference_stepszgThe number of inference steps to use for the denoising process. Can be generated in set_timesteps step.)r r2r3rMrrrr loop_inputssz"FluxDenoiseLoopWrapper.loop_inputsr5statecCs||}tt|j|j|jjd|_|j|jd;}t |jD]-\}}|j ||||d\}}|t|jdksK|d|jkrO|d|jjdkrO| q"Wdn1sZwY| ||||fS)Nr)total)r7r8r) get_block_statemaxlenrcrdr[ordernum_warmup_steps progress_bar enumerate loop_stepupdateset_block_state)rr5rfr6rmr7r8rrrrBs " zFluxDenoiseLoopWrapper.__call__N)rDrErFrGrHrJr rIr rbr rer2rLrr rBrrrrrasrac@s.eZdZeegZddgZedefddZ dS)FluxDenoiseStepdenoiserafter_denoiserrcCr)Na*Denoise step that iteratively denoise the latents. Its loop logic is defined in `FluxDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequentially: - `FluxLoopDenoiser` - `FluxLoopAfterDenoiser` This block supports both text2image and img2img tasks.rrrrrr 0r!zFluxDenoiseStep.descriptionN) rDrErFrrZ block_classes block_namesrHrJr rrrrrr,s rrc@s2eZdZdZeegZddgZede fddZ dS)FluxKontextDenoiseSteprOrsrtrcCr)Na1Denoise step that iteratively denoise the latents. Its loop logic is defined in `FluxDenoiseLoopWrapper.__call__` method At each iteration, it runs blocks defined in `sub_blocks` sequentially: - `FluxKontextLoopDenoiser` - `FluxLoopAfterDenoiser` This block supports both text2image and img2img tasks.rrrrrr Ar!z"FluxKontextDenoiseStep.descriptionN) rDrErFrGrNrZrurvrHrJr rrrrrw<s rw)typingrr2modelsr schedulersrutilsrmodular_pipelinerr r r modular_pipeline_utilsr r rr get_loggerrDloggerrrNrZrarrrwrrrrs      MZ,6