o ٷi @sddlZddlZddlZddlZddlZddlmZddlmZddl m Z ddl Z ddl ZddlZddlmZddlmmZddlmZmZddlmZmZmZddlmZddlmZdd l m!Z!dd l"m#Z#dd l$m%Z%m&Z&dd l'm(Z(dd l)m*Z*m+Z+ddl,m-Z-ddl.m/Z/ddl0m1Z1m2Z2ddl3m4Z4ddl5m6Z6ddl7m8Z8e9e:Z;eGddde!Zde+fddZ?GdddeZ@   d'deAdBd eBejCBdBd!eDeAdBd"e fd#d$ZEGd%d&d&ejFZGdS)(N)Iterable) dataclass)Any) ConfigMixinregister_to_config)PipelineImageInputVaeImageProcessoris_valid_image_imagelist) AutoencoderKL)SchedulerMixin) BaseOutput) randn_tensor)"Qwen2_5_VLForConditionalGenerationQwen2_5_VLProcessor)AutoWeightsLoader)DiffusionOutputOmniDiffusionConfig)get_local_device)DiffusersPipelineLoader)OmniGen2RotaryPosEmbedOmniGen2Transformer2DModel)OmniDiffusionRequest)OmniTextPrompt)!download_weights_from_hf_specificc@seZdZUdZejed<dS)%FlowMatchEulerDiscreteSchedulerOutputaJ Output class for the scheduler's `step` function output. Args: prev_sample (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)` for images): Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop. prev_sampleN)__name__ __module__ __qualname____doc__torch FloatTensor__annotations__r#r#i/home/ubuntu/.local/lib/python3.10/site-packages/vllm_omni/diffusion/models/omnigen2/pipeline_omnigen2.pyr/s  rc@seZdZdZgZdZed&dedefddZ e d d Z e d d Z d'defddZ d(ddZ    d)dedeejBdeedBdedBfddZddZ  d*dejdeejBdejdejdBd ed!eeBf d"d#Zd$d%ZdS)+FlowMatchEulerDiscreteSchedulera Euler scheduler. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. dynamic_time_shift (`bool`, defaults to `True`): Whether to use dynamic time shifting for the timestep schedule. Tnum_train_timestepsdynamic_time_shiftcCs6tjdd|dtjddd}||_d|_d|_dS)Nrr&dtype)r linspacefloat32 timesteps _step_index _begin_index)selfr(r)r/r#r#r$__init__Ns  z(FlowMatchEulerDiscreteScheduler.__init__cC|jS)zg The index counter for current timestep. It will increase 1 after each scheduler step. )r0r2r#r#r$ step_indexWz*FlowMatchEulerDiscreteScheduler.step_indexcCr4)zq The index for the first timestep. It should be set from pipeline with `set_begin_index` method. r1r5r#r#r$ begin_index^r7z+FlowMatchEulerDiscreteScheduler.begin_indexrr9cCs ||_dS)z Sets the begin index for the scheduler. This function should be run from pipeline before the inference. Args: begin_index (`int`): The begin index for the scheduler. Nr8)r2r9r#r#r$set_begin_indexes z/FlowMatchEulerDiscreteScheduler.set_begin_indexNcCs:|dur|j}||k}t|dkrdnd}||S)Nr&r) _timestepsnonzerolenitem)r2timestepschedule_timestepsindicesposr#r#r$index_for_timestepos   z2FlowMatchEulerDiscreteScheduler.index_for_timestepnum_inference_stepsdevicer/ num_tokenscCs|dur0||_tjdd|dtjddd}|jjr0|dur0t|d}|||||}t|j tj|d}t |tj d|j dg}||_ ||_d|_d|_dS) a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. timesteps (`list[float]`, *optional*): Custom timesteps to use. If provided, `num_inference_steps` is ignored. num_tokens (`int`, *optional*): Number of tokens, used for dynamic time shifting. Nrr&r*r,(r+rErE)rDnpr-r.configr)sqrtr from_numpytocatonesrEr/r;r0r1)r2rDrEr/rFmr;r#r#r$ set_timesteps}s  z-FlowMatchEulerDiscreteScheduler.set_timestepscCs@|jdurt|tjr||jj}|||_dS|j |_dSN) r9 isinstancer TensorrNr/rErCr0r1)r2r?r#r#r$_init_step_indexs   z0FlowMatchEulerDiscreteScheduler._init_step_index model_outputr?sample generator return_dictreturnc Cst|tst|tjst|tjrtd|jdur|||tj }|j |j}|j |jd}||||}||j }|j d7_ |sM|fSt |dS)aT Predict the sample from the previous timestep by reversing the SDE. This function propagates the diffusion process from the learned model outputs (most often the predicted noise). Args: model_output (`torch.FloatTensor`): The direct output from learned diffusion model. timestep (`float`): The current discrete timestep in the diffusion chain. sample (`torch.FloatTensor`): A current instance of a sample created by the diffusion process. generator (`torch.Generator`, *optional*): A random number generator. return_dict (`bool`): Whether or not to return a [`~FlowMatchEulerDiscreteSchedulerOutput`] or tuple. Returns: [`~FlowMatchEulerDiscreteSchedulerOutput`] or `tuple`: If return_dict is `True`, [`~FlowMatchEulerDiscreteSchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. zPassing integer indices (e.g. from `enumerate(timesteps)`) as timesteps to `EulerDiscreteScheduler.step()` is not supported. Make sure to pass one of the `scheduler.timesteps` as a timestep.Nr&)r)rTintr IntTensor LongTensor ValueErrorr6rVrNr.r;r+r0r) r2rWr?rXrYrZtt_nextrr#r#r$steps"!      z$FlowMatchEulerDiscreteScheduler.stepcCs|jjSrS)rKr(r5r#r#r$__len__sz'FlowMatchEulerDiscreteScheduler.__len__)r'T)rrSNNNN)NT)rrrr _compatiblesorderrr\boolr3propertyr6r9r:rCstrr rElistfloatrRrVr! Generatorrtuplerbrcr#r#r#r$r%=sT      %  >r% od_configcs|j}tj|r |}nt|ddg}tj|d}t|}t|}d|vr0dt |dnd}Wdn1s s&zKget_omnigen2_pre_process_func..pre_process_func..r) max_pixelsmax_side_lengthr&preprocessed_images) enumeratepromptsrTrigetrrjr~rrget_new_height_widthsampling_paramsheightwidthr rUr=shape preprocessappend) ryir|rz raw_imageimages first_rawnew_hnew_wrr{image_processorlatent_channelsr#r$pre_process_funcs@          z7get_omnigen2_pre_process_func..pre_process_func) modelospathexistsrjoinrjsonloadr=OmniGen2ImageProcessorrr)rn model_name model_pathvae_config_pathf vae_configrurr#rr$get_omnigen2_pre_process_funcs"    ,rcs|j}tj|r |}nt|ddg}tj|d}t|}t|}d|vr2dt |ddnd}Wdn1s>wYt |dddd t j ffd d }|S) Nrorpblock_out_channelsrrr&rsTrtrcs |SrS) postprocess)rrr#r$post_process_func;s z9get_omnigen2_post_process_func..post_process_func) rrrrrrrrrr=rr rU)rnrrrrrrurr#rr$get_omnigen2_post_process_func,s   "rcseZdZdZe        dded ed ed ed ed ededeffdd Z    dde j j e j Be jBdedBdedBd edBd edBdeeeff ddZ      d dededBdedBd edBd edBdedeeeeefdBde jfddZZS)!raq Image processor for OmniGen2 image resize and crop. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to downscale the image's (height, width) dimensions to multiples of `vae_scale_factor`. Can accept `height` and `width` arguments from [`image_processor.VaeImageProcessor.preprocess`] method. vae_scale_factor (`int`, *optional*, defaults to `16`): VAE scale factor. If `do_resize` is `True`, the image is automatically resized to multiples of this factor. resample (`str`, *optional*, defaults to `lanczos`): Resampling filter to use when resizing the image. max_pixels (`int`, *optional*, defaults to `1048576`): Maximum number of pixels allowed in the image. Images exceeding this limit are downscaled proportionally. max_side_length (`int`, *optional*, defaults to `1024`): Maximum length of the longer side of the image. Images exceeding this limit are downscaled proportionally. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image to [-1,1]. do_binarize (`bool`, *optional*, defaults to `False`): Whether to binarize the image to 0/1. do_convert_grayscale (`bool`, *optional*, defaults to `False`): Whether to convert the images to grayscale format. TrxlanczosrrFrvruresamplerr do_normalize do_binarizedo_convert_grayscalec s(tj||||||d||_||_dS)N)rvrurrrr)superr3rr) r2rvrurrrrrr __class__r#r$r3[s  zOmniGen2ImageProcessor.__init__Nr{rrr[c Cs$|dur t|tjjr|j}nt|tjr|jd}n|jd}|dur@t|tjjr/|j}nt|tjr;|jd}n|jd}|durG|j}|durN|j }d}|dura||kr]||}n||}||}||d}t ||d} t || |j j |j j t || |j j |j j } } | | fS)a3 Returns the height and width of the image, downscaled to the next integer multiple of `vae_scale_factor`. Args: image (`Union[PIL.Image.Image, np.ndarray, torch.Tensor]`): The image input, which can be a PIL image, NumPy array, or PyTorch tensor. If it is a NumPy array, it should have shape `[batch, height, width]` or `[batch, height, width, channels]`. If it is a PyTorch tensor, it should have shape `[batch, channels, height, width]`. height (`Optional[int]`, *optional*, defaults to `None`): The height of the preprocessed image. If `None`, the height of the `image` input will be used. width (`Optional[int]`, *optional*, defaults to `None`): The width of the preprocessed image. If `None`, the width of the `image` input will be used. Returns: `Tuple[int, int]`: A tuple containing the height and width, both resized to the nearest integer multiple of `vae_scale_factor`. Nrrr&??)rTr~rrr rUrrrrminr\rKru) r2r{rrrrmax_side_length_ratio cur_pixelsmax_pixels_ratioratio new_height new_widthr#r#r$rss8         z+OmniGen2ImageProcessor.get_new_height_widthdefault resize_mode crops_coordsc sptjjtjtjf}jjr=t|tjtjfr=|j dkr=t|tjr'| d}n|j ddkr6tj |dd}ntj |dd}t|t r^t|dtjr^|dj dkr^tdttj|dd}t|t rt|dtjr|dj dkrtdttj|dd}t|std d d d |Dt|t s|g}t|dtjjrd urfdd|D}jjr͈|d||\fdd|D}jjrۇfdd|D}n jjrfdd|D}|}|}nt|dtjr/|dj dkr tj|ddntj|dd}|}|||\jjr.|}nTt|dtjr|dj dkrGtj|ddntj|dd}jjr^|j dkr^| d}|j d} | jjkrl|S|||\jjr|}jj} | r|dkrtd|d| dtd} | r!|}jj"r#|}|S)a Preprocess the image input. Args: image (`PipelineImageInput`): The image input, accepted formats are PIL images, NumPy arrays, PyTorch tensors; Also accept list of supported formats. height (`int`, *optional*): The height in preprocessed image. If `None`, will use the `get_default_height_width()` to get default height. width (`int`, *optional*): The width in preprocessed. If `None`, will use get_default_height_width()` to get the default width. resize_mode (`str`, *optional*, defaults to `default`): The resize mode, can be one of `default` or `fill`. If `default`, will resize the image to fit within the specified width and height, and it may not maintaining the original aspect ratio. If `fill`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, filling empty with data from image. If `crop`, will resize the image to fit within the specified width and height, maintaining the aspect ratio, and then center the image within the dimensions, cropping the excess. Note that resize_mode `fill` and `crop` are only supported for PIL image input. crops_coords (`List[Tuple[int, int, int, int]]`, *optional*, defaults to `None`): The crop coordinates for each image in the batch. If `None`, will not crop the image. Returns: `torch.Tensor`: The preprocessed image. rr&r,r)axiszPassing `image` as a list of 4d np.ndarray is deprecated.Please concatenate the list along the batch dimension and pass it as a single 4d np.ndarrayzPassing `image` as a list of 4d torch.Tensor is deprecated.Please concatenate the list along the batch dimension and pass it as a single 4d torch.Tensorz9Input is in incorrect format. Currently, we only support z, css|]}t|VqdSrS)ri)rxr#r#r$ sz4OmniGen2ImageProcessor.preprocess..Ncsg|]}|qSr#)croprr)rr#r$rz5OmniGen2ImageProcessor.preprocess..csg|] }j|dqS))r)resizer)rrr2rr#r$r scg|]}|qSr#)convert_to_rgbrr5r#r$r rcrr#)convert_to_grayscalerr5r#r$rrzPassing `image` as torch tensor with value range in [-1,1] is deprecated. The expected value range for image tensor is [0,1] when passing as pytorch tensor or numpy Array. You passed `image` with value range [,]F)$r~rrJndarrayr rUrKrrTndim unsqueezer expand_dimsrjwarningswarn FutureWarning concatenaterOr r_rrvrdo_convert_rgb pil_to_numpy numpy_to_ptstackrvae_latent_channelsrrmax normalizerbinarize) r2r{rrrrrrsupported_formatschannelrr#)rrrr2rr$rs%$  ((   ,  ,      z!OmniGen2ImageProcessor.preprocess)TrxrrrTFFrd)NNNNrN)rrrrrrgr\rir3r~rrJrr rUrmrrr __classcell__r#r#rr$rCs   E rrDrEr/kwargscKs|dur2dtt|jjv}|std|jd|jd||d||j}t |}||fS|j|fd|i||j}||fS)aa Calls the scheduler's `set_timesteps` method and retrieves timesteps from the scheduler after the call. Handles custom timesteps. Any kwargs will be supplied to `scheduler.set_timesteps`. Args: scheduler (`SchedulerMixin`): The scheduler to get timesteps from. num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. If used, `timesteps` must be `None`. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. timesteps (`List[int]`, *optional*): Custom timesteps used to override the timestep spacing strategy of the scheduler. If `timesteps` is passed, `num_inference_steps` must be `None`. **kwargs (`Any`): Additional keyword arguments passed to `scheduler.set_timesteps`. Returns: timesteps (`torch.Tensor`): The timestep schedule from the scheduler. num_inference_steps (`int`): The number of inference steps. Nr/zThe current scheduler class zx's `set_timesteps` does not support custom timestep schedules. Please check whether you are using the correct scheduler.)r/rErEr#) setinspect signaturerR parameterskeysr_rr/r=) schedulerrDrEr/raccepts_timestepsr#r#r$retrieve_timesteps@s rc4s4eZdZdZdddededdffdd Z dQd ed ed ed edej dej dej dBdej dBdej fddZ dej dej fddZdeejjejjBd ededededej dej deej dBfddZ  dRdeeeBdej dBdedeejejffd d!Zdefd"d#Z $  %      dSdeeeBd&ed'eeeBdBdedej dBd(ejdBd)ejdBd*ejdBd+ejdBdedeejejejejffd,d-Zed.d/Zed0d1Zed2d3Zed4d5Ze       6  %   7 6 $ 8 9 : ;     < dTd=edeeeBdBd'eeeBdBd(ej dBd)ej dBd*ej dBd+ej dBdedBd>eejjdBded edBd edBded?ed@edAedBe!dCe!dDee!e!fdEe"ee#fdBdFeedej eej BdBdej dBdGede$f2dHdIZ% dQdJdKZ&dLdMZ'dNe(eeejfde)efdOdPZ*Z+S)UOmniGen2Pipelinea Pipeline for text-to-image generation using OmniGen2. This pipeline implements a text-to-image generation model that uses: - Qwen2.5-VL for text encoding - A custom transformer architecture for image generation - VAE for image encoding/decoding - FlowMatchEulerDiscreteScheduler for noise scheduling Args: od_config (OmniDiffusionConfig): The OmniDiffusion configuration. )prefixrnrr[Nc st||_t|_|j}tj|}t j |jdddddg|_ t j |d|d|_tj |d|d|j|_tj|dd }i}tj|rt| }t|}Wdn1s\wYd d d d dddddddddddd} | D]\} } | |vr|| } t| tr| dvrt| } | || <qwtd"i||_tj |d|d|j|_tj |d|d|_t |dr|jdurdt!|jj"j#dnd|_$t%|j$ddd |_&d!|_'dS)#aj Initialize the OmniGen2 pipeline. Args: transformer: The transformer model for image generation. vae: The VAE model for image encoding/decoding. scheduler: The scheduler for noise scheduling. text_encoder: The text encoder model. tokenizer: The tokenizer for text processing. transformerNz transformer.T) model_or_path subfolderrevisionrfall_back_to_ptr)rlocal_files_onlyvaez config.json patch_size in_channels out_channels hidden_size num_layersnum_refiner_layersnum_attention_heads num_kv_heads multiple_offfn_dim_multipliernorm_eps axes_dim_rope axes_lens text_feat_dimtimestep_scale)rrrrrrrrrrrrrrr)rrmllm processorrrr&rsrtr#)(rr3rnrrErrrrrComponentSourceweights_sourcesr%from_pretrainedrr rNrrrrritemsrTrjrmrrrrrrhasattrr=rKrrurrdefault_sample_size) r2rnrrrtransformer_config_pathtransformer_kwargsrtransformer_config param_mapping config_key param_namevaluerr#r$r3{sz      , zOmniGen2Pipeline.__init__ batch_sizenum_channels_latentsrrr+rErYlatentsc CsRt||j}t||j}||||f} |dur"t| |||d}|S||}|S)a Prepare the initial latents for the diffusion process. Args: batch_size: The number of images to generate. num_channels_latents: The number of channels in the latent space. height: The height of the generated image. width: The width of the generated image. dtype: The data type of the latents. device: The device to place the latents on. generator: The random number generator to use. latents: Optional pre-computed latents to use instead of random initialization. Returns: torch.FloatTensor: The prepared latents tensor. N)rYrEr+)r\rur rN) r2rrrrr+rErYrrr#r#r$prepare_latentss  z OmniGen2Pipeline.prepare_latentsrcCsj|j|j|jjdj}|jjjdur||jjj}|jjjdur+||jjj}|j|jjd}|S)z Encode an image into the VAE latent space. Args: img: The input image tensor to encode. Returns: torch.FloatTensor: The encoded latent representation. r*N) rencoderNr+ latent_distrXrK shift_factorscaling_factor)r2rz0r#r#r$ encode_vaes zOmniGen2Pipeline.encode_vaernum_images_per_promptrrc Cs|dkr|g}g}t|D]7\} } | dur6t| dkr6g} t| D]\} } | || j|ddq!nd} t|D]}|| qnegative_prompt_attention_maskc  s|pj}t|tr |gn|}fdd|D}|dur!t|} n|jd} |dur4j||| d\}}|j\} } } |d|d}|| || d}||d}|| |d}|r|dur|durd|nd}t|trp| |gn|}fd d|D}|durt|t|urt d t|d t|d t|tr|g}n| t|krt d |dt|d|d| d j||| d\}} |j\} } } |d|d}|| || d}| |d} | | |d} |||| fS)a Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded negative_prompt (`str` or `List[str]`, *optional*): The prompt not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). For Lumina-T2I, this should be "". do_classifier_free_guidance (`bool`, *optional*, defaults to `True`): whether to use classifier free guidance or not num_images_per_prompt (`int`, *optional*, defaults to 1): number of images that should be generated per prompt device: (`torch.device`, *optional*): torch device to place the resulting embeddings on prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. For Lumina-T2I, it's should be the embeddings of the "" string. max_sequence_length (`int`, defaults to `256`): Maximum sequence length to use for the prompt. crr#rH)r_promptr5r#r$rrz2OmniGen2Pipeline.encode_prompt..Nr)r|rEr*r&r,rcrr#rM)r_negative_promptr5r#r$rrz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.) rErTrir=rr@repeatviewtype TypeErrorr_)r2r|rIrJr!rEr?rKr>rLr*rseq_lenr'r#r5r$ encode_promptsd %          zOmniGen2Pipeline.encode_promptcCr4rS)_num_timestepsr5r#r#r$ num_timestepszOmniGen2Pipeline.num_timestepscCr4rS)_text_guidance_scaler5r#r#r$text_guidance_scalerYz$OmniGen2Pipeline.text_guidance_scalecCr4rS)_image_guidance_scaler5r#r#r$image_guidance_scalerYz%OmniGen2Pipeline.image_guidance_scalecCr4rS) _cfg_ranger5r#r#r$ cfg_rangerYzOmniGen2Pipeline.cfg_rangerr@rgrFreq input_imagesmax_input_image_side_length align_resrDr[r]r_attention_kwargsr/verbosec'Cst|jdkrtdt|jd|jd}t|tr|n|dp$|}t|tr,dn|d|}|dur>|dur>tdt|tsRd|di}vrR|d} |jj p]| p]|j |j } |jj pi| pi|j |j } |jj po|}|jjpu|}|jjr~|jj}||_|jjdur|jjn||_||_||_|jjdkr|jjn| } |durt|trd}n|durt|trt|}n|jd}|j}|j||jd k|| ||||||d \}}}}|jj}|j| || | |||d }| durg} t| dkr|r|ddjd |j |ddjd |j } } | | } }!n)| | } }!| | }"| |"d}#t|#d }#t | |#ddt | |#dd} } t| dkrJd|_|j!j"j#}$|$|| |$| | |j|||}t%j&|j!j"j'|j!j"j(dd}%|j)||||%|||||||||d }&t*j+|&|!| fdd}&t,|&dS)Nr&z{OmniGen2 only supports a single prompt per request. Only the first prompt will be used; %d extra prompt(s) will be ignored.rr|rJzrLr*)rrr!rrrEr+r,rrxi')theta) rr$r? freqs_cisrKr>rLrDr/rEr+rh step_funcbilinear)sizemode)output)-r=rr7r8rTrirr_rrrrurrYrDguidance_scale_providedguidance_scalerZguidance_scale_2r\r^_attention_kwargsnum_outputs_per_promptrjrrErVr[rr+r(rr\rrKrrr get_freqs_cisrr processingF interpolater)'r2rcr|rJr?rKr>rLr*rdr!rrrrerfrDr[r]r_rgr/rYrrhrl first_promptr}rrEr+r$ ori_width ori_heightrrrrkr{r#r#r$forwards                  zOmniGen2Pipeline.forwardc Cst|j|| | |jd|jdd\} }t| |_t| D]\}}|j||||||d}|jd|t| kr@|jdkrEnn|jnd}|jd|t| kr[|jdkr`nn|j nd}|dkr|dkr|j||||||d}|j|||||dd}|||||||}n|dkr|j|||||dd}||||}|jj |||dd d}|j | d }| dur| ||jq|j | d }|j j jdur||j j j}|j j jdur||j j j}|j j|dd d}|S) Nrir,)rF)r`rr?rkr>ref_image_hidden_statesrr&rF)rZr*)rrrr=rWrpredictr_r[r]rbrNrrKrrdecode)r2rr$r?rkrKr>rLrDr/rEr+rhrlrr` model_predr[r]model_pred_refmodel_pred_uncondr{r#r#r$rws 4 4      zOmniGen2Pipeline.processingcCsl||jd|j}|j\}} } } i} dtt|jjj vr'|| d<|j|||||fi| } | S)Nrr~) expandrrNr+rrrrr}rr)r2r`rr?rkr>r~r?rrrroptional_kwargsrr#r#r$rs zOmniGen2Pipeline.predictweightscCst|}||SrS)r load_weights)r2rloaderr#r#r$rs zOmniGen2Pipeline.load_weightsrS)Nr)) TNr&NNNNNr))NNNNNNrNr&NNrrTr`rarrbNNNNFN),rrrrrrir3r\r r+rErlr!rr rjr~rr(rmrUr@rHrgrVrhrXr[r]r_no_gradrr^rkdictrrr}rwrrrrrr#r#rr$rmsjb  & )  C     c           8 \,r)NNN)Hrrloggingrrcollections.abcr dataclassesrtypingrnumpyrJ PIL.Imager~r torch.nnnntorch.nn.functional functionalrxdiffusers.configuration_utilsrrdiffusers.image_processorrrr diffusers.models.autoencodersr %diffusers.schedulers.scheduling_utilsr diffusers.utilsr diffusers.utils.torch_utilsr transformersrr vllm.model_executor.models.utilsrvllm_omni.diffusion.datarr%vllm_omni.diffusion.distributed.utilsr1vllm_omni.diffusion.model_loader.diffusers_loaderr8vllm_omni.diffusion.models.omnigen2.omnigen2_transformerrrvllm_omni.diffusion.requestrvllm_omni.inputs.datar2vllm_omni.model_executor.model_loader.weight_utilsr getLoggerrr7rr%rrrr\rirErjrModulerr#r#r#r$sn                0 @    -