o GiW@sddlZddlmZddlZddlmZddlm Z ddl m Z m Z ddl mZmZddlmZmZddlmZd d lmZmZmZmZmZeeZejjGd d d eZ e Gd ddej!ee Z"dS)N) FrozenDict) ConfigMixinflax_register_to_config) BaseOutputlogging)FlaxTimestepEmbedding FlaxTimesteps)FlaxModelMixin)FlaxCrossAttnDownBlock2DFlaxCrossAttnUpBlock2DFlaxDownBlock2DFlaxUNetMidBlock2DCrossAttn FlaxUpBlock2Dc@seZdZUdZejed<dS)FlaxUNet2DConditionOutputa The output of [`FlaxUNet2DConditionModel`]. Args: sample (`jnp.ndarray` of shape `(batch_size, num_channels, height, width)`): The hidden states output conditioned on `encoder_hidden_states` input. Output of last layer of model. sampleN)__name__ __module__ __qualname____doc__jnpndarray__annotations__rra/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/models/unets/unet_2d_condition_flax.pyr%s rc@seZdZUdZdZeed<dZeed<dZeed<dZ e e dfed <d Z e e dfed <d Z e d Bed<dZee eBed<dZe edfed<dZeed<dZee edfBed<d Zee edfBd Bed<dZeed<dZeed<dZeed<ejZejed<dZeed<d Zeed!<dZeed"<dZeed#<d$Zee edfBed%<d Z e d Bed&<d Z!ed Bed'<d(Z"eed)<d Z#ed Bed*<d+e$j%d,e&fd-d.Z'd;d/d0Z(  d) num_groupsepsilon)4loggerwarningr-r1 ValueErrorr0nnConvr5conv_inr r6rPr7 time_projr time_embeddingr+ isinstanceboollenr%intr:r; add_embeddingr< add_time_proj enumerater r3r.r4r8r9rappend down_blocksr*r mid_blocklistreversedr(minrr up_blocks GroupNorm conv_norm_outr"conv_out)rRr-time_embed_dimr1r+r:r{output_channelidown_block_type input_channelis_final_block down_blockrreversed_block_out_channelsreversed_num_attention_heads%reversed_transformer_layers_per_block up_block_typergup_blockrrrsetups                     zFlaxUNet2DConditionModel.setuprrTrUrYdown_block_additional_residualsmid_block_additional_residual return_dicttrainc Cst|tjstj|gtjd}nt|tjr*t|jdkr*|jtjd}t |d}| |} | | } d} |j dkr|durLt d|jd|j d|d} | dur]t |jd |d } | durnt |jd |t| } t| | jdd f} tj| | gd d }||} | dur| | n| } t|d}||}|f}|jD]!}t|tr||| || d\}}n ||| | d\}}||7}q|durd}t||D] \}}||7}||f7}q|}|jdur|j|| || d}|dur||7}|jD]2}||jd d}|d|jd }t|tr)||| ||| d}q||| || d}q||}t |}|!|}t|d}|sO|fSt"|dS)a  Args: sample (`jnp.ndarray`): (batch, channel, height, width) noisy inputs tensor timestep (`jnp.ndarray` or `float` or `int`): timesteps encoder_hidden_states (`jnp.ndarray`): (batch_size, sequence_length, hidden_size) encoder hidden states added_cond_kwargs: (`dict`, *optional*): A kwargs dictionary containing additional embeddings that if specified are added to the embeddings that are passed along to the UNet blocks. down_block_additional_residuals: (`tuple` of `torch.Tensor`, *optional*): A tuple of tensors that if specified are added to the residuals of down unet blocks. mid_block_additional_residual: (`torch.Tensor`, *optional*): A tensor that if specified is added to the residual of the middle unet block. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`models.unets.unet_2d_condition_flax.FlaxUNet2DConditionOutput`] instead of a plain tuple. train (`bool`, *optional*, defaults to `False`): Use deterministic functions and disable dropout when not training. Returns: [`~models.unets.unet_2d_condition_flax.FlaxUNet2DConditionOutput`] or `tuple`: [`~models.unets.unet_2d_condition_flax.FlaxUNet2DConditionOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is the sample tensor. rBrNrDz1Need to provide argument `added_cond_kwargs` for z! when using `addition_embed_type=`rGz has the config param `addition_embed_type` set to 'text_time' which requires the keyword argument `text_embeds` to be passed in `added_cond_kwargs`rHz has the config param `addition_embed_type` set to 'text_time' which requires the keyword argument `time_ids` to be passed in `added_cond_kwargs`rf)axis)rrrr ) deterministicrr )tembrUres_hidden_states_tupler)rrr)rrr r)r)#rsrrarrayrLrushapeastyperJ expand_dimsrqrrr;rm __class__getrxravelreshape concatenaterw transposerpr{r zipr|rr.rrrnsilurr)rRrrTrUrYrrrrt_embaug_embrGrH time_embeds add_embedsdown_block_res_samplesr res_samplesnew_down_block_res_samplesdown_block_res_sampledown_block_additional_residualrrrr__call__Qs #                         z!FlaxUNet2DConditionModel.__call__)rAN)NNNTF)-rrrrrrvrr!r"r%tuplestrr(r*r+rtr-r.r0r1r2r3floatr4rrJr5r6r7r8r9r:r;r<r>r?rMArrayrr_rrdictrrrrrrr2sj  5            0    r)#flax flax.linenlinenrnrM jax.numpynumpyrflax.core.frozen_dictrconfiguration_utilsrrutilsrrembeddings_flaxr r modeling_flax_utilsr unet_2d_blocks_flaxr rrrr get_loggerrrkstruct dataclassrModulerrrrrs