o GivA@sddlmZddlmZddlmZddlm Z ddl m Z m Z m Z eeZGdddejZGd d d ejZGd d d ejZGd ddejZGdddejZdS)N)logging)FlaxTransformer2DModel)FlaxDownsample2DFlaxResnetBlock2DFlaxUpsample2Dc@seZdZUdZeed<eed<dZeed<dZeed<dZ eed<d Z e ed <d Z e ed <d Z e ed <d Ze ed<d Ze ed<ejZejed<dZeed<ddZdddZdS)FlaxCrossAttnDownBlock2Da& Cross Attention 2D Downsizing block - original architecture from Unet transformers: https://huggingface.co/papers/2103.06104 Parameters: in_channels (:obj:`int`): Input channels out_channels (:obj:`int`): Output channels dropout (:obj:`float`, *optional*, defaults to 0.0): Dropout rate num_layers (:obj:`int`, *optional*, defaults to 1): Number of attention blocks layers num_attention_heads (:obj:`int`, *optional*, defaults to 1): Number of attention heads of each spatial transformer block add_downsample (:obj:`bool`, *optional*, defaults to `True`): Whether to add downsampling layer before each final output use_memory_efficient_attention (`bool`, *optional*, defaults to `False`): enable memory efficient attention https://huggingface.co/papers/2112.05682 split_head_dim (`bool`, *optional*, defaults to `False`): Whether to split the head dimension into a new axis for the self-attention computation. In most cases, enabling this flag should speed up the computation for Stable Diffusion 2.x and Stable Diffusion XL. dtype (:obj:`jnp.dtype`, *optional*, defaults to jnp.float32): Parameters `dtype` in_channels out_channelsdropout num_layersnum_attention_headsTadd_downsampleFuse_linear_projectiononly_cross_attentionuse_memory_efficient_attentionsplit_head_dimdtypetransformer_layers_per_blockc Cstdg}g}t|jD]:}|dkr|jn|j}t||j|j|jd}| |t |j|j |j|j |j |j |j|j|j|jd }| |q||_||_|jr]t|j|jd|_dSdS)NFlax classes are deprecated and will be removed in Diffusers v1.0.0. We recommend migrating to PyTorch classes or pinning your version of Diffusers.rr r dropout_probr r n_headsd_headdepthrrrrrr)loggerwarningrangerr r rr rappendrrrrrrrresnets attentionsrrdownsamplers_0)selfr$r%ir res_block attn_blockr+^/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/models/unets/unet_2d_blocks_flax.pysetupBs>   zFlaxCrossAttnDownBlock2D.setupcCsdd}t|j|jD]\}}||||d}||||d}||f7}q |jr.||}||f7}||fSNr+ deterministic)zipr$r%rr&)r' hidden_statestembencoder_hidden_statesr0 output_statesresnetattnr+r+r,__call__is   z!FlaxCrossAttnDownBlock2D.__call__NT)__name__ __module__ __qualname____doc__int__annotations__r floatrrrboolrrrrjnpfloat32rrr-r8r+r+r+r,r s          'r c@sheZdZUdZeed<eed<dZeed<dZeed<dZ e ed <e j Z e j ed <d d Zdd dZdS)FlaxDownBlock2Dab Flax 2D downsizing block Parameters: in_channels (:obj:`int`): Input channels out_channels (:obj:`int`): Output channels dropout (:obj:`float`, *optional*, defaults to 0.0): Dropout rate num_layers (:obj:`int`, *optional*, defaults to 1): Number of attention blocks layers add_downsample (:obj:`bool`, *optional*, defaults to `True`): Whether to add downsampling layer before each final output dtype (:obj:`jnp.dtype`, *optional*, defaults to jnp.float32): Parameters `dtype` r r r r rrTrrcCsxtdg}t|jD]}|dkr|jn|j}t||j|j|jd}| |q ||_ |j r:t |j|jd|_ dSdS)Nrrrr)r r!r"rr r rr rr#r$rrr&)r'r$r(r r)r+r+r,r-s" zFlaxDownBlock2D.setupcCsJd}|jD]}||||d}||f7}q|jr!||}||f7}||fSr.)r$rr&)r'r2r3r0r5r6r+r+r,r8s    zFlaxDownBlock2D.__call__Nr9)r:r;r<r=r>r?r r@rrrArBrCrr-r8r+r+r+r,rDxs    rDc@seZdZUdZeed<eed<eed<dZeed<dZeed<dZ eed <d Z e ed <d Z e ed <d Z e ed<d Ze ed<d Ze ed<ejZejed<dZeed<ddZdddZdS)FlaxCrossAttnUpBlock2Da" Cross Attention 2D Upsampling block - original architecture from Unet transformers: https://huggingface.co/papers/2103.06104 Parameters: in_channels (:obj:`int`): Input channels out_channels (:obj:`int`): Output channels dropout (:obj:`float`, *optional*, defaults to 0.0): Dropout rate num_layers (:obj:`int`, *optional*, defaults to 1): Number of attention blocks layers num_attention_heads (:obj:`int`, *optional*, defaults to 1): Number of attention heads of each spatial transformer block add_upsample (:obj:`bool`, *optional*, defaults to `True`): Whether to add upsampling layer before each final output use_memory_efficient_attention (`bool`, *optional*, defaults to `False`): enable memory efficient attention https://huggingface.co/papers/2112.05682 split_head_dim (`bool`, *optional*, defaults to `False`): Whether to split the head dimension into a new axis for the self-attention computation. In most cases, enabling this flag should speed up the computation for Stable Diffusion 2.x and Stable Diffusion XL. dtype (:obj:`jnp.dtype`, *optional*, defaults to jnp.float32): Parameters `dtype` r r prev_output_channelr r rrrT add_upsampleFrrrrrrc Cstdg}g}t|jD]I}||jdkr|jn|j}|dkr$|jn|j}t|||j|j|j d}| |t |j|j |j|j |j |j|j|j|j|j d }| |q||_||_|jrlt|j|j d|_dSdS)Nrrrrrr)r r!r"rr r rFrr rr#rrrrrrrr$r%rGr upsamplers_0)r'r$r%r(res_skip_channelsresnet_in_channelsr)r*r+r+r,r-s@   zFlaxCrossAttnUpBlock2D.setupc Csnt|j|jD]%\}}|d}|dd}tj||fdd}||||d}||||d}q|jr5||}|SN)axisr/)r1r$r%rB concatenaterGrH) r'r2res_hidden_states_tupler3r4r0r6r7res_hidden_statesr+r+r,r8s  zFlaxCrossAttnUpBlock2D.__call__Nr9)r:r;r<r=r>r?r r@rrrGrArrrrrBrCrrr-r8r+r+r+r,rEs"          (rEc@speZdZUdZeed<eed<eed<dZeed<dZeed<d Z e ed <e j Z e j ed <d d ZdddZdS) FlaxUpBlock2Da Flax 2D upsampling block Parameters: in_channels (:obj:`int`): Input channels out_channels (:obj:`int`): Output channels prev_output_channel (:obj:`int`): Output channels from the previous block dropout (:obj:`float`, *optional*, defaults to 0.0): Dropout rate num_layers (:obj:`int`, *optional*, defaults to 1): Number of attention blocks layers add_downsample (:obj:`bool`, *optional*, defaults to `True`): Whether to add downsampling layer before each final output dtype (:obj:`jnp.dtype`, *optional*, defaults to jnp.float32): Parameters `dtype` r r rFr r rrTrGrcCstdg}t|jD]+}||jdkr|jn|j}|dkr"|jn|j}t|||j|j|j d}| |q ||_ |j rIt |j|j d|_dSdS)Nrrrrr)r r!r"rr r rFrr rr#r$rGrrH)r'r$r(rIrJr)r+r+r,r-5s$ zFlaxUpBlock2D.setupcCsT|jD]}|d}|dd}tj||fdd}||||d}q|jr(||}|SrK)r$rBrNrGrH)r'r2rOr3r0r6rPr+r+r,r8Ns   zFlaxUpBlock2D.__call__Nr9)r:r;r<r=r>r?r r@rrGrArBrCrr-r8r+r+r+r,rQs    rQc@seZdZUdZeed<dZeed<dZeed<dZ eed<dZ e ed <dZ e ed <dZ e ed <ejZejed <dZeed <ddZdddZdS)FlaxUNetMidBlock2DCrossAttnaY Cross Attention 2D Mid-level block - original architecture from Unet transformers: https://huggingface.co/papers/2103.06104 Parameters: in_channels (:obj:`int`): Input channels dropout (:obj:`float`, *optional*, defaults to 0.0): Dropout rate num_layers (:obj:`int`, *optional*, defaults to 1): Number of attention blocks layers num_attention_heads (:obj:`int`, *optional*, defaults to 1): Number of attention heads of each spatial transformer block use_memory_efficient_attention (`bool`, *optional*, defaults to `False`): enable memory efficient attention https://huggingface.co/papers/2112.05682 split_head_dim (`bool`, *optional*, defaults to `False`): Whether to split the head dimension into a new axis for the self-attention computation. In most cases, enabling this flag should speed up the computation for Stable Diffusion 2.x and Stable Diffusion XL. dtype (:obj:`jnp.dtype`, *optional*, defaults to jnp.float32): Parameters `dtype` r r r rrrFrrrrrc Cstdt|j|j|j|jdg}g}t|jD]/}t|j|j |j|j |j |j |j |j |jd}||t|j|j|j|jd}||q||_||_dS)Nrr)r rrrrrrr)r r!rr r rr"rrrrrrrr#r$r%)r'r$r%_r*r)r+r+r,r-~s@     z!FlaxUNetMidBlock2DCrossAttn.setupTcCsP|jd||}t|j|jddD]\}}||||d}||||d}q|S)Nrrr/)r$r1r%)r'r2r3r4r0r7r6r+r+r,r8s z$FlaxUNetMidBlock2DCrossAttn.__call__Nr9)r:r;r<r=r>r?r r@rrrrArrrBrCrrr-r8r+r+r+r,rR]s        *rR) flax.linenlinennn jax.numpynumpyrButilsrattention_flaxr resnet_flaxrrr get_loggerr:r Moduler rDrErQrRr+r+r+r,s     ^?aE