o Gi:@sddlZddlmZddlmZddlmZmZddlm Z ddl m Z dd l m Z d d lmZd d lmZeeZ ddejdejdBdefddZGddde ZdS)N)PipelineImageInput)AutoencoderKLLTXVideo) deprecate get_logger) randn_tensor)VideoProcessor)DiffusionPipeline)LTXLatentUpsamplerModel)LTXPipelineOutputsampleencoder_output generator sample_modecCsRt|dr|dkr|j|St|dr|dkr|jSt|dr%|jStd)N latent_distrargmaxlatentsz3Could not access latents of provided encoder_output)hasattrrrmoderAttributeError)rrrrh/home/ubuntu/.local/lib/python3.10/site-packages/diffusers/pipelines/ltx/pipeline_ltx_latent_upsample.pyretrieve_latentss   rcseZdZdZdededdffdd Z      d9d ejdBd e d ej dBd ej dBd ej dBdejdBdejfddZ d:dejdejdefddZdejdedejfddZe d:dejdejdejdedejf ddZe d:dejdejdejdedejf ddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Ze  * +  ,  , ,  - .d;d eedBd/e d0e dejdBd1eeeBd2eeeBdBd3ed4ed ej eej BdBd5edBd6efd7d8ZZS)<LTXLatentUpsamplePipelinevaelatent_upsamplerreturnNcsft|j||dt|dddur|jjnd|_t|dddur'|jjnd|_t |jd|_ dS)N)rrr )vae_scale_factor) super__init__register_modulesgetattrrspatial_compression_ratiovae_spatial_compression_ratiotemporal_compression_ratiovae_temporal_compression_ratiorvideo_processor)selfrr __class__rrr$/s z"LTXLatentUpsamplePipeline.__init__r video batch_sizedtypedevicerrcs|dur |j||dSj|jjdttr:t|kr,tdtd|dfddt|D}n fddD}tj |dd |} |jj jj }|S) Nr2r1z/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.cs,g|]}tj|d|qSrrrencode unsqueeze).0irr,r/rr Ts z=LTXLatentUpsamplePipeline.prepare_latents..cs$g|]}tj|dqSr4r5)r8vid)rr,rrr;Xs$rdim) torr1 isinstancelistlen ValueErrorrangetorchcat_normalize_latents latents_mean latents_std)r,r/r0r1r2rr init_latentsrr:rprepare_latents@s"    z)LTXLatentUpsamplePipeline.prepare_latents?reference_latentsfactorc Cs|}t|dD]8}t|dD].}tj|||fdd\}}tj|||fdd\} } |||f| | |||||f<qq t|||}|S)a5 Applies Adaptive Instance Normalization (AdaIN) to a latent tensor based on statistics from a reference latent tensor. Args: latent (`torch.Tensor`): Input latents to normalize reference_latents (`torch.Tensor`): The reference latents providing style statistics. factor (`float`): Blending factor between original and transformed latent. Range: -10.0 to 10.0, Default: 1.0 Returns: torch.Tensor: The transformed latent tensor rr Nr=)clonerDsizerEstd_meanlerp) r,rrMrNresultr9cr_sdr_meani_sdi_meanrrradain_filter_latent^s&z-LTXLatentUpsamplePipeline.adain_filter_latent compressioncCsD|d}t|}td||d}dd||}||}|S)aL Applies a non-linear tone-mapping function to latent values to reduce their dynamic range in a perceptually smooth way using a sigmoid-based compression. This is useful for regularizing high-variance latents or for conditioning outputs during generation, especially when controlling dynamic behavior with a `compression` factor. Args: latents : torch.Tensor Input latent tensor with arbitrary shape. Expected to be roughly in [-1, 1] or [0, 1] range. compression : float Compression strength in the range [0, 1]. - 0.0: No tone-mapping (identity transform) - 1.0: Full compression effect Returns: torch.Tensor The tone-mapped latent tensor of the same shape as input. g?g@rLg?)rEabssigmoid)r,rrZ scale_factor abs_latents sigmoid_termscalesfilteredrrrtone_map_latentszs  z*LTXLatentUpsamplePipeline.tone_map_latentsrHrIscaling_factorcCsP|ddddd|j|j}|ddddd|j|j}||||}|SNr viewr?r2r1rrHrIrcrrrrGz,LTXLatentUpsamplePipeline._normalize_latentscCsP|ddddd|j|j}|ddddd|j|j}||||}|Srdrfrhrrr_denormalize_latentsriz.LTXLatentUpsamplePipeline._denormalize_latentscC*d|jjd}tdd||jdS)z Enable sliced VAE decoding. When this option is enabled, the VAE will split the input tensor in slices to compute decoding in several steps. This is useful to save some memory and allow larger batch sizes. z%Calling `enable_vae_slicing()` on a `zl` is deprecated and this method will be removed in a future version. Please use `pipe.vae.enable_slicing()`.enable_vae_slicing0.40.0N)r.__name__rrenable_slicingr, depr_messagerrrrlz,LTXLatentUpsamplePipeline.enable_vae_slicingcCrk)z Disable sliced VAE decoding. If `enable_vae_slicing` was previously enabled, this method will go back to computing decoding in one step. z&Calling `disable_vae_slicing()` on a `zm` is deprecated and this method will be removed in a future version. Please use `pipe.vae.disable_slicing()`.disable_vae_slicingrmN)r.rnrrdisable_slicingrprrrrsrrz-LTXLatentUpsamplePipeline.disable_vae_slicingcCrk)a Enable tiled VAE decoding. When this option is enabled, the VAE will split the input tensor into tiles to compute decoding and encoding in several steps. This is useful for saving a large amount of memory and to allow processing larger images. z$Calling `enable_vae_tiling()` on a `zk` is deprecated and this method will be removed in a future version. Please use `pipe.vae.enable_tiling()`.enable_vae_tilingrmN)r.rnrr enable_tilingrprrrrusz+LTXLatentUpsamplePipeline.enable_vae_tilingcCrk)z Disable tiled VAE decoding. If `enable_vae_tiling` was previously enabled, this method will go back to computing decoding in one step. z%Calling `disable_vae_tiling()` on a `zl` is deprecated and this method will be removed in a future version. Please use `pipe.vae.disable_tiling()`.disable_vae_tilingrmN)r.rnrrdisable_tilingrprrrrwrrz,LTXLatentUpsamplePipeline.disable_vae_tilingcCs||jdks||jdkrtd|d|d|dur%|dur%td|dur1|dur1tdd|kr>dksCtdtddS) Nrz8`height` and `width` have to be divisible by 32 but are z and .z1Only one of `video` or `latents` can be provided.z/One of `video` or `latents` has to be provided.r z8`tone_map_compression_ratio` must be in the range [0, 1])r(rC)r,r/heightwidthrtone_map_compression_ratiorrr check_inputssz<XLatentUpsamplePipeline.check_inputspilTrzr{decode_timestepdecode_noise_scale adain_factorr| output_type return_dictc  CsH|j|||||d|durd} n|jd} |j} |dur[t|}||jdkrJ||j|jd}|d|}td|jdt|d|d|jj|||d}|j | t j d }|j || t j | | |d }| ||jj|jj|jjj}| |jj}||}|d kr||||}n|}|d kr|||}| d kr|||jj|jj|jjj}|}ne|jjjsd}nJt|j| | |jd }t|ts|g| }|dur|}n t|ts|g| }t j|| |jd }t j|| |jd ddddddf}d||||}|jj||ddd}|jj|| d}|| s|fSt |dS)N)r/rzr{rr|r rz-Video length expected to be of the form `k * z + 1` but is z. Truncating to z frames.)rzr{r3)r/r0r1r2rrrlatent)rr2r1F)r)r)frames)!r}shape_execution_devicerBr*loggerwarningr+preprocess_videor?rEfloat32rKrjrrHrIconfigrcrr1rYrbrGtimestep_conditioningrr@rAtensordecodepostprocess_videomaybe_free_model_hooksr )r,r/rzr{rrrrr|rrrr0r2 num_frameslatents_upsampledtimestepnoiserrr__call__s           z"LTXLatentUpsamplePipeline.__call__)Nr NNNN)rL) Nr~rNrNrrNrT) rn __module__ __qualname__model_cpu_offload_seqrr r$rETensorintr1r2 GeneratorrKfloatrYrb staticmethodrGrjrlrsrurwr}no_gradrArstrboolr __classcell__rrr-rr,s                r)Nr)rEimage_processorrmodelsrutilsrrutils.torch_utilsrr+rpipeline_utilsr modeling_latent_upsamplerr pipeline_outputr rnrrrrrrrrrrs&