o i1@sNddlmZddlZddlmZddlZddlmZmZddl m Z ddl m Z ddl mZddlmZdd lmZdd lmZmZdd lmZdd lmZdd lmZddlmZmZmZm Z m!Z!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+m,Z,ddl-m.Z.m/Z/ddl0m1Z1GdddZ2e3dddZ4e5dkre4dSdS)) annotationsN)Iterator)MultiModalGuiderMultiModalGuiderParams)GaussianNoiser) LTX2Scheduler)TemporalRegionMask)LoraPathStrengthAndSDOps)Registry) TilingConfigget_video_chunks_number)QuantizationPolicy)SpatioTemporalScaleFactors)video_editing_arg_parser)AudioConditioner AudioDecoderDiffusionStageImageConditioner PromptEncoder VideoDecoder)DISTILLED_SIGMA_VALUES detect_params)GuidedDenoiserSimpleDenoiser)audio_latent_from_file get_devicevideo_latent_from_file) encode_videoget_videostream_metadata) ModalitySpecc @sHeZdZdZ     d1d2ddZddddddddddd d3d/d0ZdS)4RetakePipelinea=Regenerate a time region (retake) of an existing video. Given a source video file and a time window ``[start_time, end_time]`` (in seconds), this pipeline keeps the video/audio outside that window unchanged and *regenerates* the content inside the window from a text prompt using the LTX-2 diffusion model. Parameters ---------- checkpoint_path : str Path to the LTX-2 model checkpoint. gemma_root : str Root directory containing Gemma text-encoder weights. loras : list[LoraPathStrengthAndSDOps] Optional LoRA configs applied to the transformer. device : torch.device Target device (default: CUDA if available). quantization : QuantizationPolicy | None Optional quantization policy for the transformer. distilled : bool Set to ``True`` if using distilled model or passing distillation lora with full model. If set to ``True``, distilled sigma schedule (``DISTILLED_SIGMA_VALUES``) and a simple (non-guided) denoising function will be used during ``__call__``. NTFcheckpoint_pathstr gemma_rootloraslist[LoraPathStrengthAndSDOps]devicetorch.device | None quantizationQuantizationPolicy | NoneregistryRegistry | None distilledbool torch_compilec Cs|pt|_tj|_||_t|||j|j|d|_t||j|j|d|_ t ||j|j|d|_ t ||j|jt ||||d|_t||j|j|d|_t||j|j|d|_dS)N)r!r#dtyper&r*)r!r/r&r*)r!r/r&r$r(r*r.)rr&torchbfloat16r/r,rprompt_encoderrimage_conditionerraudio_conditionerrtuplestager video_decoderr audio_decoder) selfr!r#r$r&r(r*r,r.r:E/home/ubuntu/LTX-2/packages/ltx-pipelines/src/ltx_pipelines/retake.py__init__CsV   zRetakePipeline.__init__() negative_promptnum_inference_stepsvideo_guider_paramsaudio_guider_paramsregenerate_videoregenerate_audioenhance_prompt tiling_configstreaming_prefetch_countmax_batch_size video_pathprompt start_timefloatend_timeseedintr@rArBMultiModalGuiderParams | NonerCrDrErFrGTilingConfig | NonerH int | NonerIreturn+tuple[Iterator[torch.Tensor], torch.Tensor]c $ s||krtd|d|dtjjd|}t|d}jtfdd} fdd}j rC|gn||g}j || ||d }|d j |d j }}t|| rht||jd gng|| d }t||d ur| rt||jd gng||d uo| d }j rttjtjjd}t||d}n-tj|djtjjd}|dj |dj }}t||d}t| |d}t||||d}j|||jjjj||||d \} }!| j| |}" |!j}#|"|#fS)aRegenerate ``[start_time, end_time]`` of the source video (retake). Parameters ---------- video_path : str Path to the source video file (must contain video; audio is optional). prompt : str Text prompt describing the *regenerated* section. start_time, end_time : float Time window (in seconds) of the section to regenerate. seed : int Random seed for reproducibility. negative_prompt : str Negative prompt for CFG guidance (ignored in distilled mode). num_inference_steps : int Number of Euler denoising steps (ignored in distilled mode which uses a fixed 8-step schedule). video_guider_params, audio_guider_params : MultiModalGuiderParams | None Guidance parameters for video and audio modalities. Ignored in distilled mode. regenerate_video : bool If ``True`` (default), regenerate video inside ``[start_time, end_time]``. If ``False``, video is preserved as-is (no regeneration). regenerate_audio : bool If True, regenerate audio in the [start_time, end_time] window; if False, audio is preserved as-is (no regeneration). enhance_prompt : bool Whether to enhance the prompt via the text encoder. Returns ------- tuple[Iterator[torch.Tensor], torch.Tensor] ``(video_frames_iterator, audio_waveform)`` z start_time (z) must be less than end_time ())r&) generatorct|jdS)N) video_encoder file_path output_shaper/r&)rr&encr/r[r9rJr:r;z)RetakePipeline.__call__..crX)N) audio_encoderrZr[r/r&)rr&r\r^r:r;r_r`)enhance_first_promptenhance_prompt_seedrHr)rLrNfps)context conditioningsinitial_latentfrozenN)r/r&) v_context a_context)stepsr?)paramsnegative_context)rirj video_guider audio_guider) denoisersigmasnoiserwidthheightframesrdvideoaudiorHrI)! ValueErrorr0 Generatorr& manual_seedrr/rr3r4r,r2video_encodingaudio_encodingrrrdtensorrtofloat32rrexecuterrr6rsrtrur7latentr8)$r9rJrKrLrNrOr@rArBrCrDrErFrGrHrIrWrrinitial_video_latentinitial_audio_latentprompts_to_encodecontexts v_context_p a_context_pvideo_modality_specaudio_modality_specrqrp v_context_n a_context_nrnro video_state audio_state decoded_video decoded_audior:r^r;__call__~s3        zRetakePipeline.__call__)NNNTF)r!r"r#r"r$r%r&r'r(r)r*r+r,r-r.r-) rJr"rKr"rLrMrNrMrOrPr@r"rArPrBrQrCrQrDr-rEr-rFr-rGrRrHrSrIrPrTrU)__name__ __module__ __qualname____doc__r<rr:r:r:r;r *s& Cr rTNonec Csfttjtdd}d|_|}|j|jkrt dt }t |j }|jd|jdkrJ|jd|j|jd}t d|jd|d |jd dksX|jd dkret d |jd |jd t|j|j|jrrt|jnd|j|j|jd}t|j}t }||j |j|j|j|j|j|j||j |j!d \}} t"|j|} t#|t$|j%| |j&| ddS)z6CLI entry point for retake (regenerate a time region).T)r,z7Retake: regenerate a time region of a video with LTX-2.z%start_time must be less than end_timer?rz8Video frame count must satisfy 8k+1 (e.g. 97, 193). Got z; use a video with z frames. z4Video width and height must be multiples of 32. Got x.r:)r!r#r$r(r,r.) rJrKrLrNrOrBrCrGrHrI)rvrdrw output_pathvideo_chunks_numberN)'logging getLoggersetLevelINFOr description parse_argsrLrNrxrdefaultrrJrutimersrtr distilled_checkpoint_pathr#lorar5r(r,compilerr rKrOrBrCrHrIr rrPrdr) parserargs video_scalesrcsnappedpipelinerlrG video_iterrwrr:r:r;mainsZ      r__main__)rTr)6 __future__rrcollections.abcrr0ltx_core.components.guidersrrltx_core.components.noisersrltx_core.components.schedulersr+ltx_core.conditioning.types.noise_mask_condrltx_core.loaderr ltx_core.loader.registryr ltx_core.model.video_vaer r ltx_core.quantizationr ltx_core.typesrltx_pipelines.utils.argsrltx_pipelines.utils.blocksrrrrrrltx_pipelines.utils.constantsrrltx_pipelines.utils.denoisersrrltx_pipelines.utils.helpersrrrltx_pipelines.utils.media_iorrltx_pipelines.utils.typesrr inference_moderrr:r:r:r;s6            l 5