o ۷i @sZddlZddlZddlZddlZddlmmZddl m Z m Z m Z Gddde Z dS)N)VaeImageProcessoris_valid_imageis_valid_image_imagelistc @seZdZdZddedBdedBdejfddZ dd ejd ede j ejBe e j j Bfd d Zededed edeeeffddZedejdededejfddZdS)VideoProcessorzSimple video processor.Nheightwidthreturnc s,t|tr!t|dtjr!|djdkr!tdttj|dd}t|trBt|dt j rB|djdkrBtdtt j |dd}t|tjt j frU|jdkrUt|}n%t|tr`t |dsdt |rh|g}nt|trvt |drv|}ntdt jfdd|Ddd }|dd d d d }|S)a Preprocesses input video(s). Keyword arguments will be forwarded to `VaeImageProcessor.preprocess`. Args: video (`list[PIL.Image]`, `list[list[PIL.Image]]`, `torch.Tensor`, `np.array`, `list[torch.Tensor]`, `list[np.array]`): The input video. It can be one of the following: * list of the PIL images. * list of list of PIL images. * 4D Torch tensors (expected shape for each tensor `(num_frames, num_channels, height, width)`). * 4D NumPy arrays (expected shape for each array `(num_frames, height, width, num_channels)`). * list of 4D Torch tensors (expected shape for each tensor `(num_frames, num_channels, height, width)`). * list of 4D NumPy arrays (expected shape for each array `(num_frames, height, width, num_channels)`). * 5D NumPy arrays: expected shape for each array `(batch_size, num_frames, height, width, num_channels)`. * 5D Torch tensors: expected shape for each array `(batch_size, num_frames, num_channels, height, width)`. height (`int`, *optional*, defaults to `None`): The height in preprocessed frames of the video. If `None`, will use the `get_default_height_width()` to get default height. width (`int`, *optional*`, defaults to `None`): The width in preprocessed frames of the video. If `None`, will use get_default_height_width()` to get the default width. Returns: `torch.Tensor` of shape `(batch_size, num_channels, num_frames, height, width)`: A 5D tensor holding the batched channels-first video(s). rzPassing `video` as a list of 5d np.ndarray is deprecated.Please concatenate the list along the batch dimension and pass it as a single 5d np.ndarray)axiszPassing `video` as a list of 5d torch.Tensor is deprecated.Please concatenate the list along the batch dimension and pass it as a single 5d torch.TensorzeInput is in incorrect format. Currently, we only support numpy.ndarray, torch.Tensor, PIL.Image.Imagecs$g|]}j|fdqS))rr) preprocess).0imgrkwargsselfrO/home/ubuntu/vllm_env/lib/python3.10/site-packages/diffusers/video_processor.py Vs$z3VideoProcessor.preprocess_video..)dimr) isinstancelistnpndarrayndimwarningswarn FutureWarning concatenatetorchTensorcatrr ValueErrorstackpermute)rvideorrrrrrpreprocess_videos0(( "zVideoProcessor.preprocess_videorr( output_typec Ks|jd}g}t|D]}||dddd}|j||fi|}||q |dkr2t|}|S|dkr=t|}|S|dksHt|d|S) aI Converts a video tensor to a list of frames for export. Keyword arguments will be forwarded to `VaeImageProcessor.postprocess`. Args: video (`torch.Tensor`): The video as a tensor. output_type (`str`, defaults to `"np"`): Output type of the postprocessed `video` tensor. rrrrrptpilz9 does not exist. Please choose one of ['np', 'pt', 'pil']) shaperanger' postprocessappendrr&r"r%) rr(r*r batch_sizeoutputs batch_idx batch_vid batch_outputrrrpostprocess_video]s     z VideoProcessor.postprocess_videoratioscsDt||t|fddd}||}t|dt|dfS)a Returns the binned height and width based on the aspect ratio. Args: height (`int`): The height of the image. width (`int`): The width of the image. ratios (`dict`): A dictionary where keys are aspect ratios and values are tuples of (height, width). Returns: `tuple[int, int]`: The closest binned height and width. cstt|S)N)absfloat)ratioarrrsz:VideoProcessor.classify_height_width_bin..)keyrr)r9minkeysint)rrr7 closest_ratio default_hwrr;rclassify_height_width_binxs z(VideoProcessor.classify_height_width_binsamples new_width new_heightcCs|jd|jd}}||ks||krt||||}t||}t||}|j\}} } } } |ddddd|| | | | }tj|||fddd}||d} | |}||d}||}|d d d d ||| |f}||| | ||ddddd}|S) a4 Resizes and crops a tensor of videos to the specified dimensions. Args: samples (`torch.Tensor`): A tensor of shape (N, C, T, H, W) where N is the batch size, C is the number of channels, T is the number of frames, H is the height, and W is the width. new_width (`int`): The desired width of the output videos. new_height (`int`): The desired height of the output videos. Returns: `torch.Tensor`: A tensor containing the resized and cropped videos. rrrrrbilinearF)sizemode align_cornersN)r-maxrAr'reshapeF interpolate)rErFrG orig_height orig_widthr: resized_widthresized_heightncthwstart_xend_xstart_yend_yrrrresize_and_crop_tensors"  "     z%VideoProcessor.resize_and_crop_tensor)NN)r)__name__ __module__ __qualname____doc__rAr"r#r)strrrrPILImager6 staticmethoddicttuplerDr]rrrrrs"B $$r)rnumpyrrcr"torch.nn.functionalnn functionalrNimage_processorrrrrrrrrs