o ϯi!@svddlZddlmZddlmZmZmZddlZddl Z ddl m Z m Z m Z ddlmZeeZGdddeZdS) N)Fraction)BinaryIODictOptional) pts_to_secs secs_to_pts thwc_to_cthw)Videoc @seZdZdZ dZ  ddedeededdfd d Z e deefd d Z e de fd dZ ddZde de deeeejffddZ ddedede fddZdS)EncodedVideoTorchVisionz Accessing clips from an encoded video using Torchvision video reading API (torch.ops.video_reader.read_video_from_memory) as the decoding backend. g?NTfile video_name decode_audioreturnc Csttj|tjd|_||_||_| \|_ |_ |_ }|_ |_|_}|durB|durBtt||j |j t||j|j|_dS|durQt||j |j |_dS|dur`t||j|j|_dSdS)N)dtype)torchtensornp frombuffergetvalueuint8 _video_tensor _video_name _decode_audio_torch_vision_decode_video_video_video_time_base_video_start_pts_audio_audio_time_base_audio_start_ptsmaxr _duration)selfr r rvideo_durationaudio_durationr&_/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/encoded_video_torchvision.py__init__sB        z EncodedVideoTorchVision.__init__cC|jS)zQ Returns: name: the name of the stored video if set. )rr#r&r&r'nameIzEncodedVideoTorchVision.namecCr))zZ Returns: duration: the video's duration/end-time in seconds. )r"r*r&r&r'durationQr,z EncodedVideoTorchVision.durationcCsdS)Nr&r*r&r&r'closeYszEncodedVideoTorchVision.close start_secend_secc sd}|jdur&t||j|jddt||j|jddfdd|jD}d}|jrZ|jrZt||j|jddt||j|jddfdd|jD}tj |dd}| tj }|dusdt |dkrvt d |d |d |jd d}|durtt| tj }||d S)a Retrieves frames from the encoded video at the specified start and end times in seconds (the video always starts at 0 seconds). Returned frames will be in [start_sec, end_sec). Note that 1) if you want to avoid float precision issue and need accurate frames, please use Fraction for start_sec and end_sec. 2) As end_sec is exclusive, so you may need to use `get_clip(start_sec, duration + EPS)` to get the last frame. Args: start_sec (float): the clip start time in seconds end_sec (float): the clip end time in seconds Returns: clip_data: A dictionary mapping the entries at "video" and "audio" to a tensors. "video": A tensor of the clip's RGB frames with shape: (channel, time, height, width). The frames are of type torch.float32 and in the range [0 - 255]. "audio": A tensor of the clip's audio samples with shape: (samples). The samples are of type torch.float32 and in the range [0 - 255]. Returns None if no video or audio found within time range. Nceil) round_modec$g|]\}}|kr|kr|qSr&r&.0fpts) video_end_ptsvideo_start_ptsr&r' z4EncodedVideoTorchVision.get_clip..cr3r&r&r4) audio_end_ptsaudio_start_ptsr&r'r:r;r)axiszNo video found within z and z- seconds. Video starts at time 0 and ends at .)videoaudio)rrrrrrrr rcattofloat32lenloggerwarningr-r stack)r#r/r0 video_frames audio_samplesr&)r<r=r8r9r'get_clip\s`     z EncodedVideoTorchVision.get_clipr start_ptsend_ptsc"Csd}d}d\}}}}||} } d\} } d\} }||}}d\}}ztjj|j|jdd||||| | | | |j| |||||}WntyX}zt d|j d||d}~ww|\ }}}}}}}}}}|dkrpt |d }nt |}t t ||}t |d} tt |dt |d}d}d} d}d}!|jr|dkrt |d }!nt |}!t t ||}t |d}tt |dt |d} ||| ||| ||!fS) zH Decode the video in the PTS range [start_pts, end_pts] N)rrrr)rr)rrrrzFailed to decode video of name z. rL)rops video_readerread_video_from_memoryrSEEK_FRAME_MARGINr ExceptionrFrGrintlistzipr)"r#rMrN video_and_pts audio_and_ptswidthheight min_dimension max_dimensionr9r8video_timebase_numvideo_timebase_densampleschannelsr=r<audio_timebase_numaudio_timebase_den tv_resultevframes vframes_pts vtimebase_ vdurationaframes aframe_pts atimebase adurationr$video_time_baseaudio_time_baser%r&r&r'rs     z2EncodedVideoTorchVision._torch_vision_decode_video)NT)rrL)__name__ __module__ __qualname____doc__rRrrstrboolr(propertyr+floatr-r.rrTensorrKrTrr&r&r&r'r sF + Xr )logging fractionsrtypingrrrnumpyrrutilsrrr r@r getLoggerrprFr r&r&r&r's