o ϯi '@sddlmZddlZddlZddlZddlZddlZddlmZm Z m Z m Z ddl Z ddlZddlZddlmZddlmZddlmZddlmZzddlZWn ey[d ZYnwd ZeeZGd d d eZ ddddZdS)) annotationsN)CallableDictListOptional) g_pathmgr)optional_threaded_foreach) thwc_to_cthw)VideoFTc@seZdZdZ   d,d-ddZe   d.d/ddZe  d0d1ddZed2ddZ ed2ddZ d3d d!Z d4d5d'd(Z d6d*d+Z dS)7 FrameVideoz FrameVideo is an abstractions for accessing clips based on their start and end time for a video where each frame is stored as an image. PathManager is used for frame image reading, allowing non-local uri's to be used. NFdurationfloatfpsvideo_frame_to_path_fnCallable[[int], str]video_frame_paths List[str]multithreaded_ioboolreturnNonecCsdtstd||_||_||_|du|duksJd||_||_tj tj |j dd|_ dS)az Args: duration (float): the duration of the video in seconds. fps (float): the target fps for the video. This is needed to link the frames to a second timestamp in the video. video_frame_to_path_fn (Callable[[int], str]): a function that maps from a frame index integer to the file path where the frame is located. video_frame_paths (List[str]): Dictionary of frame paths for each index of a video. multithreaded_io (bool): controls whether parllelizable io operations are performed across multiple threads. zVopencv2 is required to use FrameVideo. Please install with 'pip install opencv-python'NzGOnly one of video_frame_to_path_fn or video_frame_paths can be providedr) frame_index) _HAS_CV2 ImportError _duration_fps_multithreaded_io_video_frame_to_path_fn_video_frame_pathsospathbasenamedirname_video_frame_to_path_name)selfr rrrrr'Q/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/frame_video.py__init__(s" zFrameVideo.__init__>@r!strpath_order_cacheOptional[Dict[str, List[str]]]cs|dur|vr||||StsJdt}dd}|j|dfdd|D}|dur=||<||||S)a6 Args: path (str): path to frame video directory. fps (float): the target fps for the video. This is needed to link the frames to a second timestamp in the video. multithreaded_io (bool): controls whether parllelizable io operations are performed across multiple threads. path_order_cache (dict): An optional mapping from directory-path to list of frames in the directory in numerical order. Used for speedup by caching the frame paths. Nz is not a directorycSsddtd|DS)NcSs g|] }|r t|n|qSr')isdigitint).0cr'r'r( js zCFrameVideo.from_directory..natural_keys..z(\d+))resplit)textr'r'r( natural_keysisz/FrameVideo.from_directory..natural_keys)keycsg|] }tj|qSr')r r!join)r0fr!r'r(r2msz-FrameVideo.from_directory..)from_frame_pathsrisdirlssort)clsr!rrr,rel_frame_pathsr6 frame_pathsr'r:r(from_directoryPs  zFrameVideo.from_directorycCs,t|dks Jd|t|||||dS)a Args: video_frame_paths (List[str]): a list of paths to each frames in the video. fps (float): the target fps for the video. This is needed to link the frames to a second timestamp in the video. multithreaded_io (bool): controls whether parllelizable io operations are performed across multiple threads. rzvideo_frame_paths is empty)rr)len)r?rrrr'r'r(r;rs zFrameVideo.from_frame_pathscC|jSN)r%r&r'r'r(nameszFrameVideo.namecCrD)zZ Returns: duration: the video's duration/end-time in seconds. )rrFr'r'r(r szFrameVideo.durationtime_secr/cCst|j|SrE)mathceilr)r&rHr'r'r(_get_frame_index_for_timesz$FrameVideo._get_frame_index_for_time start_secend_sec frame_filter*Optional[Callable[[List[int]], List[int]]]!Dict[str, Optional[torch.Tensor]]c s|dks |jkrtd|d|djddSt|j}|}t|tj}tt||}|r>||}fdd|D}t |j d }t | t j}||dd S) a Retrieves frames from the stored video at the specified start and end times in seconds (the video always starts at 0 seconds). Returned frames will be in [start_sec, end_sec). Given that PathManager may be fetching the frames from network storage, to handle transient errors, frame reading is retried N times. Note that as end_sec is exclusive, so you may need to use `get_clip(start_sec, duration + EPS)` to get the last frame. Args: start_sec (float): the clip start time in seconds end_sec (float): the clip end time in seconds frame_filter (Optional[Callable[List[int], List[int]]]): function to subsample frames in a clip before loading. If None, no subsampling is peformed. Returns: clip_frames: A tensor of the clip's RGB frames with shape: (channel, time, height, width). The frames are of type torch.float32 and in the range [0 - 255]. Raises an exception if unable to load images. clip_data: "video": A tensor of the clip's RGB frames with shape: (channel, time, height, width). The frames are of type torch.float32 and in the range [0 - 255]. Raises an exception if unable to load images. "frame_indices": A list of indices for each frame relative to all frames in the video. Returns None if no frames are found. rzNo frames found within z and z, seconds. Video startsat time 0 and ends at .Ncsg|]}|qSr')r$r0irFr'r(r2sz'FrameVideo.get_clip..) multithreaded)video frame_indicesaudio)rloggerwarningminrKrCrlistrange_load_images_with_retriesrr totorchfloat32) r&rLrMrNstart_frame_indexend_frame_indexrV clip_paths clip_framesr'rFr(get_clips*#   zFrameVideo.get_cliprcCs(|jr||S|jr|j|Std)Nz@One of _video_frame_to_path_fn or _video_frame_paths must be set)rr Exception)r&rr'r'r(r$s  zFrameVideo._video_frame_to_path)NNF) r rrrrrrrrrrr)r*FN)r!r+rrr,r-)r*F)rrrrrr)rr)rHrrr/rE)rLrrMrrNrOrrP)rr/rr+)__name__ __module__ __qualname____doc__r) classmethodrBr;propertyrGr rKrer$r'r'r'r(r !s.  (!    . image_indexr/ image_pathr+rrc stD]L}t|d }t|tj}tj|tj d}t |tj }Wdn1s/wY|dur?||<dSt d|ddtdqdS)Nrb)flagszReading attempt /z failed.gư>)r\ropennp frombufferreaduint8cv2imdecode IMREAD_COLORcvtColor COLOR_BGR2RGBloggingrYtimesleep)rqrrrSr9img_strimg_bgrimg_rgbimgsror'r( fetch_images  z._load_images_with_retries..fetch_imagecss|]}|duVqdSrEr')r0imgr'r'r( sz,_load_images_with_retries..zFailed to load images from {}N)rqr/rrr+rr) r enumerateanyrfformatr_ as_tensorrwstack)rnrorTrr'rr(r]s  r])rmT)rnrror/rTrrrp) __future__rrrIr r3rtypingrrrrnumpyrwr_torch.utils.dataiopath.common.file_iorpytorchvideo.data.utilsrutilsr rUr r{rr getLoggerrgrXr r]r'r'r'r(s2        @