o ϯi/@sddlmZddlZddlmZmZmZmZmZm Z m Z ddl Z ddl mZddlmZddlmZddlmZeeZGd d d e jjjZe jjjdd d d fdddZdS)) annotationsN)AnyCallableDictListOptionalTupleType) ClipSampler)VideoPathHandler)LabeledVideoPaths)MultiProcessSamplerc@sXeZdZdZdZejjjdddfdddZ e ddZ e ddZ d ddZ ddZdS)!LabeledVideoDatasetz LabeledVideoDataset handles the storage, loading, decoding and clip sampling for a video dataset. It assumes each video is stored as either an encoded video (e.g. mp4, avi) or a frame video (e.g. a folder of jpg, or png) NTpyavlabeled_video_paths List[Tuple[str, Optional[dict]]] clip_samplerr video_samplerType[torch.utils.data.Sampler] transformOptional[Callable[[dict], Any]] decode_audiobooldecoderstrreturnNonecCs||_||_||_||_||_d|_|tjjj kr(t |_||j|jd|_ n||j|_ d|_ d|_ d|_d|_t|_dS)a Args: labeled_video_paths (List[Tuple[str, Optional[dict]]]): List containing video file paths and associated labels. If video paths are a folder it's interpreted as a frame video, otherwise it must be an encoded video. clip_sampler (ClipSampler): Defines how clips should be sampled from each video. See the clip sampling documentation for more information. video_sampler (Type[torch.utils.data.Sampler]): Sampler for the internal video container. This defines the order videos are decoded and, if necessary, the distributed split. transform (Callable): This callable is evaluated on the clip output before the clip is returned. It can be used for user defined preprocessing and augmentations on the clips. The clip output format is described in __next__(). decode_audio (bool): If True, also decode audio from video. decoder (str): Defines what type of decoder used to decode a video. Not used for frame videos. N) generator) _decode_audio _transform _clip_sampler_labeled_videos_decoder_video_random_generatortorchutilsdata RandomSampler Generator_video_sampler_video_sampler_iter_loaded_video_label _loaded_clip_next_clip_start_timer video_path_handler)selfrrrrrrr3[/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/labeled_video_dataset.py__init__s"    zLabeledVideoDataset.__init__cCs|jS)z Returns: The video sampler that defines video sample order. Note that you'll need to use this property to set the epoch for a torch.utils.data.DistributedSampler. )r,r2r3r3r4rWsz!LabeledVideoDataset.video_samplercCs t|jS)zC Returns: Number of videos in dataset. )lenrr6r3r3r4 num_videos`s zLabeledVideoDataset.num_videosdictc sz|js tt|j|_t|jD]"}|jr|j\}}}n;t|j}z|j|\}}|j j ||j |j d}|||f|_Wnt yW}ztd||WYd}~qd}~ww||j|j|\}}} } } t|tr| ddkri|_g} tt|D] } ||| || }|dus|ddurd|_n| |q||jdur| dD]fdd| D|j<qn | dkr||||_||_|jdup|jddu}t| tr| dsn| s|r|jdd|_d |_|j|rtd |j|q|jd}|jd }||j|| | d ||durd |ini}|jdur0||}|dur0q|Std |jd)a Retrieves the next clip based on the clip sampling strategy and video sampler. Returns: A dictionary with the following format. .. code-block:: text { 'video': , 'label': , 'video_label': 'video_index': , 'clip_index': , 'aug_index': , } rrz-Failed to load video with error: {}; trial {}Nrvideocsg|]}|qSr3r3).0xkeyr3r4 sz0LabeledVideoDataset.__next__..r z Failed to load clip {}; trial {}audio)r; video_name video_index clip_index aug_indexzFailed to load video after z retries.) r-iterrr,range_MAX_CONSECUTIVE_FAILURESr.nextr$r1video_from_pathr!r% Exceptionloggerdebugformatr#r0duration isinstancelistr/r7get_clipappendkeyscloseresetnamer" RuntimeError)r2i_tryr; info_dictrD video_pathe clip_startclip_endrErF is_last_cliploaded_clip_listi clip_dict video_is_nullframes audio_samples sample_dictr3r>r4__next__hs              zLabeledVideoDataset.__next__cCs@d|_tjj}|jdur|dur|j|j}|j||S)N) r-r'r(r)get_worker_infor&seedid manual_seed)r2 worker_info base_seedr3r3r4__iter__s    zLabeledVideoDataset.__iter__)rrrr rrrrrrrrrr)rr9)__name__ __module__ __qualname____doc__rIr'r(r)r*r5propertyrr8rhror3r3r3r4rs ;    }rTr data_pathrrr rrr4Optional[Callable[[Dict[str, Any]], Dict[str, Any]]]video_path_prefixrrrrc Cs(t|}||_t||||||d}|S)a A helper function to create ``LabeledVideoDataset`` object for Ucf101 and Kinetics datasets. Args: data_path (str): Path to the data. The path type defines how the data should be read: * For a file path, the file is read and each line is parsed into a video path and label. * For a directory, the directory structure defines the classes (i.e. each subdirectory is a class). clip_sampler (ClipSampler): Defines how clips should be sampled from each video. See the clip sampling documentation for more information. video_sampler (Type[torch.utils.data.Sampler]): Sampler for the internal video container. This defines the order videos are decoded and, if necessary, the distributed split. transform (Callable): This callable is evaluated on the clip output before the clip is returned. It can be used for user defined preprocessing and augmentations to the clips. See the ``LabeledVideoDataset`` class for clip output format. video_path_prefix (str): Path to root directory with the videos that are loaded in ``LabeledVideoDataset``. All the video paths before loading are prefixed with this path. decode_audio (bool): If True, also decode audio from video. decoder (str): Defines what type of decoder used to decode a video. r:)r from_path path_prefixr) rvrrrrxrrrdatasetr3r3r4labeled_video_datasets *r|)rvrrr rrrrwrxrrrrrrr) __future__rloggingtypingrrrrrrr torch.utils.datar'pytorchvideo.data.clip_samplingr pytorchvideo.data.videor rr r(r getLoggerrprMr)IterableDatasetrr*r|r3r3r3r4s" $     e