o ϯi;@sddlmZddlZddlmZddlmZmZmZm Z m Z m Z m Z ddl Z ddlmZddlmZmZddlmZGdd d ZGd d d Zd dee jjjdfdddZdS)) annotationsN) defaultdict)AnyCallableDictOptionalSetTupleType) g_pathmgr)ClipInfo ClipSampler)LabeledVideoDatasetc@steZdZdZeeddZdZdZe ddd Z e d d!ddZ e d d"ddZ e d#ddZe d$ddZd S)%AvaLabeledVideoFramePathsz Pre-processor for Ava Actions Dataset stored as image frames - `_` This class handles the parsing of all the necessary csv files containing frame paths and frame labels. iiiinprcCs|d}|d}|d}g}g}g}i}tt|D]P} || || } } d|| d|| d|| d|| d} | |vrXt||| <||| |g|g|| } || | || | q|||d S) Nlabels extra_infoboxesz{:.2f},{:.2f},{:.2f},{:.2f}r)rrr)rangelenformatappend)clsrrrr labels_aggextra_info_agg boxes_aggbb_dicti box_labelbox_extra_infobbox_keyidxr&I/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/ava.py_aggregate_bboxes_labelss0(   z2AvaLabeledVideoFramePaths._aggregate_bboxes_labelsNframe_paths_filestrframe_labels_filevideo_path_prefixlabel_map_file Optional[str]returncCs|dur t|\}}nd}t||\}}} t|| |} g} | D]7} | | D].} | | | }t|ddkrZt|}| |d<| |d<tj || d}| ||fq,q$| S)a Args: frame_labels_file (str): Path to the file containing containing labels per key frame. Acceptible file formats are, Type 1: Type 2: frame_paths_file (str): Path to a file containing relative paths to all the frames in the video. Each line in the file is of the form video_path_prefix (str): Path to be augumented to the each relative frame path to get the global frame path. label_map_file (str): Path to a .pbtxt containing class id's and class names. If not set, label_map is not loaded and bbox labels are not pruned based on allowable class_id's in label_map. Returs: A list of tuples of the the form (video_frames directory, label dictionary). Nrr video_index clip_index) rread_label_mapload_image_listsload_and_parse_labels_csvkeysrr(ospathdirnamer)rr)r+r,r-_allowed_class_ids image_pathsvideo_idx_to_namevideo_name_to_idxvideo_frame_labelslabeled_video_pathsvideo_idframe_video_secrvideo_frames_dirr&r&r'from_csv@s<     z"AvaLabeledVideoFramePaths.from_csvr=dictr: Optional[Set]c CsPi}t|d}|D]}|d}|d}||}t|d} | tjdks0| tjdkr1q | tj} tt t|dd} |ddkrIdnt |d} |d urX| |vrXq t|d } ||vrfi||<| ||vrtt t||| <||| d  | ||| d  | ||| d  | q Wd |S1swY|S)av Parses AVA per frame labels .csv file. Args: frame_labels_file (str): Path to the file containing labels per key frame. Acceptible file formats are, Type 1: Type 2: video_name_to_idx (dict): Dictionary mapping video names to indices. allowed_class_ids (set): A set of integer unique class (bbox label) id's that are allowed in the dataset. If not set, all class id's are allowed in the bbox labels. Returns: (dict): A dictionary of dictionary containing labels per each keyframe in each video. Here, the label for each keyframe is again a dict of the form, { 'labels': a list of bounding boxes 'boxes':a list of action lables for the bounding box 'extra_info': ist of extra information cotaining either detections iou's or person id's depending on the csv format. } r,rrrNrrr) r openstripsplitfloatrAVA_VALID_FRAMESAVA_VIDEO_START_SEClistmapintrr) r+r=r: labels_dictflinerow video_name video_idx frame_secbboxlabelrr&r&r'r4s8!     &&z3AvaLabeledVideoFramePaths.load_and_parse_labels_csvr cCsg}i}g}t|dN}||D]?}|}t|dks!J|d}||vr;t|} | ||<|||i||} t|d} tj ||d|| | <qWdn1s]wYg} t t|D]} | gt || }|D] }| | || |qyqj| ||fS)aU Loading image paths from the corresponding file. Args: frame_paths_file (str): Path to a file containing relative paths to all the frames in the video. Each line in the file is of the form video_path_prefix (str): Path to be augumented to the each relative frame path to get the global frame path. Returns: (tuple): A tuple of the following, image_paths_list: List of list containing absolute frame paths. Wherein the outer list is per video and inner list is per timestamp. video_idx_to_name: A dictionary mapping video index to name video_name_to_idx: A dictionary maoping video name to index rFrrrN) r rLreadlinerNrrrTr6r7joinrsorted)r)r,r;r=r<rVrWrXrYr%data_keyframe_idimage_paths_listr! sorted_keyskeyr&r&r'r3s:      z*AvaLabeledVideoFramePaths.load_image_listscCsi}t}d}d}t|d;}|D]-}|dr!|dd}q|ds+|dr?t|dd }|||<||qWd ||fS1sMwY||fS) a Read label map and class ids. Args: label_map_file (str): Path to a .pbtxt containing class id's and class names Returns: (tuple): A tuple of the following, label_map (dict): A dictionary mapping class id to the associated class names. class_ids (set): A set of integer unique class id's rJrFz name:"rz id:z label_id: rHN)setr rL startswithrNrTrMadd)r- label_map class_idsnameclass_idrVrWr&r&r'r2s$    z(AvaLabeledVideoFramePaths.read_label_map)rrN) r)r*r+r*r,r*r-r.r/r)r+r*r=rDr:rE)r)r*r,r*r/r )r-r*r/r )__name__ __module__ __qualname____doc__rRrrPFPSrQ classmethodr(rC staticmethodr4r3r2r&r&r&r'rs" #?I 4rc@s.eZdZdZdddZdddZdddZdS)TimeStampClipSamplerz A sepcialized clip sampler for sampling video clips around specific timestamps. This is particularly used in datasets like Ava wherein only a specific subset of clips in the video have annotations clip_samplerr r/NonecCs ||_dS)z Args: clip_sampler (`pytorchvideo.data.ClipSampler`): Strategy used for sampling between the untrimmed clip boundary. N)ry)selfryr&r&r'__init__"s zTimeStampClipSampler.__init__last_clip_timerOvideo_duration annotationDict[str, Any]r cCs0|d}||jjd}t|||jjdddS)a  Args: last_clip_time (float): Not used for TimeStampClipSampler. video_duration: (float): Not used for TimeStampClipSampler. annotation (Dict): Dict containing time step to sample aroud. Returns: clip_info (ClipInfo): includes the clip information of (clip_start_time, clip_end_time, clip_index, aug_index, is_last_clip). The times are in seconds. clip_index, aux_index and is_last_clip are always 0, 0 and True, respectively. r1g@rT)ry_clip_durationr )r{r}r~rcenter_frame_secclip_start_secr&r&r'__call__*s  zTimeStampClipSampler.__call__cCsdSrpr&)r{r&r&r'resetAszTimeStampClipSampler.resetN)ryr r/rz)r}rOr~rOrrr/r )r/rz)rqrrrsrtr|rrr&r&r&r'rxs   rxrJr)r*r+r,r-r.ryr video_samplerType[torch.utils.data.Sampler] transformOptional[Callable[[dict], Any]]r/rzcCs&t||||}t|t|||ddS)a Args: frame_paths_file (str): Path to a file containing relative paths to all the frames in the video. Each line in the file is of the form frame_labels_file (str): Path to the file containing containing labels per key frame. Acceptible file formats are, Type 1: Type 2: video_path_prefix (str): Path to be augumented to the each relative frame path to get the global frame path. label_map_file (str): Path to a .pbtxt containing class id's and class names. If not set, label_map is not loaded and bbox labels are not pruned based on allowable class_id's in label_map. clip_sampler (ClipSampler): Defines how clips should be sampled from each video. video_sampler (Type[torch.utils.data.Sampler]): Sampler for the internal video container. This defines the order videos are decoded and, if necessary, the distributed split. transform (Optional[Callable]): This callable is evaluated on the clip output and the corresponding bounding boxes before the clip and the bounding boxes are returned. It can be used for user defined preprocessing and augmentations to the clips. If transform is None, the clip and bounding boxes are returned as it is. F)r?ryrr decode_audio)rrCrrx)r)r+r,r-ryrrr?r&r&r'AvaEs&r)r)r*r+r*r,r*r-r.ryrrrrrr/rz) __future__rr6 collectionsrtypingrrrrrr r torchiopath.common.file_ior pytorchvideo.data.clip_samplingr r 'pytorchvideo.data.labeled_video_datasetrrrxutilsdata RandomSamplerrr&r&r&r's$  $  -