o
ϯiw @ s d dl Z d dlmZ d dlmZ d dlmZmZm Z m
Z
m Z d dl Z d dl
mZmZmZmZmZ d dlmZmZ d dlmZ G dd d eZG d
d d eZdS ) N)fields)Enum)AnyCallableDictListOptional)EncodedVideoInfo
VideoClipInfoVideoDatasetTypeVideoFrameInfo VideoInfo)
ActionDataEpicKitchenDataset)Videoc @ s e Zd ZdZdS ) ClipSampling N)__name__
__module__ __qualname__RandomOffsetUniform r r ^/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/epic_kitchen_recognition.pyr s r c s e Zd ZdZejejddddfdedededed ed
e d e
e d e e
eef gef d
ef fdd
Zed e e
eef ge
eef f de e
eef ge
eef f fddZed e de ee gee f fddZed
e dede e
eef e
eee f gee f fddZ ZS )EpicKitchenRecognitionz
Action recognition video data set for EpicKitchen-55 Dataset.
This dataset handles the loading, decoding, and clip sampling for the videos.
g @NTvideo_info_file_pathactions_file_path video_data_manifest_file_path
clip_sampling dataset_typeseconds_per_clipframes_per_clip transformmultithreaded_ioc
s ddd t tD ddd t tD ddd t tD dd d t tD d
t||}
t|}|d ur?t|nd } t j
|||||| |
| d d S ) Nz
Args:
video_info_file_path (str):
Path or URI to manifest with basic metadata of each video.
File must be a csv (w/header) with columns:
c S g | ]}|j qS r name.0fr r r
1 z3EpicKitchenRecognition.__init__..z
actions_file_path (str):
Path or URI to manifest with action annotations for each video.
File must ber a csv (w/header) with columns:
c S r# r r$ r& r r r r) 6 r* a
video_data_manifest_file_path (str):
The path to a json file outlining the available video data for the
associated videos. File must be a csv (w/header) with columns either:
For Frame Videos:
c S r# r r$ r& r r r r) = r* z6
For Encoded Videos:
c S r# r r$ r& r r r r) @ r* a
To generate this file from a directory of video frames, see helper
functions in Module: pytorchvideo.data.epic_kitchen.utils
clip_sampling (ClipSampling):
The type of sampling to perform to perform on the videos of the dataset.
dataset_type (VideoDatasetType): The dataformat in which dataset
video data is store (e.g. video frames, encoded video etc).
seconds_per_clip (float): The length of each sampled clip in seconds.
frames_per_clip (Optional[int]): The number of frames per clip to sample.
transform (Callable[[Dict[str, Any]], Any]):
This callable is evaluated on the clip output before the clip is returned.
It can be used for user-defined preprocessing and augmentations to the clips.
The clip input is a dictionary with the following format:
{
'video_id': ,
'video': ,
'audio': ,
'label': ,
'start_time': ,
'stop_time':
}
If transform is None, the raw clip output in the above format is
returned unmodified.
multithreaded_io (bool):
Boolean to control whether parllelizable io operations are performed across
multiple threads.
)r r r r r! frame_filter clip_samplerr" ) dataclass_fieldsr
r r r r _define_clip_structure_generator_transform_generator_frame_filter_generatorsuper__init__) selfr r r r r r r r! r" define_clip_structure_fnr+ __class__r r r2 s8
8
z EpicKitchenRecognition.__init__returnc s* dt ttf dt ttf f fdd }|S )aB
Args:
transform (Callable[[Dict[str, Any]], Dict[str, Any]]): A function that performs
any operation on a clip before it is returned in the default transform function.
Returns:
A function that performs any operation on a clip and returns the transformed clip.
clipr7 c sR fdd d D }| d< D ]} | d u r t g |< qr' S )Nc s, g | ]}|j d kr|j d kr|qS ) stop_time
start_time)r: r9 )r' ar8 r r r) s zWEpicKitchenRecognition._transform_generator..transform_clip..actions)torchtensor)r8 actions_in_clipkeyr! r< r transform_clip s
zCEpicKitchenRecognition._transform_generator..transform_clip)r strr )r! rC r rB r r/ z s &
z+EpicKitchenRecognition._transform_generatorc s" dt t dt t f fdd }|S )z
Args:
frames_per_clip (int): The number of frames per clip to sample.
Returns:
A function that takes in a list of frame indicies and outputs a subsampled list.
frame_indicesr7 c s: t | }t| }ttd|| fddt| D S )Nr c s g | ]
\}}| v r|qS r r )r' ixselected_framesr r r) s zWEpicKitchenRecognition._frame_filter_generator..frame_filer..)lenintsetrange enumerate)rE
num_frames
frame_stepr rH r frame_filer s zCEpicKitchenRecognition._frame_filter_generator..frame_filer)r rK )r rR r rQ r r0 s z.EpicKitchenRecognition._frame_filter_generatorc sX |t jkstdt j d| ddtttf dtttt f dtt f fdd }|S ) a
Args:
seconds_per_clip (float): The length of each sampled clip in seconds.
clip_sampling (ClipSampling):
The type of sampling to perform to perform on the videos of the dataset.
Returns:
A function that takes a dictionary of videos and a dictionary of the actions
for each video and outputs a list of sampled clips.
zOnly z is implemented. z not implemented.videosr= r7 c sp g }| D ]/\}}t }t|j| }t|D ]}| | }| } t||| }
||
q q|S )N)itemsrandomrK durationrM r
append) rS r= clipsvideo_idvideooffset num_clipsrF r: r9 r8 r r r define_clip_structure s zVEpicKitchenRecognition._define_clip_structure_generator..define_clip_structure) r r NotImplementedErrorr rD r r r r
)r r r^ r r] r r. s
z7EpicKitchenRecognition._define_clip_structure_generator)r r r __doc__r r r FramerD floatr rK r r r boolr2 staticmethodr/ r r0 r r r
r.
__classcell__r r r5 r r s^
Z " &r )rU dataclassesr r- enumr typingr r r r r r> (pytorchvideo.data.dataset_manifest_utilsr r
r r r
pytorchvideo.data.epic_kitchenr r pytorchvideo.data.videor r r r r r r s