o ϯip/@sddlZddlmZmZddlmZddlmZmZm Z m Z m Z m Z m Z Gddde ZGddde ZGd d d eZd ed efd dZGdddeZGdddeZGdddeZGdddeZdS)N)ABCabstractmethod)Fraction)AnyDict NamedTupleOptionalTupleUnionListc@sJeZdZUdZeeefed<eeefed<eed<eed<e ed<dS)ClipInfoa Named-tuple for clip information with: clip_start_sec (Union[float, Fraction]): clip start time. clip_end_sec (Union[float, Fraction]): clip end time. clip_index (int): clip index in the video. aug_index (int): augmentation index for the clip. Different augmentation methods might generate multiple views for the same clip. is_last_clip (bool): a bool specifying whether there are more clips to be sampled from the video. clip_start_sec clip_end_sec clip_index aug_index is_last_clipN) __name__ __module__ __qualname____doc__r floatr__annotations__intboolrrS/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/data/clip_sampling.pyr s   r c@sNeZdZUdZeeed<eeed<eeed<eeed<eeed<dS) ClipInfoLista Named-tuple for clip information with: clip_start_sec (float): clip start time. clip_end_sec (float): clip end time. clip_index (int): clip index in the video. aug_index (int): augmentation index for the clip. Different augmentation methods might generate multiple views for the same clip. is_last_clip (bool): a bool specifying whether there are more clips to be sampled from the video. r rrrrN)rrrrr rrrrrrrs    rc @sjeZdZdZdeeefddfddZedeeefdeeefd e e e fde fd d Z dd d ZdS) ClipSamplerz Interface for clip samplers that take a video time, previous sampled clip time, and returns a named-tuple ``ClipInfo``. clip_durationreturnNcCst||_d|_d|_dSNr)r_clip_duration_current_clip_index_current_aug_index)selfrrrr__init__5s  zClipSampler.__init__last_clip_timevideo_duration annotationcCsdSNr)r$r&r'r(rrr__call__:szClipSampler.__call__cCsdS)zBResets any video-specific attributes in preperation for next videoNrr$rrrresetCszClipSampler.reset)rN)rrrrr rrr%rrstrrr r*r,rrrrr/s    r sampling_typercGsN|dkrt|S|dkrt|S|dkrt|S|dkr t|St|d)a Constructs the clip samplers found in ``pytorchvideo.data.clip_sampling`` from the given arguments. Args: sampling_type (str): choose clip sampler to return. It has three options: * uniform: constructs and return ``UniformClipSampler`` * random: construct and return ``RandomClipSampler`` * constant_clips_per_video: construct and return ``ConstantClipsPerVideoSampler`` *args: the args to pass to the chosen clip sampler constructor. uniformrandomconstant_clips_per_video random_multiz not supported)UniformClipSamplerRandomClipSamplerConstantClipsPerVideoSamplerRandomMultiClipSamplerNotImplementedError)r.argsrrrmake_clip_samplerHsr9c seZdZdZ   ddeeefdeeeefdedeffd d Z d eeefd eeefded e eeffddZ d ed ede e efd efddZddZZS)r3zC Evenly splits the video into clips of size clip_duration. NFư>rstride backpad_lastepscsRt||dur |n||_||_||_|jdkr|j|ks'Jd|ddS)a Args: clip_duration (Union[float, Fraction]): The length of the clip to sample (in seconds). stride (floUnion[float, Fraction]at, optional): The amount of seconds to offset the next clip by default value of None is equivalent to no stride => stride == clip_duration. eps (float): Epsilon for floating point comparisons. Used to check the last clip. backpad_last (bool): Whether to include the last frame(s) by "back padding". For instance, if we have a video of 39 frames (30 fps = 1.3s) with a stride of 16 (0.533s) with a clip duration of 32 frames (1.0667s). The clips will be (in frame numbers): with backpad_last = False - [0, 32] with backpad_last = True - [0, 32] - [8, 40], this is "back-padded" from [16, 48] to fit the last window Note that you can use Fraction for clip_duration and stride if you want to avoid float precision issue and need accurate frames in each clip. Nrz(stride must be >0 and <= clip_duration ())superr%_stride_eps _backpad_last)r$rr;r<r= __class__rrr%gs  zUniformClipSampler.__init__r&r'rcCsltt|td|j|jd}t||j}|r2td||}||8}ttd|}t||j}||fS)zK Helper to calculate the start/end clip with backpad logic r)rmaxr!r@)r$r&r'r< clip_startclip_end buffer_amountrrr_clip_start_ends z"UniformClipSampler._clip_start_endr(c Cs~|j|||jd\}}|j|||jd\}}|jr#t|||jk}n||k}|j} |jd7_|r7|t||| d|S)a Args: last_clip_time (float): the last clip end time sampled from this video. This should be 0.0 if the video hasn't had clips sampled yet. video_duration: (float): the duration of the video that's being sampled in seconds annotation (Dict): Not used by this sampler. Returns: clip_info: (ClipInfo): includes the clip information (clip_start_time, clip_end_time, clip_index, aug_index, is_last_clip), where the times are in seconds and is_last_clip is False when there is still more of time in the video to be sampled. )r<r)rIrBabsrAr"r,r ) r$r&r'r(rFrG_ next_clip_endrrrrrr*s  zUniformClipSampler.__call__cCs d|_dSr )r"r+rrrr,s zUniformClipSampler.reset)NFr:)rrrrr rrrrr%r rIrr-rr r*r, __classcell__rrrCrr3bsB )      $r3c@s2eZdZdZdededeeefdefddZ dS) r4zF Randomly samples clip of size clip_duration from the videos. r&r'r(rcCs6t||jd}ttd|}t|||jdddS)a# Args: last_clip_time (float): Not used for RandomClipSampler. video_duration: (float): the duration (in seconds) for the video that's being sampled annotation (Dict): Not used by this sampler. Returns: clip_info (ClipInfo): includes the clip information of (clip_start_time, clip_end_time, clip_index, aug_index, is_last_clip). The times are in seconds. clip_index, aux_index and is_last_clip are always 0, 0 and True, respectively. rT)rEr!rr0r/r )r$r&r'r(max_possible_clip_startr rrrr*s zRandomClipSampler.__call__N) rrrrrrr-rr r*rrrrr4s r4csTeZdZdZdededdffdd Zded ed eee fde ffd d Z Z S) r6z TODO r num_clipsrNcst|||_dSr))r?r% _num_clips)r$rrPrCrrr%s  zRandomMultiClipSampler.__init__r&r'r(c s|jdg|jdg|jdg|jdg|jdgf\}}}}}t|jD]} t|||\|| <|| <|| <|| <|| <q%t|||||Sr))rQranger?r*r) r$r&r'r(clip_start_list clip_end_listclip_index_listaug_index_listis_last_clip_listirCrrr*s6    zRandomMultiClipSampler.__call__) rrrrrrr%rr-rrr*rNrrrCrr6s r6c s`eZdZdZ ddedededdffdd Zd ed ed eee fde fd dZ ddZ Z S)r5z Evenly splits the video into clips_per_video increments and samples clips of size clip_duration at these increments. rJrclips_per_video augs_per_cliprNcst|||_||_dSr))r?r%_clips_per_video_augs_per_clip)r$rrYrZrCrrr%s  z%ConstantClipsPerVideoSampler.__init__r&r'r(c Cstt||jd}t||j}||j}|j}|j}|jd7_|j|jkr2|jd7_d|_d} |j|jksA||j|krFd|_d} | rL|t|||j||| S)ac Args: last_clip_time (float): Not used for ConstantClipsPerVideoSampler. video_duration: (float): the duration (in seconds) for the video that's being sampled. annotation (Dict): Not used by this sampler. Returns: a named-tuple `ClipInfo`: includes the clip information of (clip_start_time, clip_end_time, clip_index, aug_index, is_last_clip). The times are in seconds. is_last_clip is True after clips_per_video clips have been sampled or the end of the video is reached. rrJFT) rrEr!r[r"r#r\r,r ) r$r&r'r(rO uniform_clipr rrrrrrr*!s.    z%ConstantClipsPerVideoSampler.__call__cCsd|_d|_dSr )r"r#r+rrrr,Ps z"ConstantClipsPerVideoSampler.reset)rJ)rrrrrrr%rr-rr r*r,rNrrrCrr5s,  /r5)r0abcrr fractionsrtypingrrrrr r r r rrr-r9r3r4r6r5rrrrs $k,