o ϯiY@s2ddlZddlZddlmZddlZddlZzddlZWn ey'dZ YnwdZ dJdej de de d ej fd d Z ej jdej d ee e fd ed ej fddZ  dKdej d e d eded ej f ddZ dJdej dee de d eej fddZ dLdej de ded ej fddZ  dKdej d ej d e d eded eej ejff d!d"Z  dKdej d ej d#e d$e d eded eej ej ffd%d&Zdej d e d ej d eej ej ffd'd(Zdej d e d)e fd*d+Zdej d e d)e d ej fd,d-Zdej d e d)e d ej d eej ejff d.d/Zd0edej d ej d eej ej ffd1d2Zd ej d3e d4e d ej fd5d6Zd ej d7e d8e d ej fd9d:Z  ;dMded?e d ee e e e ffd@dAZ    ;dNdej dBe dCe ded ed?e d ej fdFdGZ!dej d ej fdHdIZ"dS)ON)TupleFTx num_samples temporal_dimreturncCsT|j|}|dkr |dksJtd|d|}t|d|d}t|||S)av Uniformly subsamples num_samples indices from the temporal dimension of the video. When num_samples is larger than the size of temporal dimension of the video, it will sample frames based on nearest neighbor interpolation. Args: x (torch.Tensor): A video tensor with dimension larger than one with torch tensor type includes int, long, float, complex, etc. num_samples (int): The number of equispaced samples to be selected temporal_dim (int): dimension of temporal to perform temporal subsample. Returns: An x-like Tensor with subsampled temporal dimension. r)shapetorchlinspaceclamplong index_select)rrrtindicesrV/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/transforms/functional.pyuniform_temporal_subsamples rsize interpolationcststdtjtjtjtjdvsJ|\dd|ddddjddd D}fd d|D}t j d d|Ddd }t t |}|dddd}|S) an Down/up samples the input torch tensor x to the given size with given interpolation mode. Args: input (Tensor): the input tensor to be down/up sampled. size (Tuple[int, int]): expected output spatial size. interpolation: model to perform interpolation, options include `nearest`, `linear`, `bilinear`, `bicubic`. z]opencv is required to use opencv transforms. Please install with 'pip install opencv-python'.)nearestlinearbilinearbicubiccSsg|] }|dqS)r)squeezenumpy).0 img_tensorrrr Gs z'_interpolate_opencv..rr)dimcs$g|]}tj|fdqS))r)cv2resizer img_array!_opencv_pytorch_interpolation_maprnew_hnew_wrrrKscSsg|] }tj|ddqS)raxis)np expand_dimsr$rrrrTsr*)_HAS_CV2 ImportErrorr" INTER_NEAREST INTER_LINEAR INTER_AREA INTER_CUBICpermutesplitr, concatenater from_numpyascontiguousarray)rrrimg_array_listresized_img_array_listr%rrr&r_interpolate_opencv,s0   r;rpytorchbackendc Cst|jdks J|jtjksJ|dvsJ|j\}}}}||kr2ttt|||}|} n|}ttt|||} |dkrRtj j j ||| f|ddS|dkr_t ||| f|dSt |d) a Determines the shorter spatial dim of the video (i.e. width or height) and scales it to the given size. To maintain aspect ratio, the longer side is then scaled accordingly. Args: x (torch.Tensor): A video tensor of shape (C, T, H, W) and type torch.float32. size (int): The size the shorter side is scaled to. interpolation (str): Algorithm used for upsampling, options: nearest' | 'linear' | 'bilinear' | 'bicubic' | 'trilinear' | 'area' backend (str): backend used to perform interpolation. Options includes `pytorch` as default, and `opencv`. Note that opencv and pytorch behave differently on linear interpolation on some versions. https://discuss.pytorch.org/t/pytorch-linear-interpolation-is-different-from-pil-opencv/71181 Returns: An x-like Tensor with scaled spatial dims. )r<opencvr<F)rmode align_cornersr?)rrz backend not supported.)lenr dtyper float32intmathfloorfloatnn functional interpolater;NotImplementedError) rrrr=crhwr(r)rrrshort_side_scale\s   rPframes frame_ratioscCs6|j|}g}|D]}t||||}||q |S)a^ Prepare output as a list of tensors subsampled from the input frames. Each tensor maintain a unique copy of subsampled frames, which corresponds to a unique pathway. Args: frames (tensor): frames of images sampled from the video. Expected to have torch tensor (including int, long, float, complex, etc) with dimension larger than one. frame_ratios (tuple): ratio to perform temporal down-sampling for each pathways. temporal_dim (int): dimension of temporal. Returns: frame_list (tuple): list of tensors as output. )r rappend)rQrRrtemporal_length frame_listratiopathwayrrr#uniform_temporal_subsample_repeateds   rXtargets num_class label_smoothcCst||ks Jdd|krdksJdJd||}d||}tj|jd|f||dkr9tjnd|jd}|d|d d||S) ac This function converts target class indices to one-hot vectors, given the number of classes. Args: targets (torch.Tensor): Index labels to be converted. num_class (int): Total number of classes. label_smooth (float): Label smooth value for non-target classes. Label smooth is disabled by default (0). z/Class Index must be less than number of classesr?z/Label smooth value needs to be between 0 and 1.rYN)rCdevicer) r maxitemfullr r r^scatter_view)rZr[r\non_target_value target_valueone_hot_targetsrrrconvert_to_one_hots$  rhimagesboxesc Csb|j\}}}}t||||}|j\} } } } ||kr%|t| |9}||fS|t| |9}||fS)a> Perform a spatial short scale jittering on the given images and corresponding boxes. Args: images (tensor): images to perform scale jitter. Dimension is `channel` x `num frames` x `height` x `width`. boxes (tensor): Corresponding boxes to images. Dimension is `num boxes` x 4. size (int): The size the shorter side is scaled to. interpolation (str): Algorithm used for upsampling, options: nearest' | 'linear' | 'bilinear' | 'bicubic' | 'trilinear' | 'area' backend (str): backend used to perform interpolation. Options includes `pytorch` as default, and `opencv`. Note that opencv and pytorch behave differently on linear interpolation on some versions. https://discuss.pytorch.org/t/pytorch-linear-interpolation-is-different-from-pil-opencv/71181 Returns: (tensor): the scaled images with dimension of `channel` x `num frames` x `height` x `width`. (tensor): the scaled boxes with dimension of `num boxes` x 4. )r rPrH) rirjrrr=rMrrNrO_r(r)rrrshort_side_scale_with_boxessrlmin_sizemax_sizecCs&t||dd}t|||||S)a~ Perform a spatial short scale jittering on the given images and corresponding boxes. Args: images (tensor): images to perform scale jitter. Dimension is `channel` x `num frames` x `height` x `width`. boxes (tensor): Corresponding boxes to images. Dimension is `num boxes` x 4. min_size (int): the minimal size to scale the frames. max_size (int): the maximal size to scale the frames. interpolation (str): Algorithm used for upsampling, options: nearest' | 'linear' | 'bilinear' | 'bicubic' | 'trilinear' | 'area' backend (str): backend used to perform interpolation. Options includes `pytorch` as default, and `opencv`. Note that opencv and pytorch behave differently on linear interpolation on some versions. https://discuss.pytorch.org/t/pytorch-linear-interpolation-is-different-from-pil-opencv/71181 Returns: (tensor): the scaled images with dimension of `channel` x `num frames` x `height` x `width`. (tensor): the scaled boxes with dimension of `num boxes` x 4. rr)r randintrarl)rirjrmrnrr=rrrr"random_short_side_scale_with_boxessrqc Cs|jd|kr|jd|kr|S|jd}|jd}d}||kr+ttjd||}d}||kr}tjjj|dd||d||||||||||||f||f|d|dd||dddddf<q|S)a Crop the given images to random size and aspect ratio. A crop of random size relative to the original size and a random aspect ratio is made. This crop is finally resized to given size. This is popularly used to train the Inception networks. Args: frames (torch.Tensor): Video tensor to be resized with shape (C, T, H, W). target_height (int): Desired height after cropping. target_width (int): Desired width after cropping. scale (Tuple[float, float]): Scale range of Inception-style area based random resizing. Should be between 0.0 and 1.0. aspect_ratio (Tuple[float, float]): Aspect ratio range of Inception-style area based random resizing. Should be between 0.0 and +infinity. shift (bool): Bool that determines whether or not to sample two different boxes (for cropping) for the first and last frame. If True, it then linearly interpolates the two boxes for other frames. If False, the same box is cropped for every frame. Default is False. log_uniform_ratio (bool): Whether to use a log-uniform distribution to sample the aspect ratio. Default is True. interpolation (str): Algorithm used for upsampling. Currently supports 'nearest', 'bilinear', 'bicubic', 'area'. Default is 'bilinear'. num_tries (int): The number of times to attempt a randomly resized crop. Falls back to a central crop after all attempts are exhausted. Default is 10. Returns: cropped (tensor): A cropped video tensor of shape (C, T, target_height, target_width). rrz1min and max of scale range must be greater than 0z8min and max of aspect_ratio range must be greater than 0rr N)rr@cSg|]}t|qSrrErrrrrrIz'random_resized_crop..)stepscSrrrrrrrrJrcSrrrrrrrrKrcSrrrrrrrrLr) r rr rIrJrKr tolistzerosr)rQrrrrrrrrchannelsrrvrwrrrNrOrzi_j_h_w_i_sj_sh_sw_sindrrrrandom_resized_cropsR)      (    ( rcCs |d}|S)z Divide the given tensor x by 255. Args: x (torch.Tensor): The input tensor. Returns: y (torch.Tensor): Scaled tensor by dividing 255. go@r)ryrrrdiv_255\s r)r)rr<)rY)Tr)FTrr)#rrFtypingrrr,r r"r/r.TensorrErjitignorestrr;rPrXrHrhndarrayrlrqr|rrrrrurtboolrrrrrrrsh      2 +   $ + " #      L    V