o ߥiD@sddlZddlZddlZddlZddlZddlmZddlZddlZ ddl Z ddl Z ddl mZddlmZdddZdd Zd d Zd d ZddZdddZdddZdS)N) OrderedDict)Image) transformsc Csdttj|tjdt|tddgttj|ddttj ddddtddgd}|S) z The implementation of transforms functions. The default image resolution is 224. The normalize parameter follows the mainstream setting. ) interpolation)g3<4'?gwg M?gy{ ?)gB91?gwt.?g U?)g?g?)scaler) brightness saturationhue) clip_test clip_train) rComposeResizerBICUBIC CenterCrop NormalizeRandomResizedCropRandomHorizontalFlip ColorJitter) input_res tsfm_dictrb/home/ubuntu/.local/lib/python3.10/site-packages/modelscope/models/cv/vop_retrieval/basic_utils.pyinit_transform_dicts" rcCsFt|}|dj|d}|dj|d}|d}|d}||||gS)a7 Loading dataset from 'feature_path' as a retrieval docs. The default dataset is MSRVTT-9K. Args: feature_path: 'VoP_msrvtt9k_features.pkl' mydevice: device(type='cuda', index=0) Returns: [text_embeds, vid_embeds_pooled, vid_ids, texts] text_embeds)device vid_embedsvid_idstexts)torchloadto) feature_pathmydevicefeature_contentrvid_embeds_pooledrrrrr load_data,s  r&cCs8t|d }t|WdS1swYdS)z Load json files. rN)openjsonr )filenamefrrr load_json@s $r,cCsP|dkr&t|tj|tj|t|dtjj_ dtjj_ dSdS)z Set random seed. rTFN) r manual_seednprandomseedcudamanual_seed_allbackendscudnn deterministic benchmark)r0rrrset_seedHs     r7cCsBt|}|d}t}|D] \}}|||dd<q|}|S)z0 Load pre-train parameters for VoP. state_dictzmodule.)rr ritemsreplace)checkpoint_path checkpointr8new_state_dictkvrrrget_state_dictUs rArandcCs@t||}tjd||ddt}g}t|ddD]\}}||||ddfq|dkr:dd|D} nd d|D} g} | D]A} |tj | | \} } | sid }t |D] }| \} } | rhnq\| snd St | tj } t| } | d dd} | | qEt| |kr| | dt| |ks| | fS) a Get indexes of sampled frames. Args: cap: cv2.VideoCapture num_frames: int - number of frames to sample vlen: video length, int(cap.get(cv2.CAP_PROP_FRAME_COUNT)), 325 sample: 'rand' | 'uniform' how to sample Returns: frames: torch.tensor of stacked sampled video frames of dim (num_frames, C, H, W) frame_idxs: list(int) indices of where the frames where sampled r)startstopnumNrBcSs$g|]}tt|d|dqS)rrC)r/choicerange.0xrrr zs$z$get_valid_frames..cSs g|] }|d|ddqS)rrCrrJrrrrM|s )NNrN)minr.linspaceastypeint enumerateappendsetcv2CAP_PROP_POS_FRAMESreadrIcvtColor COLOR_BGR2RGBr from_numpypermutelenclone)cap num_framesvlensample acc_samples intervalsrangesidxinterv frame_idxsframesindexretframen_tries_rrrget_valid_framesdsB         rpcCs^t|}|s J|t|tj}t||||\}}t| d}| ||fS)a Get indexes of sampled frames. Args: video_path: the local video path num_frames: Frame number, 12 frames for each video sample: 'rand' | 'uniform' how to sample Returns: frames: torch.tensor of stacked sampled video frames of dim (num_frames, C, H, W) frame_idxs: list(int) indices of where the frames where sampled ) rW VideoCaptureisOpenedrSgetCAP_PROP_FRAME_COUNTrprstackfloatrelease) video_pathrarcr`rbrjrirrrload_frames_from_videos rz)r)rB)ospickler/shutilzipfile collectionsrrWnumpyr.rujsonr)PILr torchvisionrrr&r,r7rArprzrrrrs&     2