o ϯi9 @sddlmZddlmZddlmZddlmZmZd ed ed ed Z d d dd gd d gdd ggdd gd d gdd gggdgdgdggdgddZ d ddd gd d gdd ggdd gd d gdd gggdgdgdggdgddZ d dd ddgddgd d gddd gd d gdd ggdd gd d gdd gggdgdgdggdgdd Z  d'dededed ejfd!d"Z  d'dededed ejfd#d$Z  d'dededed ejfd%d&ZdS)()AnyN)%create_multiscale_vision_transformers)MODEL_ZOO_ROOT_DIRhub_model_builderz{}/kinetics/MVIT_B_16x4.pythz'{}/kinetics/MVIT_B_32x3_f294077834.pythz%{}/imagenet/MVIT_B_16_f292487636.pyth)mvit_base_16x4mvit_base_32x3 mvit_base_16g@)rrr)r rrr)r rrr)rr)r r r ) spatial_size temporal_size embed_dim_mulatten_head_mulpool_q_stride_sizepool_kv_stride_adaptivepool_kvq_kernel T)rrr)rr r ) rrdepthconv_patch_embed_kernelconv_patch_embed_strideconv_patch_embed_padding use_2d_patchrrrrrF pretrainedprogresskwargsreturncKtdt||tdtd|S)a Multiscale Vision Transformers model architecture [1] trained with an 16x4 setting on the Kinetics400 dataset. Model with pretrained weights has top1 accuracy of 78.9%. [1] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, Christoph Feichtenhofer, "Multiscale Vision Transformers" https://arxiv.org/pdf/2104.11227.pdf Args: pretrained (bool): If True, returns a model pre-trained on Kinetics400 dataset. progress (bool): If True, displays a progress bar of the download to stderr. kwargs: Use these to modify any of the other model settings. All the options are defined in create_multiscale_vision_transformers. NOTE: to use the pretrained model, do not modify the model configuration via the kwargs. Only modify settings via kwargs to initialize a new model without pretrained weights. rmodel_builder_funcrr checkpoint_pathdefault_configN)rrcheckpoint_pathsmvit_video_base_configrr r!r(r(_/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/hub/vision_transformers.pyr9rcKr#)a Multiscale Vision Transformers model architecture [1] trained with an 32x3 setting on the Kinetics400 dataset. Model with pretrained weights has top1 accuracy of 80.3%. [1] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, Christoph Feichtenhofer, "Multiscale Vision Transformers" https://arxiv.org/pdf/2104.11227.pdf Args: pretrained (bool): If True, returns a model pre-trained on Kinetics400 dataset. progress (bool): If True, displays a progress bar of the download to stderr. kwargs: Use these to modify any of the other model settings. All the options are defined in create_multiscale_vision_transformers. NOTE: to use the pretrained model, do not modify the model configuration via the kwargs. Only modify settings via kwargs to initialize a new model without pretrained weights. rr$Nr()rrr)mvit_video_base_32x3_configr+r(r(r,r\r-rcKr#)az Multiscale Vision Transformers model architecture [1] with a depth 16 trained on ImageNet-1k dataset. Model with pretrained weights has top1 accuracy of 83%. [1] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, Christoph Feichtenhofer, "Multiscale Vision Transformers" https://arxiv.org/pdf/2104.11227.pdf Args: pretrained (bool): If True, returns a model pre-trained on Kinetics400 dataset. progress (bool): If True, displays a progress bar of the download to stderr. kwargs: Use these to modify any of the other model settings. All the options are defined in create_multiscale_vision_transformers. NOTE: to use the pretrained model, do not modify the model configuration via the kwargs. Only modify settings via kwargs to initialize a new model without pretrained weights. r r$Nr()rrr)mvit_image_base_16_configr+r(r(r,r sr )FT)typingrtorch.nnnn'pytorchvideo.models.vision_transformersrutilsrrformatr)r*r.r/boolModulerrr r(r(r(r,s       $ $