o ϯi @sdZddlmZddlmZzddlZddlmZmZddl m Z ddl m Z Wne y<ZzdZWYdZ[ndZ[wwddlmZGd d d ejZdS) z timm model adapter Wraps timm (https://github.com/rwightman/pytorch-image-models) models for use as a vision tower in CLIP model. ) OrderedDictN)Mlp to_2tuple)RotAttentionPool2d)AttentionPool2d)freeze_batch_norm_2dcs>eZdZdZ     dfdd Zdd d Zd d ZZS) TimmModelzt timm model adapter # FIXME this adapter is a work in progress, may change in ways that break weight compat avglinearFc sJttdur tdt||_tj||d|_|jj dd}|s&dnd} |dvr;| dks2J|jj ddd n|rBt |d ni} |jj di| |jj } t } |d krdt| ||d | d <|} n|d krst| |d| d <|} n|syJd|dkrt|| d<t| || d<n|dkrt| d|||d| d<t| |_dS)Nz-Please `pip install timm` to use timm models.) pretrained pool_sizer)abs_attnrot_attnr) global_poolr) feat_size out_featurespoolr)rz9projection layer needed if non-attention pooling is used.r dropprojmlp)r)r)super__init__timm RuntimeErrorr image_size create_modeltrunk default_cfggetreset_classifierdict num_featuresrAbsAttentionPool2drnnDropoutLinearr Sequentialhead) self model_name embed_dimrrrrrr feature_ndim reset_kwargsprev_chs head_layers __class__U/home/ubuntu/.local/lib/python3.10/site-packages/laion_clap/clap_module/timm_model.pyrs6     zTimmModel.__init__rc s|s|jD]}d|_q|rt|jd Sd Sz ddlm}m}Wn ty-tdw|j }||j|}t | |t dD]}||} | D] }d|j |_qQqI|ry||j|dd} fdd | D} t|j| d Sd S) zw lock modules Args: unlocked_groups (int): leave last n layer groups unlocked (default: 0) Fr)group_parameters group_modulesz^Please install latest timm `pip install git+https://github.com/rwightman/pytorch-image-models`rT)reversecsh|] \}}|kr|qSr6r6).0kv max_layer_idr6r7 dsz!TimmModel.lock..N)r! parameters requires_gradrtimm.models.helpersr8r9 ImportErrorr group_matchermaxkeysrange get_parameteritems) r-unlocked_groupsfreeze_bn_statsparamr8r9matchergparams group_idxgroupgmodulesr6r>r7lockGs6    zTimmModel.lockcCs||}||}|S)N)r!r,)r-xr6r6r7forwardgs  zTimmModel.forward)r r r r F)rF)__name__ __module__ __qualname____doc__rrSrU __classcell__r6r6r4r7r s . r )rY collectionsrtorch.nnr(rtimm.models.layersrr#timm.models.layers.attention_pool2drrr'rDeutilsrModuler r6r6r6r7s