o ϯi @sddlmZddlmZmZddlmZddejdeddfdd Z ddejd eddfd d Z  ddejdede ddfddZ dS)N) c2_msra_fillc2_xavier_fill)#SpatioTemporalClsPositionalEncoding{Gz?model fc_init_stdreturncCs|D]g}t|tjtjfr t|n/t|tjjjrD|jdur9t |dr2|j r2|jj dn|jj d|j durD|j j t|tjrkt |drW|jrWt|n |jj jd|d|j durk|j j q|S)a Performs ResNet style weight initialization. That is, recursively initialize the given model in the following way for each type: Conv - Follow the initialization of kaiming_normal: https://pytorch.org/docs/stable/_modules/torch/nn/init.html#kaiming_normal_ BatchNorm - Set weight and bias of last BatchNorm at every residual bottleneck to 0. Linear - Set weight to 0 mean Gaussian with std deviation fc_init_std and bias to 0. Args: model (nn.Module): Model to be initialized. fc_init_std (float): the expected standard deviation for fully-connected layer. Nblock_final_bng? xavier_init)meanstd)modules isinstancennConv2dConv3dr batchnorm _NormBaseweighthasattrr datafill_biaszero_Linearr rnormal_)rrmrS/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/weight_init.py_init_resnet_weightss&         r {Gz?trunc_normal_stdcCs|D]P}t|tjr)tjj|j|dt|tjr(|jdur(tj|jdqt|tj r@tj|jdtj|jdqt|t rT| D] }tjj||dqIqdS)z Weight initialization for vision transformers. Args: model (nn.Module): Model to be initialized. trunc_normal_std (float): the expected standard deviation for fully-connected layer and ClsPositionalEncoding. )r Nrr ) rrrrinit trunc_normal_rr constant_ LayerNormr parameters)rr"rweightsrrr_init_vit_weights2s    r)resnetinit_stdstylecCs4|dvsJ|dkrt||S|dkrt||St)aS Performs weight initialization. Options include ResNet style weight initialization and transformer style weight initialization. Args: model (nn.Module): Model to be initialized. init_std (float): The expected standard deviation for initialization. style (str): Options include "resnet" and "vit". )r*vitr*r-)r r)NotImplementedError)rr+r,rrrinit_net_weightsHs   r/)r)r!)rr*) torch.nnrfvcore.nn.weight_initrrpytorchvideo.layersrModulefloatr r)strr/rrrrs  *