o ϯi@sTddlZddlmZmZddlZddlmZddlmmZ Gdddej Z dS)N)CallableOptionalcseZdZdZdddddejfdejdeejdeejd ed ed e d e d dffdd Z ddZ d e fddZ d e fddZeddZeddZddZdejdejd ejfddZZS) BYOLz Bootstrap Your Own Latent A New Approach to Self-Supervised Learning Details can be found in: https://arxiv.org/pdf/2006.07733.pdf NiigGz?backbone projector predictor feature_dimpredictor_innermmtnormreturnc sttjd||_||_|durt||}||_ t ||_ |j D]}d|_q)|durPttj||dd||tjddtj||dd|_dS||_dS)a3 Args: backbone (nn.Module): backbone for byol, input shape depends on the forward input size. Standard inputs include `B x C`, `B x C x H x W`, and `B x C x T x H x W`. projector (nn.Module): stand projector is a mlp with 2 to 3 hidden layers, with (synchronized) BatchNorm and ReLU activation. predictor (nn.Module): predictor MLP of BYOL of similar structure as the projector MLP. feature_dim (int): output feature dimension. predictor_inner (int): inner channel size for predictor. mmt (float): momentum update ratio for the momentum backbone. norm (callable): normalization to be used in projector, default is synchronized batchnorm. z PYTORCHVIDEO.model.BYOL.__init__NF)biasT)inplace)super__init__torch_C_log_api_usage_oncer rnn Sequentialrcopydeepcopy backbone_mmt parameters requires_gradLinearReLUr) selfrrrrr r r p __class__L/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/byol.pyrs*     z BYOL.__init__cCstd||g}| }|S)z Similarity loss for byol. Args: q and k (nn.tensor): inputs to calculate the similarity, expected to have the same shape of `N x C`. znc,nc->n)reinsummean)rqk similaritylossr!r!r"sim_lossDs z BYOL.sim_losscCs ||_dS)z Update the momentum. This function can be used to perform momentum annealing. Args: mmt (float): update the momentum. Nr )rr r!r!r" update_mmtOs zBYOL.update_mmtcCs|jS)z\ Get the momentum. This function can be used to perform momentum annealing. r*)rr!r!r"get_mmtWsz BYOL.get_mmtcCsBt|j|jD]\}}|j|j|jd|j|_q dS)z2 Momentum update on the backbone. g?N)ziprrrdatar )rparam param_mmtr!r!r"_momentum_update_backbone]s  zBYOL._momentum_update_backbonecCs@t ||}Wdn1swYtj|ddS)zi Forward momentum backbone. Args: x (tensor): input to be forwarded. Ndim)rno_gradrF normalize)rxprojr!r!r"forward_backbone_mmtgs  zBYOL.forward_backbone_mmtcCs"||}||}tj|ddS)z` Forward backbone. Args: x (tensor): input to be forwarded. r2r3)rrr6r7)rr8r9predr!r!r"forward_backboners  zBYOL.forward_backbonex1x2cCsx||}||}t|||}||}Wdn1s'wY||||||d}|S)aP Args: x1 (torch.tensor): a batch of image with augmentation. The input tensor shape should able to be feed into the backbone. x2 (torch.tensor): the size batch of image with different augmentation. The input tensor shape should able to be feed into the backbone. N)r<rr5r1r:r))rr=r>pred_1pred_2 proj_mmt_1 proj_mmt_2r(r!r!r"forward|s     z BYOL.forward)__name__ __module__ __qualname____doc__r SyncBatchNormModulerintfloatrrr)r+r,rr5r1r:r<TensorrD __classcell__r!r!rr"r sD  2    $ r) rtypingrrrtorch.nnrtorch.nn.functional functionalr6rJrr!r!r!r"s