o ϯi:@slddlmZddlZddlmZddlmZddlmm Z ddl m Z ddl mZGdddejZdS))OptionalN)differentiable_all_gather)set_attributesc s\eZdZdZ  ddejdeejdeddffdd Zd e j d e j de j fd d Z Z S)SimCLRz A Simple Framework for Contrastive Learning of Visual Representations Details can be found from: https://arxiv.org/abs/2002.05709 NQ?mlpbackbone temperaturereturncs&ttjdt|tdS)Nz"PYTORCHVIDEO.model.SimCLR.__init__)super__init__torch_C_log_api_usage_oncerlocals)selfrrr  __class__N/home/ubuntu/.local/lib/python3.10/site-packages/pytorchvideo/models/simclr.pyr s  zSimCLR.__init__x1x2cCs|jdur ||}||}tj|ddd}|jdur!||}||}tj|ddd}tjt|dd}td||g}||j }| d}t rWt rWt }nd}ttt|||d||j}tjj||}|S)aP Args: x1 (torch.tensor): a batch of image with augmentation. The input tensor shape should able to be feed into the backbone. x2 (torch.tensor): the size batch of image with different augmentation. The input tensor shape should able to be feed into the backbone. N)pdimr)rz nc,kc->nk)rrF normalizer catreinsumdivr sizedist is_availableis_initializedget_ranktensorlistrangelongtodevicenn functional cross_entropy)rrrprod batch_size device_indgtlossrrrforward s.         zSimCLR.forward)Nr) __name__ __module__ __qualname____doc__r,Modulerfloatr r Tensorr4 __classcell__rrrrr s $ r)typingrr torch.distributed distributedr"torch.nnr,torch.nn.functionalr-rfvcore.nn.distributedrpytorchvideo.layers.utilsrr9rrrrrs