o .wiB@sddlZddlZddlmZmZmZmZddlZddlmZm Z ddl m Z ddl m Z dddd Ze s8d d gZdFd edede jjjfdd ZGdddej jZGdddej jZGdddej jZdGdededefddZdHdedeedfdefdd ZdId"ed#edefd$d%ZdJd'ed(edefd)d*Z Gd+d,d,e jZ!Gd-d.d.e jZ"Gd/d0d0e jZ#Gd1d2d2e#Z$d3ed4edefd5d6Z%d7ed8ed e jd4edeeeeefff d9d:Z&dKde d?defd@dAZ' B ; dLd7ed8edCe dDd>e d?d4edef dEd Z(dS)MN)List NamedTupleOptionalUnion)Tensornn)Literal)_TORCHVISION_AVAILABLESqueezeNet1_1_WeightsAlexNet_Weights VGG16_Weights) squeezenet1_1alexnetvgg16)learned_perceptual_image_patch_similarity_get_tv_model_featuresFnet pretrainedreturncCsZtstdddl}|r!t|jt|}t|j||jd}|jSt|j|dd}|jS)aAGet torchvision network. Args: net: Name of network pretrained: If pretrained weights should be used >>> _ = _get_tv_model_features("alexnet", pretrained=True) >>> _ = _get_tv_model_features("squeezenet1_1", pretrained=True) >>> _ = _get_tv_model_features("vgg16", pretrained=True) zSTorchvision is not installed. Please install torchvision to use this functionality.rN)weights)r ModuleNotFoundError torchvisiongetattrmodels _weight_mapDEFAULTfeatures)rrr model_weightsmodelr`/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/torchmetrics/functional/image/lpips.pyr-s cBeZdZdZd dededdffdd Zd edefd d ZZ S) SqueezeNetzSqueezeNet implementation.FT requires_gradrrNc sttd|}d|_g}tdtddtddtddtddtddtdd g}|D]}tj}|D] }|t |||q7| |q.t ||_ |s\| D]} d | _qVdSdS) Nr  F)super__init__rN_slicesrangetorchr Sequential add_modulestrappend ModuleListslices parametersr#) selfr#rpretrained_featuresr6feature_ranges feature_rangeseqiparam __class__rr r-Hs  :    zSqueezeNet.__init__xcCs:Gdddt}g}|jD] }||}||q ||S)Process input.c@sFeZdZUeed<eed<eed<eed<eed<eed<eed<dS) z*SqueezeNet.forward.._SqueezeOutputrelu1relu2relu3relu4relu5relu6relu7N__name__ __module__ __qualname__r__annotations__rrrr _SqueezeOutput]s  rO)rr6r4)r8rArOrelusslice_rrr forwardZs   zSqueezeNet.forwardFT rKrLrM__doc__boolr-rrrR __classcell__rrr?r r"Esr"cr!)AlexnetzAlexnet implementation.FTr#rrNcs2ttd|}tj|_tj|_tj|_tj|_ tj|_ d|_ t dD] }|j t|||q/t ddD] }|j t|||qBt ddD] }|j t|||qUt ddD] }|j t|||qht ddD] }|j t|||q{|s|D]}d|_qdSdS)Nrr&r%r'r(r*Fr,r-rr0rr1slice1slice2slice3slice4slice5r.r/r2r3r7r#)r8r#ralexnet_pretrained_featuresrAr>r?rr r-p.         zAlexnet.__init__rAc Cf||}|}||}|}||}|}||}|}||}|}Gdddt}||||||S)rBc@6eZdZUeed<eed<eed<eed<eed<dS)z(Alexnet.forward.._AlexnetOutputsrCrDrErFrGNrJrrrr _AlexnetOutputs  rcrZr[r\r]r^r) r8rAhh_relu1h_relu2h_relu3h_relu4h_relu5rcrrr rR     zAlexnet.forwardrSrTrrr?r rXmrXcr!)Vgg16zVgg16 implementation.FTr#rrNcs2ttd|}tj|_tj|_tj|_tj|_ tj|_ d|_ t dD] }|j t|||q/t ddD] }|j t|||qBt ddD] }|j t|||qUt ddD] }|j t|||qht ddD] }|j t|||q{|s|D]}d|_qdSdS) Nrr& FrY)r8r#rvgg_pretrained_featuresrAr>r?rr r-r`zVgg16.__init__rAc Cra)rBc@rb)z"Vgg16.forward.._VGGOutputsrelu1_2relu2_2relu3_3relu4_3relu5_3NrJrrrr _VGGOutputsrdrzre) r8rArf h_relu1_2 h_relu2_2 h_relu3_3 h_relu4_3 h_relu5_3rzrrr rRrlz Vgg16.forwardrSrTrrr?r rnrmrnTin_tenskeep_dimcCs|jddg|dS)z2Spatial averaging over height and width of images.r%)keepdimmean)rrrrr _spatial_averagesr@rout_hw.cCstj|ddd|S)z+Upsample input with bilinear interpolation.bilinearF)sizemode align_corners)rUpsample)rrrrr _upsamplesr:0yE>in_featepscCs&t|tj|dddd}||S)zNormalize input tensor.r%T)dimr)r0sqrtsum)rr norm_factorrrr _normalize_tensorsrrrArcCsN|jd|kr|jd|krtjjj|||fddStjjj|||fdddS)zlhttps://github.com/toshas/torch-fidelity/blob/master/torch_fidelity/sample_similarity_lpips.py#L127C22-L132.area)rrF)rr)shaper0r functional interpolate)rArrrr _resize_tensorsrcsFeZdZUdZeed<eed<d fdd Zdedefd d ZZS) ScalingLayerzScaling layer.shiftscalerNcsbt|jdtgddddddfdd|jdtgddddddfdddS)Nr)gQgI +gMbȿF) persistentr)gZd;O?gy&1?g?)r,r-register_bufferr0r)r8r?rr r-s *.zScalingLayer.__init__inpcCs||j|jSrB)rr)r8rrrr rRszScalingLayer.forward)rN) rKrLrMrUrrNr-rRrWrrr?r rs rc sFeZdZdZddedededdffd d Zd edefd d ZZ S) NetLinLayerz,A single linear layer which does a 1x1 conv.rFchn_inchn_out use_dropoutrNc sHt|r tgng}|tj||dddddg7}tj||_dS)NrrF)stridepaddingbias)r,r-rDropoutConv2dr1r)r8rrrlayersr?rr r-s zNetLinLayer.__init__rAcCs ||Sr)r)r8rArrr rRs zNetLinLayer.forward)rF) rKrLrMrUintrVr-rrRrWrrr?r rs  rcseZdZ         ddededded ed ed ed eed edeeddffdd Z dde de dedede e e e e e fff ddZ ZS)_LPIPSTalexFNrrrvggsqueezespatial pnet_rand pnet_tuner model_path eval_moderesizerc st||_||_||_||_| |_t|_|jdvr%t } gd|_ n|jdkr2t } gd|_ n |jdkr>t } gd|_ t |j |_| |j |jd|_t|j d|d |_t|j d |d |_t|j d |d |_t|j d |d |_t|j d |d |_|j|j|j|j|jg|_|jdkrt|j d|d |_t|j d|d |_|j|j|jg7_t|j|_|r|durtjtjt |jdd|d}|j!t"j#|dddd|r|$|js|%D]} d| _&qdSdS)aInitializes a perceptual loss torch.nn.Module. Args: pretrained: This flag controls the linear layers should be pretrained version or random net: Indicate backbone to use, choose between ['alex','vgg','squeeze'] spatial: If input should be spatial averaged pnet_rand: If backbone should be random or use imagenet pre-trained weights pnet_tune: If backprop should be enabled for both backbone and linear layers use_dropout: If dropout layers should be added model_path: Model path to load pretained models from eval_mode: If network should be in evaluation mode resize: If input should be resized to this size )rr)rrr)rrrr)rrrrrrr)rr#r)rrr%rror&Nz..z lpips_models/z.pthcpu) map_locationF)strict)'r,r- pnet_typerrrrr scaling_layerrnchnsrXr"lenLrrlin0lin1lin2lin3lin4linslin5lin6rr5ospathabspathjoininspectgetfileload_state_dictr0loadevalr7r#) r8rrrrrrrrrnet_typer>r?rr r- sT          z_LPIPS.__init__in0in1 retperlayer normalizec CsN|rd|d}d|d}||||}}|jdur,t||jd}t||jd}|j||j|}}iii} } } t|jD]} t|| t|| | | <| | <| | | | d| | <qFg} t|jD]-} |jr| t |j | | | t |j dddqm| t|j | | | ddqmt| }|r|| fS|S)Nr%r)r)rT)r)rrrrrRr/rrrr4rrtuplerrr)r8rrrr in0_input in1_inputouts0outs1feats0feats1diffskkresvalrrr rRUs*   "0"z_LPIPS.forward) TrFFFTNTN)FF)rKrLrMrVrrr3rr-rrrrrRrWrrr?r r sV   Lrcs*eZdZdZdeddffdd ZZS) _NoTrainLpipsz8Wrapper to make sure LPIPS never leaves evaluation mode.rrcs tdS)z.Force network to always be in evaluation mode.F)r,train)r8rr?rr r{s z_NoTrainLpips.train)rKrLrMrUrVrrWrrr?r rxsrimgrcCsD|r|dko |dkn|dk}|jdko!|jddko!|S)z1Check that input is a valid image to the network.g?grrorr)maxminndimr)rr value_checkrrr _valid_imgs rimg1img2c Cst||r t||s4td|jd|jd||gd||gd|r,ddgnddgd ||||d }||jdfS) NzeExpected both input arguments to be normalized tensors with shape [N, 3, H, W]. Got input with shape z and z and values in range z+ when all values are expected to be in the rrrz range.)r)r ValueErrorrrrr)rrrrlossrrr _lpips_updates rr sum_scorestotal reduction)rrcCs|dkr||S|S)Nrr)rrrrrr _lpips_computesrrrrcCs:t|dj|j|jd}t||||\}}t|||S)aoThe Learned Perceptual Image Patch Similarity (`LPIPS_`) calculates perceptual similarity between two images. LPIPS essentially computes the similarity between the activations of two image patches for some pre-defined network. This measure has been shown to match human perception well. A low LPIPS score means that image patches are perceptual similar. Both input image patches are expected to have shape ``(N, 3, H, W)``. The minimum size of `H, W` depends on the chosen backbone (see `net_type` arg). Args: img1: first set of images img2: second set of images net_type: str indicating backbone network type to use. Choose between `'alex'`, `'vgg'` or `'squeeze'` reduction: str indicating how to reduce over the batch dimension. Choose between `'sum'` or `'mean'`. normalize: by default this is ``False`` meaning that the input is expected to be in the [-1,1] range. If set to ``True`` will instead expect input to be in the ``[0,1]`` range. Example: >>> from torch import rand >>> from torchmetrics.functional.image.lpips import learned_perceptual_image_patch_similarity >>> img1 = (rand(10, 3, 100, 100) * 2) - 1 >>> img2 = (rand(10, 3, 100, 100) * 2) - 1 >>> learned_perceptual_image_patch_similarity(img1, img2, net_type='squeeze') tensor(0.1005) )r)devicedtype)rtorrrrr)rrrrrrrrrrr rs!)F)T)r)r)rr)rrF))rrtypingrrrrr0rrtyping_extensionsrtorchmetrics.utilities.importsr r__doctest_skip__r3rVmodules containerr1rModuler"rXrnrrrrfloatrrrrrrrrrrrrrr sX  (22 o0(