o .wi/@sHddlZddlmZddlZddlmZmZddlmZddl m Z ddl m Z m Z e r8ddlmZddlmZd d d Ze d Zd Ze rGe sJd gZGdddejZGdddeZ d%dedejdededeeeeefff ddZ d&dedeeefde ddefdd Z !  " d'ded#ede ddededef d$d ZdS)(N)Union)Tensornn normalize)Literal)_TORCH_GREATER_EQUAL_2_2_TORCHVISION_AVAILABLE) transforms)resnet50))r d)kadid10kkoniq10kz=https://github.com/miccunifi/ARNIQA/releases/download/weightsarniqac seZdZdZddeddffdd Zddd Zdd ed ede eeffd dZ dedefddZ dd ed edefddZ Z S)_ARNIQAzInitializes a No-Reference Image Quality Assessment ARNIQA torch.nn.Module. Args: regressor_dataset: dataset used for training the regressor, choose between [``koniq10k``, ``kadid10k``] rregressor_datasetreturnNcstts tdtstdt}||vr$td|d|d||_ gd|_ gd|_ t }|j j|_tjt|dd}||_t|jd d |_|d tjd dfd d}||j||jdS)Nz'ARNIQA metric requires PyTorch >= 2.2.0zARNIQA metric requires that torchvision is installed. Either install as `pip install torchmetrics[image]` or `pip install torchvision`.z,Argument `regressor_dataset` must be one of , but got .)g ףp= ?gv/?gCl?)gZd;O?gy&1?g?r modulercSs ||D]}d|_qdS)NF)eval parameters requires_grad)rpra/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/torchmetrics/functional/image/arniqa.py_freezeUs z!_ARNIQA.__init__.._freeze)super__init__r RuntimeErrorr ModuleNotFoundError_AVAILABLE_REGRESSOR_DATASETSkeys ValueErrorrimagenet_norm_meanimagenet_norm_stdr fc in_featuresfeat_dimr SequentiallistchildrenencoderLinear regressor _load_weightsModule)selfrvalid_regressor_datasetsr0r  __class__rrr"8s0     z_ARNIQA.__init__cCstjjtdddd}dd|D}|jj|ddt<tj dt d d tjjtd |j d ddd }| d |d<| dd|d<|jj|ddWddS1s`wYdS)z/Loads the weights of the encoder and regressor.z /ARNIQA.pthTcpu)progress map_locationcSs&i|]\}}d|vr|dd|qS) projectorzmodel.)replace).0kvrrr bs z)_ARNIQA._load_weights..)strictignoreztorch.serialization)categoryrz /regressor_z.pthweightsweightbiasesrbiasN)torchhubload_state_dict_from_url _base_urlitemsr0load_state_dictwarningscatch_warningsfilterwarnings UserWarningr state_dictpop unsqueezer2)r5encoder_state_dictfiltered_encoder_state_dictregressor_state_dictrrrr3]s$  "z_ARNIQA._load_weightsFimgrcCsd|jdd\}}t|d|df|}|r.tj|j|jd|}tj|j|jd|}||fS)zPreprocesses the input to the model. Obtains the half-scale version of the input image and applies normalization if needed. Nr)meanstd)shaper Resize Normalizer(r))r5rZrhwimg_dsrrr_preprocess_inputqs z_ARNIQA._preprocess_inputscorecCst|j\}}||||S)zKScales the quality score to be in the [0, 1] range, where higher is better.)r%r)r5re min_score max_scorerrr _scale_score~sz_ARNIQA._scale_scorecCsz|||\}}||}|d|j}t|dd}||}|d|j}t|dd}t||f}||}||S)Nrr )dim) rdr0viewr, normalize_fnrJhstackr2rh)r5rZrrcimg_fimg_ds_ffrerrrforwards      z_ARNIQA.forward)r)rNF)__name__ __module__ __qualname____doc___TYPE_REGRESSOR_DATASETr"r3rbooltuplerdrhrp __classcell__rrr7rr0s %  rcs*eZdZdZdeddffdd ZZS)_NoTrainArniqaz9Wrapper to make sure ARNIQA never leaves evaluation mode.modercs tdS)z.Force network to always be in evaluation mode.F)r!train)r5r{r7rrr|s z_NoTrainArniqa.train)rrrsrtrurwr|ryrrr7rrzsrzFrZmodelrautocastrcCs|jdkr |jddkstd|jd|dkr!|dks2|r2td|d |d|rVtjj|jj |j d |||d }Wd n1sPwYn |j |j d ||d }| |jdfS)a Update step for ARNIQA metric. Args: img: the input image model: the pre-trained model normalize: boolean indicating whether the input image is normalized autocast: boolean indicating whether to use automatic mixed precision r z?Input image must have shape [N, 3, H, W]. Got input with shape rg?gzdInput image values must be in the [0, 1] range when normalize==True. Got input with values in range z and ) device_typedtyperN)rr) ndimr^r'maxminrJampr~devicetypertosqueeze)rZr}rr~lossrrr_arniqa_updates" rr\scores num_scores reduction)sumr\nonecCs(|}|dkr |S|dkr||S|S)zCompute step for ARNIQA metric.rr\)r)rrr sum_scoresrrr_arniqa_computes rrTrc Cspd}||vrtd|d|t|tstd|t|dj|j|jd}t||||d\}}t|||S)aARNIQA: leArning distoRtion maNifold for Image Quality Assessment metric. `ARNIQA`_ is a No-Reference Image Quality Assessment metric that predicts the technical quality of an image with a high correlation with human opinions. ARNIQA consists of an encoder and a regressor. The encoder is a ResNet-50 model trained in a self-supervised way to model the image distortion manifold to generate similar representation for images with similar distortions, regardless of the image content. The regressor is a linear model trained on IQA datasets using the ground-truth quality scores. ARNIQA extracts the features from the full- and half-scale versions of the input image and then outputs a quality score in the [0, 1] range, where higher is better. The input image is expected to have shape ``(N, 3, H, W)``. The image should be in the [0, 1] range if `normalize` is set to ``True``, otherwise it should be normalized with the ImageNet mean and standard deviation. .. note:: Using this metric requires you to have ``torchvision`` package installed. Either install as ``pip install torchmetrics[image]`` or ``pip install torchvision``. Args: img: the input image regressor_dataset: dataset used for training the regressor. Choose between [``koniq10k``, ``kadid10k``]. ``koniq10k`` corresponds to the `KonIQ-10k`_ dataset, which consists of real-world images with authentic distortions. ``kadid10k`` corresponds to the `KADID-10k`_ dataset, which consists of images with synthetically generated distortions. reduction: indicates how to reduce over the batch dimension. Choose between [``sum``, ``mean``, ``none``]. normalize: by default this is ``True`` meaning that the input is expected to be in the [0, 1] range. If set to ``False`` will instead expect input to be already normalized with the ImageNet mean and standard deviation. autocast: boolean indicating whether to use automatic mixed precision Returns: A tensor in the [0, 1] range, where higher is better, representing the ARNIQA score of the input image. If `reduction` is set to ``none``, the output will have shape ``(N,)``, otherwise it will be a scalar tensor. Raises: ModuleNotFoundError: If ``torchvision`` package is not installed ValueError: If ``regressor_dataset`` is not in [``"kadid10k"``, ``"koniq10k"``] ValueError: If ``reduction`` is not in [``"sum"``, ``"mean"``, ``"none"``] ValueError: If ``normalize`` is not a bool ValueError: If the input image is not a valid image tensor with shape [N, 3, H, W]. ValueError: If the input image values are not in the [0, 1] range when ``normalize`` is set to ``True`` Examples: >>> from torch import rand >>> from torchmetrics.functional.image.arniqa import arniqa >>> img = rand(8, 3, 224, 224) >>> # Non-normalized input >>> arniqa(img, regressor_dataset='koniq10k', normalize=True) tensor(0.5308) >>> from torch import rand >>> from torchmetrics.functional.image.arniqa import arniqa >>> from torchvision.transforms import Normalize >>> img = rand(8, 3, 224, 224) >>> img = Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])(img) >>> # Normalized input >>> arniqa(img, regressor_dataset='koniq10k', normalize=False) tensor(0.5065) )r\rrz$Argument `reduction` must be one of rz.Argument `normalize` should be a bool but got )r)rr)rr~) r' isinstancerwrzrrrrr) rZrrrr~valid_reductionr}rrrrrrsH  rq)r\)rr\TF)rPtypingrrJrrtorch.nn.functionalrrktyping_extensionsrtorchmetrics.utilities.importsrr torchvisionr torchvision.modelsr r%rvrM__doctest_skip__r4rrzrwrxintrrrrrrrst     e