o yiZ@sbddlmZmZmZmZddlZddlmZmZddlm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZddlmZmZmZddlmZmZddlmZdd l m!Z! d6d eeeefd eeeefd ee ddeedef ddZ"   d7dee#deee$ee#efdee$ddfddZ%  d8deeeeeffdeedee#de$deeeeeefff ddZ&    d9dededee#deee$ee#efdee$d e'deeeeffd!d"Z(  d:d#e$d ee ddeee$ee#efdee$ddf d$d%Z) d6deeeeeffd#e$d ee ddeedef d&d'Z*   d;deded#e$d ee ddeee$ee#efdee$d e'defd(d)Z+  ddeded2e d3deee$ee#efd#ee$d*ee$d ee ddee#dee$d e'deeeeeefeeeeeeefffd4d5Z/dS)?)ListOptionalTupleUnionN)Tensortensor)Literal) -_binary_precision_recall_curve_arg_validation%_binary_precision_recall_curve_format0_binary_precision_recall_curve_tensor_validation%_binary_precision_recall_curve_update1_multiclass_precision_recall_curve_arg_validation)_multiclass_precision_recall_curve_format4_multiclass_precision_recall_curve_tensor_validation)_multiclass_precision_recall_curve_update1_multilabel_precision_recall_curve_arg_validation)_multilabel_precision_recall_curve_format4_multilabel_precision_recall_curve_tensor_validation)_multilabel_precision_recall_curve_update)_binary_roc_compute_multiclass_roc_compute_multilabel_roc_compute)_auc_compute_without_check _safe_divide) _bincount)rank_zero_warnmacrofprtpraverage)rweightednoneweightsreturncCst|trt||ddd}nddt||D}t|}|dus%|dkr'|St|r7td|d t t|}|d krG|| S|d krb|durbt |||| }||| St d ) zOUtility function for reducing multiple average precision score into one number.?)axiscSsg|] \}}t||dqS)r$)r).0xyr*`/home/ubuntu/.local/lib/python3.10/site-packages/torchmetrics/functional/classification/auroc.py 6sz!_reduce_auroc..Nr!zUAverage precision score for one or more classes was `nan`. Ignoring these classes in z-averagerr zBReceived an incompatible combinations of inputs to make reduction.) isinstancerrziptorchstackisnananyr UserWarningmeanrsum ValueError)rrrr"residxr*r*r+ _reduce_auroc,s$     r9max_fpr thresholds ignore_indexcCsPt|||dur"t|ts$d|krdkr&ndStd|dSdSdS)Nrr%z@Arguments `max_fpr` should be a float in range (0, 1], but got: )r r-floatr6)r:r;r<r*r*r+_binary_auroc_arg_validationIs , r>r%state pos_labelcCs t|||\}}}|dus|dkrt||dSt|tr|jn|dj}t||d}tj||ddd} ||| d|| || d} t|| d|| | } t |d| | dg}t |d| | dg}t||d} d|d} dd| | || S) Nr%r$r)deviceT) out_int32rightg?) rrr-rrArr/ bucketizelerpcatview)r?r;r:r@rr__devicemax_areastopweight interp_tpr partial_aucmin_arear*r*r+_binary_auroc_computeSs  $  rQTpredstarget validate_argscCsH|rt|||t|||t||||\}}}t|||}t|||S)a` Compute Area Under the Receiver Operating Characteristic Curve (`ROC AUC`_) for binary tasks. The AUROC score summarizes the ROC curve into an single number that describes the performance of a model for multiple thresholds at the same time. Notably, an AUROC score of 1 is a perfect score and an AUROC score of 0.5 corresponds to random guessing. Accepts the following input tensors: - ``preds`` (float tensor): ``(N, ...)``. Preds should be a tensor containing probabilities or logits for each observation. If preds has values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. - ``target`` (int tensor): ``(N, ...)``. Target should be a tensor containing ground truth labels, and therefore only contain {0,1} values (except if `ignore_index` is specified). The value 1 always encodes the positive class. Additional dimension ``...`` will be flattened into the batch dimension. The implementation both supports calculating the metric in a non-binned but accurate version and a binned version that is less accurate but more memory efficient. Setting the `thresholds` argument to `None` will activate the non-binned version that uses memory of size :math:`\mathcal{O}(n_{samples})` whereas setting the `thresholds` argument to either an integer, list or a 1d tensor will use a binned version that uses memory of size :math:`\mathcal{O}(n_{thresholds})` (constant memory). Args: preds: Tensor with predictions target: Tensor with true labels max_fpr: If not ``None``, calculates standardized partial AUC over the range ``[0, max_fpr]``. thresholds: Can be one of: - If set to `None`, will use a non-binned approach where thresholds are dynamically calculated from all the data. Most accurate but also most memory consuming approach. - If set to an `int` (larger than 1), will use that number of thresholds linearly spaced from 0 to 1 as bins for the calculation. - If set to an `list` of floats, will use the indicated thresholds in the list as bins for the calculation - If set to an 1d `tensor` of floats, will use the indicated thresholds in the tensor as bins for the calculation. validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. Returns: A single scalar with the auroc score Example: >>> from torchmetrics.functional.classification import binary_auroc >>> preds = torch.tensor([0, 0.5, 0.7, 0.8]) >>> target = torch.tensor([0, 1, 1, 0]) >>> binary_auroc(preds, target, thresholds=None) tensor(0.5000) >>> binary_auroc(preds, target, thresholds=5) tensor(0.5000) )r>r r r rQ)rRrSr:r;r<rTr?r*r*r+ binary_aurocns ;    rU num_classescC0t|||d}||vrtd|d|dS)N)rr r!N)Expected argument `average` to be one of but got )r r6)rVrr;r<allowed_averager*r*r+ _multiclass_auroc_arg_validation r[c Cs^t|||\}}}t||||durt|d|ddS|ddddddfddS)Nr%) minlengthrr")rr9rr=r5)r?rVrr;rrrIr*r*r+_multiclass_auroc_computesr`cCsR|rt||||t||||t|||||\}}}t||||}t||||S)aCompute Area Under the Receiver Operating Characteristic Curve (`ROC AUC`_) for multiclass tasks. The AUROC score summarizes the ROC curve into an single number that describes the performance of a model for multiple thresholds at the same time. Notably, an AUROC score of 1 is a perfect score and an AUROC score of 0.5 corresponds to random guessing. Accepts the following input tensors: - ``preds`` (float tensor): ``(N, C, ...)``. Preds should be a tensor containing probabilities or logits for each observation. If preds has values outside [0,1] range we consider the input to be logits and will auto apply softmax per sample. - ``target`` (int tensor): ``(N, ...)``. Target should be a tensor containing ground truth labels, and therefore only contain values in the [0, n_classes-1] range (except if `ignore_index` is specified). Additional dimension ``...`` will be flattened into the batch dimension. The implementation both supports calculating the metric in a non-binned but accurate version and a binned version that is less accurate but more memory efficient. Setting the `thresholds` argument to `None` will activate the non-binned version that uses memory of size :math:`\mathcal{O}(n_{samples})` whereas setting the `thresholds` argument to either an integer, list or a 1d tensor will use a binned version that uses memory of size :math:`\mathcal{O}(n_{thresholds} \times n_{classes})` (constant memory). Args: preds: Tensor with predictions target: Tensor with true labels num_classes: Integer specifing the number of classes average: Defines the reduction that is applied over classes. Should be one of the following: - ``macro``: Calculate score for each class and average them - ``weighted``: Calculates score for each class and computes weighted average using their support - ``"none"`` or ``None``: Calculates score for each class and applies no reduction thresholds: Can be one of: - If set to `None`, will use a non-binned approach where thresholds are dynamically calculated from all the data. Most accurate but also most memory consuming approach. - If set to an `int` (larger than 1), will use that number of thresholds linearly spaced from 0 to 1 as bins for the calculation. - If set to an `list` of floats, will use the indicated thresholds in the list as bins for the calculation - If set to an 1d `tensor` of floats, will use the indicated thresholds in the tensor as bins for the calculation. validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. Returns: If `average=None|"none"` then a 1d tensor of shape (n_classes, ) will be returned with auroc score per class. If `average="macro"|"weighted"` then a single scalar is returned. Example: >>> from torchmetrics.functional.classification import multiclass_auroc >>> preds = torch.tensor([[0.75, 0.05, 0.05, 0.05, 0.05], ... [0.05, 0.75, 0.05, 0.05, 0.05], ... [0.05, 0.05, 0.75, 0.05, 0.05], ... [0.05, 0.05, 0.05, 0.75, 0.05]]) >>> target = torch.tensor([0, 1, 3, 2]) >>> multiclass_auroc(preds, target, num_classes=5, average="macro", thresholds=None) tensor(0.5333) >>> multiclass_auroc(preds, target, num_classes=5, average=None, thresholds=None) tensor([1.0000, 1.0000, 0.3333, 0.3333, 0.0000]) >>> multiclass_auroc(preds, target, num_classes=5, average="macro", thresholds=5) tensor(0.5333) >>> multiclass_auroc(preds, target, num_classes=5, average=None, thresholds=5) tensor([1.0000, 1.0000, 0.3333, 0.3333, 0.0000]) )r[rrrr`)rRrSrVrr;r<rTr?r*r*r+multiclass_aurocsJ  ra num_labels)microrr r!cCrW)N)rcrr r!NrXrY)rr6)rbrr;r<rZr*r*r+ _multilabel_auroc_arg_validation r\rdc Cs|dkr>t|tr|durt|d|ddS|d}|d}|dur5||k}||}||}t||f|ddSt||||\}} } t|| ||dur^|ddkjdddS|ddddddfddS)Nrcr%)r:r)dimr^r_)r-rrQr5flattenrr9r=) r?rbrr;r<rRrSr8rrrIr*r*r+_multilabel_auroc_compute,s&    rgcCsT|rt||||t||||t|||||\}}}t||||}t|||||S)aCompute Area Under the Receiver Operating Characteristic Curve (`ROC AUC`_) for multilabel tasks. The AUROC score summarizes the ROC curve into an single number that describes the performance of a model for multiple thresholds at the same time. Notably, an AUROC score of 1 is a perfect score and an AUROC score of 0.5 corresponds to random guessing. Accepts the following input tensors: - ``preds`` (float tensor): ``(N, C, ...)``. Preds should be a tensor containing probabilities or logits for each observation. If preds has values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. - ``target`` (int tensor): ``(N, C, ...)``. Target should be a tensor containing ground truth labels, and therefore only contain {0,1} values (except if `ignore_index` is specified). Additional dimension ``...`` will be flattened into the batch dimension. The implementation both supports calculating the metric in a non-binned but accurate version and a binned version that is less accurate but more memory efficient. Setting the `thresholds` argument to `None` will activate the non-binned version that uses memory of size :math:`\mathcal{O}(n_{samples})` whereas setting the `thresholds` argument to either an integer, list or a 1d tensor will use a binned version that uses memory of size :math:`\mathcal{O}(n_{thresholds} \times n_{labels})` (constant memory). Args: preds: Tensor with predictions target: Tensor with true labels num_labels: Integer specifing the number of labels average: Defines the reduction that is applied over labels. Should be one of the following: - ``micro``: Sum score over all labels - ``macro``: Calculate score for each label and average them - ``weighted``: Calculates score for each label and computes weighted average using their support - ``"none"`` or ``None``: Calculates score for each label and applies no reduction thresholds: Can be one of: - If set to `None`, will use a non-binned approach where thresholds are dynamically calculated from all the data. Most accurate but also most memory consuming approach. - If set to an `int` (larger than 1), will use that number of thresholds linearly spaced from 0 to 1 as bins for the calculation. - If set to an `list` of floats, will use the indicated thresholds in the list as bins for the calculation - If set to an 1d `tensor` of floats, will use the indicated thresholds in the tensor as bins for the calculation. validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. Returns: If `average=None|"none"` then a 1d tensor of shape (n_classes, ) will be returned with auroc score per class. If `average="micro|macro"|"weighted"` then a single scalar is returned. Example: >>> from torchmetrics.functional.classification import multilabel_auroc >>> preds = torch.tensor([[0.75, 0.05, 0.35], ... [0.45, 0.75, 0.05], ... [0.05, 0.55, 0.75], ... [0.05, 0.65, 0.05]]) >>> target = torch.tensor([[1, 0, 1], ... [0, 0, 0], ... [0, 1, 1], ... [1, 1, 1]]) >>> multilabel_auroc(preds, target, num_labels=3, average="macro", thresholds=None) tensor(0.6528) >>> multilabel_auroc(preds, target, num_labels=3, average=None, thresholds=None) tensor([0.6250, 0.5000, 0.8333]) >>> multilabel_auroc(preds, target, num_labels=3, average="macro", thresholds=5) tensor(0.6528) >>> multilabel_auroc(preds, target, num_labels=3, average=None, thresholds=5) tensor([0.6250, 0.5000, 0.8333]) )rdrrrrg)rRrSrbrr;r<rTr?r*r*r+multilabel_aurocIsN  rhtask)binary multiclass multilabelc Cs||dkr t|||||| S|dkr"t|tsJt||||||| S|dkr7t|ts-Jt||||||| Std|)a:Compute Area Under the Receiver Operating Characteristic Curve (`ROC AUC`_). The AUROC score summarizes the ROC curve into an single number that describes the performance of a model for multiple thresholds at the same time. Notably, an AUROC score of 1 is a perfect score and an AUROC score of 0.5 corresponds to random guessing. This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the ``task`` argument to either ``'binary'``, ``'multiclass'`` or ``multilabel``. See the documentation of :func:`binary_auroc`, :func:`multiclass_auroc` and :func:`multilabel_auroc` for the specific details of each argument influence and examples. Legacy Example: >>> preds = torch.tensor([0.13, 0.26, 0.08, 0.19, 0.34]) >>> target = torch.tensor([0, 0, 1, 1, 1]) >>> auroc(preds, target, task='binary') tensor(0.5000) >>> preds = torch.tensor([[0.90, 0.05, 0.05], ... [0.05, 0.90, 0.05], ... [0.05, 0.05, 0.90], ... [0.85, 0.05, 0.10], ... [0.10, 0.10, 0.80]]) >>> target = torch.tensor([0, 1, 1, 2, 2]) >>> auroc(preds, target, task='multiclass', num_classes=3) tensor(0.7778) rjrkrlz[Expected argument `task` to either be `'binary'`, `'multiclass'` or `'multilabel'` but got )rUr-intrarhr6) rRrSrir;rVrbrr:r<rTr*r*r+aurocs$rn)rN)NNN)Nr%)NNNT)rNN)rNNT)NN)N)NNNrNNT)0typingrrrrr/rrtyping_extensionsr=torchmetrics.functional.classification.precision_recall_curver r r r r rrrrrrr*torchmetrics.functional.classification.rocrrrtorchmetrics.utilities.computerrtorchmetrics.utilities.datartorchmetrics.utilities.printsrr9r=rmr>rQboolrUr[r`rardrgrhrnr*r*r*r+s  8         E      W    ! * \   ,