o yi3 @sddlmZmZmZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZmZded ed ed ed eeeff d dZGdddeZGdddeZdS))AnyOptionalTupleN)Tensortensor)Metric) retrieval_precision_recall_curve)_check_retrieval_inputs)_flexible_bincount dim_zero_cat precisionrecalltop_k min_precisionreturncsztfddt|||D\}}Wnty-tjd|j|jd}tt|}Ynw|dkr>tjt||j|jd}||fS)aComputes maximum recall with condition that corresponding precision >= `min_precision`. Args: top_k: tensor with all possible k precision: tensor with all values precisions@k for k from top_k tensor recall: tensor with all values recall@k for k from top_k tensor min_precision: float value specifying minimum precision threshold. Returns: Maximum recall value, corresponding it best k c3s&|]\}}}|kr||fVqdSN).0prkrra/home/ubuntu/.local/lib/python3.10/site-packages/torchmetrics/retrieval/precision_recall_curve.py +s$z7_retrieval_recall_at_fixed_precision..)devicedtype)maxzip ValueErrortorchrrrlen)r r rr max_recallbest_krrr$_retrieval_recall_at_fixed_precisions& r$c seZdZUdZdZeed<dZeed<dZeed<    dd e e d ed e d e e d e ddf fdd Z dedededdfddZdeeeeffddZZS)RetrievalPrecisionRecallCurvea* Computes precision-recall pairs for different k (from 1 to `max_k`). In a ranked retrieval context, appropriate sets of retrieved documents are naturally given by the top k retrieved documents. Recall is the fraction of relevant documents retrieved among all the relevant documents. Precision is the fraction of relevant documents among all the retrieved documents. For each such set, precision and recall values can be plotted to give a recall-precision curve. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): A float tensor of shape ``(N, ...)`` - ``target`` (:class:`~torch.Tensor`): A long or bool tensor of shape ``(N, ...)`` - ``indexes`` (:class:`~torch.Tensor`): A long tensor of shape ``(N, ...)`` which indicate to which query a prediction belongs As output to ``forward`` and ``compute`` the metric returns the following output: - ``precisions`` (:class:`~torch.Tensor`): A tensor with the fraction of relevant documents among all the retrieved documents. - ``recalls`` (:class:`~torch.Tensor`): A tensor with the fraction of relevant documents retrieved among all the relevant documents - ``top_k`` (:class:`~torch.Tensor`): A tensor with k from 1 to `max_k` All ``indexes``, ``preds`` and ``target`` must have the same dimension and will be flatten at the beginning, so that for example, a tensor of shape ``(N, M)`` is treated as ``(N * M, )``. Predictions will be first grouped by ``indexes`` and then will be computed as the mean of the metric over each query. Args: max_k: Calculate recall and precision for all possible top k from 1 to max_k (default: `None`, which considers all possible top k) adaptive_k: adjust `k` to `min(k, number of documents)` for each query empty_target_action: Specify what to do with queries that do not have at least a positive ``target``. Choose from: - ``'neg'``: those queries count as ``0.0`` (default) - ``'pos'``: those queries count as ``1.0`` - ``'skip'``: skip those queries; if all queries are skipped, ``0.0`` is returned - ``'error'``: raise a ``ValueError`` ignore_index: Ignore predictions where the target is equal to this number. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Raises: ValueError: If ``empty_target_action`` is not one of ``error``, ``skip``, ``neg`` or ``pos``. ValueError: If ``ignore_index`` is not `None` or an integer. ValueError: If ``max_k`` parameter is not `None` or an integer larger than 0. Example: >>> from torchmetrics import RetrievalPrecisionRecallCurve >>> indexes = tensor([0, 0, 0, 0, 1, 1, 1]) >>> preds = tensor([0.4, 0.01, 0.5, 0.6, 0.2, 0.3, 0.5]) >>> target = tensor([True, False, False, True, True, False, True]) >>> r = RetrievalPrecisionRecallCurve(max_k=4) >>> precisions, recalls, top_k = r(preds, target, indexes=indexes) >>> precisions tensor([1.0000, 0.5000, 0.6667, 0.5000]) >>> recalls tensor([0.5000, 0.5000, 1.0000, 1.0000]) >>> top_k tensor([1, 2, 3, 4]) Fis_differentiableThigher_is_betterfull_state_updateNnegmax_k adaptive_kempty_target_action ignore_indexkwargsrc stjd i|d|_d}||vrtd|d||_|dur*t|ts*td||_|dur>t|tr:|dks>td||_t|t sJtd||_ |j d gdd |j d gdd |j d gdd dS)NF)errorskipr)posz7Argument `empty_target_action` received a wrong value `z`.z3Argument `ignore_index` must be an integer or None.rz,`max_k` has to be a positive integer or Nonez `adaptive_k` has to be a booleanindexes)defaultdist_reduce_fxpredstargetr) super__init__allow_non_binary_targetrr, isinstanceintr-r*boolr+ add_state)selfr*r+r,r-r.empty_target_action_options __class__rrr8~s$ z&RetrievalPrecisionRecallCurve.__init__r5r6r2cCsT|durtdt||||j|jd\}}}|j||j||j|dS)zGCheck shape, check and convert dtypes, flatten and add to accumulators.Nz!Argument `indexes` cannot be None)r9r-)rr r9r-r2appendr5r6)r>r5r6r2rrrupdates  z$RetrievalPrecisionRecallCurve.updatecst|j}t|jt|j}t|\}}|||}t| }|j }|dur3t |}gg}}t tj |ddtj ||ddD]^\}} | s|jdkrZtd|jdkrv|tj|jd|tj|jdqI|jdkr|tj|jd|tj|jdqIt|| ||j\} } } || || qI|rtfdd |Djddnt|} |rtfd d |Djddnt|} tjd |d jd} | | | fS) Nr)dimr/zC`compute` method was provided with a query with no positive target.r1)rr)cg|]}|qSrtorxr5rr z9RetrievalPrecisionRecallCurve.compute..crErrFrHrJrrrKrL)r r2r5r6r sortr detachcputolistr*rrsplitsumr,rrBonesrzerosrr+stackmeanrGarange)r>r2r6indices split_sizesr* precisionsrecalls mini_preds mini_targetr r _rrrJrcomputes@          24 z%RetrievalPrecisionRecallCurve.compute)NFr)N)__name__ __module__ __qualname____doc__r&r<__annotations__r'r(rr;strrr8rrCrr` __classcell__rrr@rr%7s0  B  " r%cspeZdZdZdZ     ddedeed ed e d eed e d dffdd Z d e e e fffdd ZZS)RetrievalRecallAtFixedPrecisiona Computes `IR Recall at fixed Precision`_. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): A float tensor of shape ``(N, ...)`` - ``target`` (:class:`~torch.Tensor`): A long or bool tensor of shape ``(N, ...)`` - ``indexes`` (:class:`~torch.Tensor`): A long tensor of shape ``(N, ...)`` which indicate to which query a prediction belongs .. note:: All ``indexes``, ``preds`` and ``target`` must have the same dimension. .. note:: Predictions will be first grouped by ``indexes`` and then `RetrievalRecallAtFixedPrecision` will be computed as the mean of the `RetrievalRecallAtFixedPrecision` over each query. As output to ``forward`` and ``compute`` the metric returns the following output: - ``max_recall`` (:class:`~torch.Tensor`): A tensor with the maximum recall value retrieved documents. - ``best_k`` (:class:`~torch.Tensor`): A tensor with the best k corresponding to the maximum recall value Args: min_precision: float value specifying minimum precision threshold. max_k: Calculate recall and precision for all possible top k from 1 to max_k (default: `None`, which considers all possible top k) adaptive_k: adjust `k` to `min(k, number of documents)` for each query empty_target_action: Specify what to do with queries that do not have at least a positive ``target``. Choose from: - ``'neg'``: those queries count as ``0.0`` (default) - ``'pos'``: those queries count as ``1.0`` - ``'skip'``: skip those queries; if all queries are skipped, ``0.0`` is returned - ``'error'``: raise a ``ValueError`` ignore_index: Ignore predictions where the target is equal to this number. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Raises: ValueError: If ``empty_target_action`` is not one of ``error``, ``skip``, ``neg`` or ``pos``. ValueError: If ``ignore_index`` is not `None` or an integer. ValueError: If ``min_precision`` parameter is not float or between 0 and 1. ValueError: If ``max_k`` parameter is not `None` or an integer larger than 0. Example: >>> from torchmetrics import RetrievalRecallAtFixedPrecision >>> indexes = tensor([0, 0, 0, 0, 1, 1, 1]) >>> preds = tensor([0.4, 0.01, 0.5, 0.6, 0.2, 0.3, 0.5]) >>> target = tensor([True, False, False, True, True, False, True]) >>> r = RetrievalRecallAtFixedPrecision(min_precision=0.8) >>> r(preds, target, indexes=indexes) (tensor(0.5000), tensor(1)) TrNFr)rr*r+r,r-r.rc sTtjd||||d|t|tr!d|kr dks%tdtd||_dS)N)r*r+r,r-rg?z:`min_precision` has to be a positive float between 0 and 1r)r7r8r:floatrr)r>rr*r+r,r-r.r@rrr8s  z(RetrievalRecallAtFixedPrecision.__init__cs t\}}}t||||jSr)r7r`r$r)r>r[r\rr@rrr`0sz'RetrievalRecallAtFixedPrecision.compute)rNFr)N)rarbrcrdr'rirr;r<rfrr8rrr`rgrrr@rrhs0;"rh)typingrrrr rr torchmetricsr8torchmetrics.functional.retrieval.precision_recall_curvertorchmetrics.utilities.checksr torchmetrics.utilities.datar r rir$r%rhrrrrs*      &