o .wi5@sddlmZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z mZddlmZmZmZddlmZdd lmZdd lmZdd lmZmZesTd d gZGddde ZGdddeZGddde ZdS))Sequence)AnyOptionalUnion)Tensor)Literal)_ClassificationTaskWrapper)BinaryConfusionMatrixMulticlassConfusionMatrix)"_binary_cohen_kappa_arg_validation_cohen_kappa_reduce&_multiclass_cohen_kappa_arg_validation)Metric)ClassificationTaskNoMultilabel)_MATPLOTLIB_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPEBinaryCohenKappa.plotMulticlassCohenKappa.plotc seZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed < dd e de e de eddededd f fdd ZdefddZ dde eeeefde edefddZZS)BinaryCohenKappaa Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for binary tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): A int or float tensor of shape ``(N, ...)``. If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Additionally, we convert to int tensor with thresholding using the value in ``threshold``. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. tip:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``bc_kappa`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: threshold: Threshold for transforming probability to binary (0,1) predictions ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (preds is int tensor): >>> from torch import tensor >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Example (preds is float tensor): >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0.35, 0.85, 0.48, 0.01]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Fis_differentiableThigher_is_betterfull_state_updateplot_lower_bound?plot_upper_bound?N threshold ignore_indexweightslinear quadraticnone validate_argskwargsreturnc <tj||fddd||rt|||||_||_dSNF) normalizer%)super__init__r r r%)selfrrr r%r& __class__d/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/torchmetrics/classification/cohen_kappa.pyr,d   zBinaryCohenKappa.__init__cCt|j|jSzCompute metric.r confmatr r-r0r0r1computerzBinaryCohenKappa.computevalaxcC |||S)a9Plot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting a single value >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> metric.update(rand(10), randint(2,(10,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting multiple values >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> values = [ ] >>> for _ in range(10): ... values.append(metric(rand(10), randint(2,(10,)))) >>> fig_, ax_ = metric.plot(values) _plotr-r:r;r0r0r1plotv (r)rNNTNN)__name__ __module__ __qualname____doc__rbool__annotations__rrrfloatrrintrrr,rr8rrrrr@ __classcell__r0r0r.r1r$sB  9     rc seZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <d Z e ed < ddedeedeeddededd f fdd ZdefddZ ddeeeeefdeedefddZZS)MulticlassCohenKappaa" Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for multiclass tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): Either an int tensor of shape ``(N, ...)` or float tensor of shape ``(N, C, ..)``. If preds is a floating point we apply ``torch.argmax`` along the ``C`` dimension to automatically convert probabilities/logits into an int tensor. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. tip:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``mcck`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: num_classes: Integer specifying the number of classes ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (pred is integer tensor): >>> from torch import tensor >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([2, 1, 0, 1]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) Example (pred is float tensor): >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([[0.16, 0.26, 0.58], ... [0.22, 0.61, 0.17], ... [0.71, 0.09, 0.20], ... [0.05, 0.82, 0.13]]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) FrTrrrrrrClassplot_legend_nameN num_classesrr r!r%r&r'c r(r))r+r,r r r%)r-rOrr r%r&r.r0r1r,r2zMulticlassCohenKappa.__init__cCr3r4r5r7r0r0r1r8r9zMulticlassCohenKappa.computer:r;cCr<)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a single value >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric.update(randn(20,3).softmax(dim=-1), randint(3, (20,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a multiple values >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> values = [] >>> for _ in range(20): ... values.append(metric(randn(20,3).softmax(dim=-1), randint(3, (20,)))) >>> fig_, ax_ = metric.plot(values) r=r?r0r0r1r@rAr)NNTrB)rCrDrErFrrGrHrrrrIrrNstrrJrrrr,rr8rrrrr@rKr0r0r.r1rLsB  <      rLc@sbeZdZdZ     ddeddedded eed eed d eed e de de fddZ dS) CohenKappaa0Calculate `Cohen's kappa score`_ that measures inter-annotator agreement. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the ``task`` argument to either ``'binary'`` or ``'multiclass'``. See the documentation of :class:`~torchmetrics.classification.BinaryCohenKappa` and :class:`~torchmetrics.classification.MulticlassCohenKappa` for the specific details of each argument influence and examples. Legacy Example: >>> from torch import tensor >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> cohenkappa = CohenKappa(task="multiclass", num_classes=2) >>> cohenkappa(preds, target) tensor(0.5000) rNTclstask)binary multiclassrrOr r!rr%r&r'cKs~t|}||||d|tjkrt|fi|S|tjkr7t|ts/tdt |dt |fi|Std|d)zInitialize task metric.)r rr%z+`num_classes` is expected to be `int` but `z was passed.`zTask z not supported!) rfrom_strupdateBINARYr MULTICLASS isinstancerJ ValueErrortyperL)rRrSrrOr rr%r&r0r0r1__new__=s    zCohenKappa.__new__)rNNNT) rCrDrErFr\rrIrrJrGrrr]r0r0r0r1rQ"s4  rQN) collections.abcrtypingrrrtorchrtyping_extensionsr torchmetrics.classification.baser,torchmetrics.classification.confusion_matrixr r 2torchmetrics.functional.classification.cohen_kappar r r torchmetrics.metricrtorchmetrics.utilities.enumsrtorchmetrics.utilities.importsrtorchmetrics.utilities.plotrr__doctest_skip__rrLrQr0r0r0r1s"       }