o .wi@snddlZddlmZddlmZddededdefd d Zdedefd d Zddededdefd dZdS)N)Tensor)LiteralcountsratingsmoderprobsreturncCs|dkr0|jdks |std|jdd}tjjj||jdd ddd}|j d d}|S|d krA|jdks=|rAtd |S) zUpdates the counts for fleiss kappa metric. Args: ratings: ratings matrix mode: whether ratings are provided as counts or probabilities rzIf argument ``mode`` is 'probs', ratings must have 3 dimensions with the format [n_samples, n_categories, n_raters] and be floating point.dim) num_classesrrzIf argument ``mode`` is `counts`, ratings must have 2 dimensions with the format [n_samples, n_categories] and be none floating point.) ndimis_floating_point ValueErrorargmaxtorchnn functionalone_hotshapepermutesum)rrrri/home/ubuntu/sommelier/.venv/lib/python3.10/site-packages/torchmetrics/functional/nominal/fleiss_kappa.py_fleiss_kappa_updates " rcCst|jd}|d}|jdd||}|djdd|||d}|}|d}||d|dS)z{Computes fleiss kappa from counts matrix. Args: counts: counts matrix of shape [n_samples, n_categories] rr r rgh㈵>)rrmaxmean)rtotal num_ratersp_ip_jp_barpe_barrrr_fleiss_kappa_compute,s   r'cCs"|dvrtdt||}t|S)aCalculatees `Fleiss kappa`_ a statistical measure for inter agreement between raters. .. math:: \kappa = \frac{\bar{p} - \bar{p_e}}{1 - \bar{p_e}} where :math:`\bar{p}` is the mean of the agreement probability over all raters and :math:`\bar{p_e}` is the mean agreement probability over all raters if they were randomly assigned. If the raters are in complete agreement then the score 1 is returned, if there is no agreement among the raters (other than what would be expected by chance) then a score smaller than 0 is returned. Args: ratings: Ratings of shape [n_samples, n_categories] or [n_samples, n_categories, n_raters] depedenent on `mode`. If `mode` is `counts`, `ratings` must be integer and contain the number of raters that chose each category. If `mode` is `probs`, `ratings` must be floating point and contain the probability/logits that each rater chose each category. mode: Whether `ratings` will be provided as counts or probabilities. Example: >>> # Ratings are provided as counts >>> from torch import randint >>> from torchmetrics.functional.nominal import fleiss_kappa >>> ratings = randint(0, 10, size=(100, 5)).long() # 100 samples, 5 categories, 10 raters >>> fleiss_kappa(ratings) tensor(0.0089) Example: >>> # Ratings are provided as probabilities >>> from torch import randn >>> from torchmetrics.functional.nominal import fleiss_kappa >>> ratings = randn(100, 5, 10).softmax(dim=1) # 100 samples, 5 categories, 10 raters >>> fleiss_kappa(ratings, mode='probs') tensor(-0.0075) rz5Argument ``mode`` must be one of ['counts', 'probs'].)rrr')rrrrrr fleiss_kappa=s# r()r)rrtyping_extensionsrrr'r(rrrrs