o yi@sddlZddlZddlmZmZddlmZddlmZddl m Z m Z m Z m Z mZmZmZmZmZddlZddlmZddlmZddlmZmZdd lmZmZmZmZmZm Z m!Z!dd l"m#Z#dd l$m%Z%d e&fd dZ'GdddeeZ(ded efddZ)Gddde(Z*dS)N)ABCabstractmethod)contextmanagerdeepcopy) AnyCallableDict GeneratorListOptionalSequenceTupleUnion)Tensor)Module)apply_to_collectionrank_zero_warn)_flatten_squeeze_if_scalar dim_zero_cat dim_zero_max dim_zero_mean dim_zero_min dim_zero_sum)gather_all_tensors)TorchMetricsUserErrorreturncCstjo tjSN)torch distributed is_availableis_initializedr#r#G/home/ubuntu/.local/lib/python3.10/site-packages/torchmetrics/metric.pyjit_distributed_available'r%cseZdZUdZdgZdgZdZeee d<dZ eee d<dZ eee d<de ddffd d Z edefd d Z  ddedeeefdeeeefdeddf ddZejjde de de fddZde de de fddZde de de fddZdeee fddfddZedfdedee ddfd d!Zd"edefd#d$Z dd%d&Z!   ' ddeedee d(ed)eeddf d*d+Z"dd,eddfd-d.Z#e$   ' ' ddeedee d(ed,ed)eede%f d/d0Z&d1edefd2d3Z'e(d4e d5e ddfd6d7Z)e(de fd8d9Z*dd:d;Z+ddd?Z-d@eee fddfdAdBZ.dedCe ddffdDdE Z/eddGdHZ0dIeeej1fddfdJdKZ2ddLdMZ3ddNdOZ4ddPdQZ5dIeeej1fddffdRdS Z6dTede7ffdUdV Z8ddWeddfdXdYZ9  Z dd[eee fd\ed]edeeee fffd^d_ Z:d`e;d\edae;dbedcede?fdjdkZ@ddmdnZAddodpZBddqdrZCddsdtZDddudvZEddwdxZFddydzZGdd{d|ZHdd}d~ZIdddZJdddZKdddZLdddZMdddZNdddZOdddZPdddZQdddZRdddZSdddZTdddZUdddZVdddZWdddZXdddZYdddZZdddZ[dddZ\dddZ]dddZ^dddZ_dddZ`dddZade?ddfddZbdecfddZdddZeZfS)MetricaBase class for all metrics present in the Metrics API. Implements ``add_state()``, ``forward()``, ``reset()`` and a few other things to handle distributed synchronization and per-step metric computation. Override ``update()`` and ``compute()`` functions to implement your own metric. Use ``add_state()`` to register metric state variables which keep track of state on each call of ``update()`` and are synchronized across processes when ``compute()`` is called. Note: Metric state variables can either be :class:`~torch.Tensor` or an empty list which can we used to store :class:`~torch.Tensor`. Note: Different metrics only override ``update()`` and not ``forward()``. A call to ``update()`` is valid, but it won't return the metric value at the current step. A call to ``forward()`` automatically calls ``update()`` and also returns the metric value at the current step. Args: kwargs: additional keyword arguments, see :ref:`Metric kwargs` for more info. - compute_on_cpu: If metric state should be stored on CPU during computations. Only works for list states. - dist_sync_on_step: If metric state should synchronize on ``forward()``. Default is ``False`` - process_group: The process group on which the synchronization is called. Default is the world. - dist_sync_fn: function that performs the allgather option on the metric state. Default is an custom implementation that calls ``torch.distributed.all_gather`` internally. - distributed_available_fn: function that checks if the distributed backend is available. Defaults to a check of ``torch.distributed.is_available()`` and ``torch.distributed.is_initialized()``. - sync_on_compute: If metric state should synchronize when ``compute`` is called. Default is ``True``- deviceis_differentiableNhigher_is_betterfull_state_updatekwargsrc spttjd|jjtd|_| dd|_ t |j t s+t d|j | dd|_t |jt s@t d|j| dd|_| d d|_|jdur`t|js`t d |j| d t|_| d d |_t |jt s|t d|jt|j|_||j|_||j|_d|_d|_d|_|j|_d |_ d|_!d|_"i|_#i|_$i|_%d|_&d|_'dS)Nztorchmetrics.metric.cpucompute_on_cpuFzCExpected keyword argument `compute_on_cpu` to be an `bool` but got dist_sync_on_stepzFExpected keyword argument `dist_sync_on_step` to be an `bool` but got process_group dist_sync_fnzLExpected keyword argument `dist_sync_fn` to be an callable function but got distributed_available_fnsync_on_computeTzCExpected keyword argument `sync_on_compute` to be a `bool` but got r)(super__init__r_C_log_api_usage_once __class____name__r(_devicepopr. isinstancebool ValueErrorr/r0r1callabler%r2r3inspect signatureupdate_update_signature _wrap_update _wrap_computecompute _computed_forward_cache _update_count_to_sync_should_unsync _enable_grad_dtype_convert _defaults _persistent _reductions _is_synced_cache)selfr,r8r#r$r5RsP          zMetric.__init__cCs |jdkS)Nr)rIrSr#r#r$_update_calleds zMetric._update_calledFnamedefaultdist_reduce_fx persistentcCst|ttfrt|tr|rtd|dkrt}n(|dkr t}n!|dkr't}n|dkr.t}n|dkr5t}n |durAt |sAtdt|trJ| }t |||t ||j |<||j|<||j|<dS) aAdds metric state variable. Only used by subclasses. Args: name: The name of the state variable. The variable will then be accessible at ``self.name``. default: Default value of the state; can either be a :class:`~torch.Tensor` or an empty list. The state will be reset to this value when ``self.reset()`` is called. dist_reduce_fx (Optional): Function to reduce state across multiple processes in distributed mode. If value is ``"sum"``, ``"mean"``, ``"cat"``, ``"min"`` or ``"max"`` we will use ``torch.sum``, ``torch.mean``, ``torch.cat``, ``torch.min`` and ``torch.max``` respectively, each with argument ``dim=0``. Note that the ``"cat"`` reduction only makes sense if the state is a list, and not a tensor. The user can also pass a custom function in this parameter. persistent (Optional): whether the state will be saved as part of the modules ``state_dict``. Default is ``False``. Note: Setting ``dist_reduce_fx`` to None will return the metric state synchronized across different processes. However, there won't be any reduction function applied to the synchronized metric state. The metric states would be synced as follows - If the metric state is :class:`~torch.Tensor`, the synced value will be a stacked :class:`~torch.Tensor` across the process dimension if the metric state was a :class:`~torch.Tensor`. The original :class:`~torch.Tensor` metric state retains dimension and hence the synchronized output will be of shape ``(num_process, ...)``. - If the metric state is a ``list``, the synced value will be a ``list`` containing the combined elements from all processes. Note: When passing a custom function to ``dist_reduce_fx``, expect the synchronized metric state to follow the format discussed in the above note. Raises: ValueError: If ``default`` is not a ``tensor`` or an ``empty list``. ValueError: If ``dist_reduce_fx`` is not callable or one of ``"mean"``, ``"sum"``, ``"cat"``, ``None``. zPstate variable must be a tensor or any empty list (where you can append tensors)summeanmaxmincatNzH`dist_reduce_fx` must be callable or one of ['mean', 'sum', 'cat', None])r<rlistr>rrrrrr? contiguoussetattrrrNrOrP)rSrWrXrYrZr#r#r$ add_states(-   zMetric.add_stateargscOsT|jrtd|js|jdus|jr|j|i||_|jS|j|i||_|jS)aZ``forward`` serves the dual purpose of both computing the metric on the current batch of inputs but also add the batch statistics to the overall accumululating metric state. Input arguments are the exact same as corresponding ``update`` method. The returned output is the exact same as the output of ``compute``. zfThe Metric shouldn't be synced when performing ``forward``. HINT: Did you forget to call ``unsync`` ?.N)rQrr+r/_forward_full_state_updaterH_forward_reduce_state_updaterSrdr,r#r#r$forwards zMetric.forwardc sj|i|j}j_d_j}d_fddjD}d_j|i| }| D] \}}t ||q9|_d_ d_j _d_d_|_jra|S)aforward computation using two calls to `update` to calculate the metric value on the current batch and accumulate global state. Doing this secures that metrics that need access to the full metric state during `update` works as expected. Fci|]}|t|qSr#getattr.0attrrUr#r$ z5Metric._forward_full_state_update..TN)rBrIr/rJrKr.rNrLresetrFitemsrbrQr3rG_move_list_states_to_cpu) rSrdr,rI_temp_compute_on_cpucache batch_valrnvalr#rUr$res.z!Metric._forward_full_state_updatecsfddjD}j}j_d_j}d_d_j |i| }|d_t  |Wdn1sHwYd_d_j_d_d_|_jrg|S)zforward computation using single call to `update` to calculate the metric value on the current batch and accumulate global state. This can be done when the global metric state is a sinple reduction of batch states. crir#rjrlrUr#r$ro"rpz7Metric._forward_reduce_state_update..FTN)rNkeysrIrqr/rJrKr.rLrBrFrno_grad_reduce_statesrQr3rGrs)rSrdr, global_staterIrtrvr#rUr$rfs.   z#Metric._forward_reduce_state_updateincoming_statecCs|jD]{}t||}||}|j|}|tkr||}n\|tkr1|jd|||j}nI|tkr|t krGt ||}n3|t krP||}n*|durat|trat ||g}n|durqt|trqt||g}n |t ||g}t|||qdS)zAdds an incoming metric state to the current state of the metric. Args: incoming_state: a dict containing a metric state similar metric itself rxN)rNryrkrPrrrIfloatrrr]rr^rr<rstackr`rrb)rSr}rn local_stater| reduce_fnreducedr#r#r$r{Bs*    zMetric._reduce_statesr1r0csBfddjD}jD] \}}|tkr/t||tr/t||dkr/t||g||<qt|t||p8jd}jD]^\}}t||trZt||dkrZt |gq@t||dtrmt ||||<nt||dtr~t ||||<t |s|dustd|dur|||n||}t ||q@dS)Ncrir#rjrlrUr#r$ro`rpz%Metric._sync_dist..rx)grouprz%reduction_fn must be callable or None)rPrrrr<r`lenrrr0rbrrrr? TypeError)rSr1r0 input_dictrn reduction_fn output_dictrr#rUr$ _sync_dist_s0& zMetric._sync_distrBcs*tdtdtddffdd }|S)Nrdr,rc sd_jd7_tj6z |i|Wn$ty>}zdt|vr8tdjjdjjd||d}~wwWdn1sIwYj rW dSdS)NrxzExpected all tensors to be onzEncountered different devices in metric calculation (see stacktrace for details). This could be due to the metric class not being on the same device as input. Instead of `metric=z(...)` try to do `metric=zF(...).to(device)` where device corresponds to the device of the input.) rGrIrset_grad_enabledrL RuntimeErrorstrr8r9r.rs)rdr,errrSrBr#r$ wrapped_funcs0  z)Metric._wrap_update..wrapped_func functoolswrapsr)rSrBrr#rr$rDszMetric._wrap_updatecCs>|jD]}t||}t|trt||dd|DqdS)z+Move list states to cpu to save GPU memory.cSsg|]}|dqS)r-)tormcur_vr#r#r$ sz3Metric._move_list_states_to_cpu..N)rNryrkr<r rb)rSkey current_valr#r#r$rss   zMetric._move_list_states_to_cpuT should_syncdistributed_availablecsjr |r td|durjdurj}t|r|nd}|r"|s$dS|dur*t}fddjD_j||dd_dS)apSync function for manually controlling when metrics states should be synced across processes. Args: dist_sync_fn: Function to be used to perform states synchronization process_group: Specify the process group on which synchronization is called. default: `None` (which selects the entire world) should_sync: Whether to apply to state synchronization. This will have an impact only when running in a distributed setting. distributed_available: Function to determine if we are running inside a distributed setting z#The Metric has already been synced.Ncrir#rjrlrUr#r$rorpzMetric.sync..)r0T)rQrr2r?rrNrRr)rSr1r0rris_distributedr#rUr$syncs  z Metric.sync should_unsynccCsX|sdS|js td|jdurtd|jD] \}}t|||qd|_d|_dS)zUnsync function for manually controlling when metrics states should be reverted back to their local states. Args: should_unsync: Whether to perform unsync Nz&The Metric has already been un-synced.z5The internal cache should exist to unsync the Metric.F)rQrrRrrrb)rSrrnrwr#r#r$unsyncs  z Metric.unsyncccs0|j||||ddV|j|jo|ddS)a1Context manager to synchronize the states between processes when running in a distributed setting and restore the local cache states after yielding. Args: dist_sync_fn: Function to be used to perform states synchronization process_group: Specify the process group on which synchronization is called. default: `None` (which selects the entire world) should_sync: Whether to apply to state synchronization. This will have an impact only when running in a distributed setting. should_unsync: Whether to restore the cache state so that the metrics can continue to be accumulated. distributed_available: Function to determine if we are running inside a distributed setting )r1r0rrN)r)rrrQ)rSr1r0rrrr#r#r$ sync_contextszMetric.sync_contextrFcs*tdtdtdtffdd }|S)Nrdr,rcsjdkrtdjjdtjdurjSjjjj d|i|}t |_WdjS1s.wrapped_funcr)rSrFrr#rr$rEszMetric._wrap_compute___cOdS)zHOverride this method to update the state variables of your metric class.Nr#)rSrrr#r#r$rBz Metric.updatecCr)zOverride this method to compute the final metric value from state variables synchronized across the distributed backend.Nr#rUr#r#r$rFrzMetric.computecCstd|_d|_d|_|jD]#\}}t||}t|tr+t||| |j qt||gqd|_ d|_dS)zSThis method automatically resets the metric state variables to their default value.rNF)rIrHrGrNrrrkr<rrbdetachclonerr(rRrQ)rSrnrXrr#r#r$rq$s   z Metric.resetcCst|S)zMake a copy of the metric.rrUr#r#r$r5sz Metric.clonecCsdd|jDS)NcSsi|] \}}|dvr||qS))rBrFrCr#rmkvr#r#r$ro;sz'Metric.__getstate__..)__dict__rrrUr#r#r$ __getstate__9szMetric.__getstate__statecCs:|j|t|j|_||j|_||j|_dSr)rrBr@rArCrDrErF)rSrr#r#r$ __setstate__=s zMetric.__setstate__rcs*|dvr td|dt||dS)N)r*r)r+zCan't change const `z`.)rr4 __setattr__)rSrWrrTr#r$rDszMetric.__setattr__ torch.devicecCs|jS)z Return the device of the metric.)r:rUr#r#r$r(Isz Metric.devicedst_typecC|SzrMethod override default and prevent dtype casting. Please use `metric.set_dtype(dtype)` instead. r#)rSrr#r#r$typeNz Metric.typecCrrr#rUr#r#r$r~Urz Metric.floatcCrrr#rUr#r#r$double\rz Metric.doublecCrrr#rUr#r#r$halfcrz Metric.halfcsd|_t|}d|_|S)zSpecial version of `type` for transferring all metric states to specific dtype Arguments: dst_type (type or string): the desired type TF)rMr4r)rSroutrTr#r$ set_dtypejs zMetric.set_dtypefncs0t}ttfdddD}|js|r|S|jD]N\}}t|tr2||j|<nt|t rCfdd|D|j|<t ||}t|trVt |||q!t|t rit ||fdd|Dq!t d|t jd|jd j|_|jd urt|jt|_|jd urt|jt|_|S) aEOverwrite _apply function such that we can also move metric states to the correct device. This method is called by the base ``nn.Module`` class whenever `.to`, `.cuda`, `.float`, `.half` etc. methods are called. Dtype conversion is garded and will only happen through the special `set_dtype` method. c3s|]}|vVqdSrr#)rmf)fsr#r$ |sz Metric._apply..)z Module.typez Module.halfz Module.floatz Module.doublezModule.bfloat16cg|]}|qSr#r#rmrrr#r$rz!Metric._apply..crr#r#rrr#r$rrzPExpected metric state to be either a Tensoror a list of Tensor, but encountered rx)r(N)r4_applyranyrMrNrrr<rr rkrbrrzerosr(r:rGrrH)rSrthiscondrrrrT)rrr$rts0         z Metric._applymodecCs|jD]}||j|<qdS)zRMethod for post-init to change if metric states should be saved to its state_dict.N)rO)rSrrr#r#r$rZs  zMetric.persistent destinationprefix keep_varscsxtj|||d}|jD]-}|j|sq t||}|s1t|tr%|}n t|tr1dd|D}t ||||<q |S)N)rrrcSs"g|] }t|tr |n|qSr#)r<rrrr#r#r$rs"z%Metric.state_dict..) r4 state_dictrNrOrkr<rrr`r)rSrrrrrrTr#r$rs      zMetric.state_dictrlocal_metadatastrict missing_keysunexpected_keys error_msgsc sJ|jD]}||} | |vrt|||| qt|||d|||dS)z$Loads metric states from state_dict.TN)rNrbr;r4_load_from_state_dict) rSrrrrrrrrrWrTr#r$rs zMetric._load_from_state_dictc sbtjjtjjf|jjfdd|D}tddD}|s+|s+i}|S|r/|}|S)zFfilter kwargs such that they match the update signature of the metric.cs0i|]\}}|vr|jvr||qSr#)rykindr_params _sign_paramsr#r$ros*z)Metric._filter_kwargs..css|] }|jtjjkVqdSr)rr@ Parameter VAR_KEYWORDrr#r#r$rsz(Metric._filter_kwargs..) r@rVAR_POSITIONALrrC parametersrrrvalues)rSr,filtered_kwargsexists_var_keywordr#rr$_filter_kwargss zMetric._filter_kwargscCs\|jjt|g}|jD]}t||}t|dr"t|ts"||q | |q t t |S)N__iter__) r8r9idrNrkhasattrr<rextendappendhashtuple)rS hash_valsrrwr#r#r$__hash__s     zMetric.__hash__othercCttj||SrCompositionalMetricraddrSrr#r#r$__add__zMetric.__add__cCrrrr bitwise_andrr#r#r$__and__rzMetric.__and__cCrr)rreqrr#r#r$__eq__rz Metric.__eq__cCrrrr floor_dividerr#r#r$ __floordiv__rzMetric.__floordiv__cCrr)rrgerr#r#r$__ge__rz Metric.__ge__cCrr)rrgtrr#r#r$__gt__rz Metric.__gt__cCrr)rrlerr#r#r$__le__rz Metric.__le__cCrr)rrltrr#r#r$__lt__ rz Metric.__lt__cCrrrrmatmulrr#r#r$ __matmul__ rzMetric.__matmul__cCrrrrfmodrr#r#r$__mod__rzMetric.__mod__cCrrrrmulrr#r#r$__mul__rzMetric.__mul__cCrr)rrnerr#r#r$__ne__rz Metric.__ne__cCrrrr bitwise_orrr#r#r$__or__rz Metric.__or__cCrrrrpowrr#r#r$__pow__rzMetric.__pow__cCttj||Srrrr#r#r$__radd__rzMetric.__radd__cCrrrrr#r#r$__rand__"szMetric.__rand__cCrrrrr#r#r$ __rfloordiv__&rzMetric.__rfloordiv__cCrrrrr#r#r$ __rmatmul__)rzMetric.__rmatmul__cCrrrrr#r#r$__rmod__,rzMetric.__rmod__cCrrrrr#r#r$__rmul__/rzMetric.__rmul__cCrrr rr#r#r$__ror__2rzMetric.__ror__cCrrrrr#r#r$__rpow__5rzMetric.__rpow__cCrrrrsubrr#r#r$__rsub__8rzMetric.__rsub__cCrrrr true_dividerr#r#r$ __rtruediv__;rzMetric.__rtruediv__cCrrrr bitwise_xorrr#r#r$__rxor__>rzMetric.__rxor__cCrrrrr#r#r$__sub__ArzMetric.__sub__cCrrrrr#r#r$ __truediv__DrzMetric.__truediv__cCrrr!rr#r#r$__xor__GrzMetric.__xor__cCttj|dSrrrabsrUr#r#r$__abs__JrzMetric.__abs__cCr'r)rr bitwise_notrUr#r#r$__inv__MrzMetric.__inv__cCs|Sr)r,rUr#r#r$ __invert__PzMetric.__invert__cCs tt|dSr)r_negrUr#r#r$__neg__S zMetric.__neg__cCr'rr(rUr#r#r$__pos__VrzMetric.__pos__idxcstfdd|dS)Ncs|Srr#xr3r#r$Zsz$Metric.__getitem__..)r)rSr3r#r6r$ __getitem__Yr&zMetric.__getitem__cCs t|fSr)r'__str__rUr#r#r$__getnewargs__\r1zMetric.__getnewargs__cCstd)Nz#Metrics does not support iteration.)NotImplementedErrorrUr#r#r$r_r.zMetric.__iter__)NFrN)NNTN)T)NNTTN)rr')rrF)NrF)rr'rr')gr9 __module__ __qualname____doc____jit_ignored_attributes____jit_unused_properties__r)r r=__annotations__r*r+rr5propertyrVrrr`rrrcrjitunusedrhrerfr r{rrrDrsrrrr rrErrBrFrqrrrrr(dtyperr~rrrrrrZrdictr rrintrrrrrrrrrrrr r rrrrrrrrrrrr r#r$r%r&r*r,r-r0r2r8rr:r __classcell__r#r#rTr$r'+s8  =  F+'    ' !        )                                   r'r5cCs t| Sr)rr)r4r#r#r$r/cr1r/c seZdZdZdedeeeee fdeeeee dfddffdd Z d d e ed e e ddfd d Z d e de ddfddZde fddZejjd e de de fddZd!ddZd"deddfddZdefddZdedefddZZS)#rz`Composition of two metrics with a specific operator which will be executed upon metrics compute.operatormetric_ametric_bNrcsRt||_t|tr|d|n||_t|tr$|d|dS||_dS)a Args: operator: the operator taking in one (if metric_b is None) or two arguments. Will be applied to outputs of metric_a.compute() and (optionally if metric_b is not None) metric_b.compute() metric_a: first metric whose compute() result is the first argument of operator metric_b: second metric whose compute() result is the second argument of operator. For operators taking in only one input, this should be None rLrMN)r4r5opr<rregister_bufferrLrM)rSrKrLrMrTr#r$r5js    zCompositionalMetric.__init__r1r0cCsdSrr#)rSr1r0r#r#r$rszCompositionalMetric._sync_distrdr,cOs`t|jtr|jj|i|jjdi|t|jtr.|jj|i|jjdi|dSdSNr#)r<rLr'rBrrMrgr#r#r$rBs   $zCompositionalMetric.updatecCsZt|jtr |j}n|j}t|jtr|j}n|j}|dur'||S|||Sr)r<rLr'rFrMrN)rSval_aval_br#r#r$rFs      zCompositionalMetric.computecOst|jtr|j|i|jjdi|n|j}t|jtr-|j|i|jjdi|n|j}|dur6dS|durGt|jtrBdS||S|||SrP)r<rLr'rrMrN)rSrdr,rQrRr#r#r$rhs       zCompositionalMetric.forwardcCs4t|jtr |jt|jtr|jdSdSr)r<rLr'rqrMrUr#r#r$rqs   zCompositionalMetric.resetFrcCs<t|jtr |jj|dt|jtr|jj|ddSdS)N)r)r<rLr'rZrM)rSrr#r#r$rZs  zCompositionalMetric.persistentcCs8d|jjdt|jdt|jd}|jj|}|S)Nz( z( z, z ) ))rNr9reprrLrMr8)rS _op_metricsrepr_strr#r#r$__repr__s( zCompositionalMetric.__repr__rFcCs|Srr#)rSrFr#r#r$rEsz!CompositionalMetric._wrap_compute)NNr<r=)r9r>r?r@rrr'rIr~rr5r rrrBrFrrErFrhrqr=rZrrVrErJr#r#rTr$rgs(  r)+rr@abcrr contextlibrcopyrtypingrrr r r r r rrrrtorch.nnrtorchmetrics.utilitiesrrtorchmetrics.utilities.datarrrrrrr"torchmetrics.utilities.distributedr!torchmetrics.utilities.exceptionsrr=r%r'r/rr#r#r#r$s.   ,  $ >