o 5tir@sdZddlZddlZddlZddlmZmZmZddlmZm Z m Z ddl m Z m Z ddlmZddlmZeeZeGd d d ZeGd d d eZeGd ddeZeGdddeZdS)zP EvaluationModuleInfo records information we know about a dataset and a metric. N)asdict dataclassfield)ListOptionalUnion)FeaturesValue)config) get_loggerc@s$eZdZUdZeed<eed<eeeefed<e edZ eed<e edZ eed<e edZ eed<e e dZeeed <e e dZeeed <d Zeed <d Zeeed<dZeed<d Zeeed<d Zeeed<d Zeeed<ddZddZedddZededdfddZd S)EvaluationModuleInfoalBase class to store information about an evaluation used for `MetricInfo`, `ComparisonInfo`, and `MeasurementInfo`. `EvaluationModuleInfo` documents an evaluation, including its name, version, and features. See the constructor arguments and properties for a full list. Note: Not all fields are known on construction and may be updated later. descriptioncitationfeatures)default_factoryinputs_descriptionhomepagelicense codebase_urlsreference_urlsF streamableNformatmetric module_type module_name config_name experiment_idcCsH|jdur |jD]\}}t|tstd|d|jjq dSdS)NzSWhen using 'numpy' format, all features should be a `datasets.Value` feature. Here z is an instance of )rritems isinstancer ValueError __class____name__)selfkeyvaluer&A/home/ubuntu/.local/lib/python3.10/site-packages/evaluate/info.py __post_init__?s  z"EvaluationModuleInfo.__post_init__cCsttj|tjddd}tt||Wdn1s wYttj|tj ddd}| |j WddS1sDwYdS)aZWrite `EvaluationModuleInfo` as JSON to `metric_info_dir`. Also save the license separately in LICENSE. Args: metric_info_dir (`str`): The directory to save `metric_info_dir` to. Example: ```py >>> my_metric.info.write_to_directory("/path/to/directory/") ``` wutf-8encodingN) openospathjoinr METRIC_INFO_FILENAMEjsondumprLICENSE_FILENAMEwriter)r#metric_info_dirfr&r&r'write_to_directoryHs "z'EvaluationModuleInfo.write_to_directoryreturncCshtd||stdttj|tjdd }t |}Wdn1s*wY| |S)aCreate `EvaluationModuleInfo` from the JSON file in `metric_info_dir`. Args: metric_info_dir (`str`): The directory containing the `metric_info` JSON file. This should be the root directory of a specific metric version. Example: ```py >>> my_metric = EvaluationModuleInfo.from_directory("/path/to/directory/") ``` zLoading Metric info from zMCalling EvaluationModuleInfo.from_directory() with undefined metric_info_dir.r*r+N) loggerinfor r-r.r/r0r r1r2load from_dict)clsr6r7metric_info_dictr&r&r'from_directory\s  z#EvaluationModuleInfo.from_directoryr?cs4ddt|D|difdd|DS)NcSsh|]}|jqSr&)name).0r7r&r&r' usz1EvaluationModuleInfo.from_dict..csi|] \}}|vr||qSr&r&)rBkv field_namesr&r' vsz2EvaluationModuleInfo.from_dict..r&) dataclassesfieldsr)r>r?r&rFr'r=ss zEvaluationModuleInfo.from_dict)r9r )r" __module__ __qualname____doc__str__annotations__rrrrrrrlistrrrboolrrrrrrr(r8 classmethodr@dictr=r&r&r&r'r "s,      r c@eZdZUdZdZeed<dS) MetricInfoa Information about a metric. `EvaluationModuleInfo` documents a metric, including its name, version, and features. See the constructor arguments and properties for a full list. Note: Not all fields are known on construction and may be updated later. rrNr"rKrLrMrrNrOr&r&r&r'rUy rUc@rT)ComparisonInfoaInformation about a comparison. `EvaluationModuleInfo` documents a comparison, including its name, version, and features. See the constructor arguments and properties for a full list. Note: Not all fields are known on construction and may be updated later. comparisonrNrVr&r&r&r'rXrWrXc@rT)MeasurementInfoaInformation about a measurement. `EvaluationModuleInfo` documents a measurement, including its name, version, and features. See the constructor arguments and properties for a full list. Note: Not all fields are known on construction and may be updated later. measurementrNrVr&r&r&r'rZrWrZ)rMrIr2r.rrrtypingrrrdatasets.featuresrr r utils.loggingr r"r:r rUrXrZr&r&r&r's$  V