o 5tii@sdZddlmZddlZddlmZddlmZmZddl m Z ddl m Z e e ZGd d d eZdd dZdddZ ddededeefddZdS)z List and inspect metrics.)OptionalN)DownloadConfig)EVALUATION_MODULE_TYPESHF_LIST_ENDPOINT)evaluation_module_factory) get_loggerc@s eZdZdS)SplitsNotFoundErrorN)__name__ __module__ __qualname__r r D/home/ubuntu/.local/lib/python3.10/site-packages/evaluate/inspect.pyr sr TFcCs^|durg}tD] }|t|||dq|S|tvr&td|dtdt|||d}|S)aList all evaluation modules available on the Hugging Face Hub. Args: module_type (`str`, *optional*, defaults to `None`): Type of evaluation modules to list. Has to be one of `'metric'`, `'comparison'`, or `'measurement'`. If `None`, all types are listed. include_community (`bool`, *optional*, defaults to `True`): Include community modules in the list. with_details (`bool`, *optional*, defaults to `False`): Return the full details on the metrics instead of only the ID. Returns: `List[Union[str, dict]]` Example: ```py >>> from evaluate import list_evaluation_modules >>> list_evaluation_modules(module_type="metric") ``` N)include_community with_detailszInvalid module type 'z'. Has to be one of .)rextend_list_evaluation_modules_type ValueError) module_typerrevaluations_listr r rlist_evaluation_modules#s rcsttjd}||}|sfdd|D}|D]$}|ddddkr>|ddd|d<d |d <qd |d <q|rNfd d|DSd d|DS)N)typecs,g|]}|ddddkr|qS)id/r evaluate-)split.0elementrr r Qs,z1_list_evaluation_modules_type..rrrrrF communityTc s*g|]}|d|d|dddqS)rr"likesr)namerr"r#)getrr r rr!\s cSsg|]}|dqS)rr rr r rr!fs)requestsr%rformatraise_for_statusjsonr)rrrrdrr r rrJs    rpath local_pathdownload_configc Ks>t|f||d|}td|d|d|jd|d dS)a Allow inspection/modification of a evaluation script by copying it on local drive at local_path. Args: path (``str``): path to the evaluation script. Can be either: - a local path to script or the directory containing the script (if the script has the same name as the directory), e.g. ``'./metrics/accuracy'`` or ``'./metrics/accuracy/accuracy.py'`` - a dataset identifier on the Hugging Face Hub (list all available datasets and ids with ``evaluate.list_evaluation_modules()``) e.g. ``'accuracy'``, ``'bleu'`` or ``'word_length'`` local_path (``str``): path to the local folder to copy the datset script to. download_config (Optional ``datasets.DownloadConfig``: specific download configuration parameters. **download_kwargs: optional attributes for DownloadConfig() which will override the attributes in download_config if supplied. )r.force_local_pathz"The processing scripts for metric z can be inspected at z. The main class is in zH. You can modify this processing scripts and use it with `evaluate.load(z)`.N)rprint module_path)r,r-r.download_kwargsevaluation_moduler r rinspect_evaluation_moduleisr4)NTF)TF)N)__doc__typingrr&datasetsrconfigrrloadingr utils.loggingrr loggerrr rrstrr4r r r rs&      '