o 5ti @s>ddlmZmZGdddeZd ddZGdddeZdS) )Datasetget_dataset_split_namesc@s:eZdZdZdedefddZddZdd Zd d Z d S) DatasetColumnzMHelper class to avoid loading a dataset column into memory when accessing it.datasetkeycCs||_||_dSNrr)selfrrr L/home/ubuntu/.local/lib/python3.10/site-packages/evaluate/evaluator/utils.py__init__s zDatasetColumn.__init__cC t|jSrlenrr r r r __len__  zDatasetColumn.__len__cCs|j||jSrrr ir r r __getitem__szDatasetColumn.__getitem__cfddttDS)Nc3s |] }j|jVqdSrr.0rrr r sz)DatasetColumn.__iter__..rangerrr rr __iter__szDatasetColumn.__iter__N __name__ __module__ __qualname____doc__rstrr rrrr r r r rs  rNcCs4t||}gd}|D] }||vr|Sq td)N) testtestingeval evaluation validationvalvaliddevtraintrainingzFNo dataset split defined! Pass an explicit value to the `split` kwarg.)r ValueError)datasubsetavailable_splitspreferred_split_ordersplitr r r choose_splits  r3c @sFeZdZdZdededededef ddZd d Zd d Zd dZ dS)DatasetColumnPairzPHelper class to avoid loading two dataset columns into memory when accessing it.r first_col second_col first_key second_keycCs"||_||_||_||_||_dS)a Args: dataset (Dataset): dataset to build an iterator on first_col (str): first column name to use in the dataset second_col (str): second column name to use in the dataset first_key (str): key name used for the first column in the returned dictionary second_key (str): key name used for the second column in the returned dictionary N)rr5r6r7r8)r rr5r6r7r8r r r r ,s  zDatasetColumnPair.__init__cCr rrrr r r rDrzDatasetColumnPair.__len__cCs4|j|j||j|j|jr|j||jiSdiSrr7rr5r8r6rr r r rGs zDatasetColumnPair.__getitem__cr)Nc3sB|]}jj|jjjrj|jndiVqdSrr9rrr r rNs z-DatasetColumnPair.__iter__..rrr rr rMs  zDatasetColumnPair.__iter__Nrr r r r r4)s   r4r)datasetsrrlistrr3r4r r r r s