o Tij@sddlmZmZddlmZddlmZddlmZddl m Z eGdddZ dd Z ed Z ed krbe e ZedZed d Zeje ejgddZejrdeeje jejdddSdSdS)) dataclassfield)Optional) load_dataset) ModelCard)HfArgumentParserc@s^eZdZUdZedddidZeed<edddidZe ed <ed dd idZ e e ed <d S) ScriptArgumentsa Arguments for the script. Args: push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the dataset to the Hugging Face Hub. repo_id (`str`, *optional*, defaults to `"trl-lib/tldr-preference"`): Hugging Face repository ID to push the dataset to. dataset_num_proc (`int` or `None`, *optional*, defaults to `None`): Number of workers to use for dataset processing. Fhelpz4Whether to push the dataset to the Hugging Face Hub.)defaultmetadata push_to_hubztrl-lib/tldr-preferencez2Hugging Face repository ID to push the dataset to.repo_idNz0Number of workers to use for dataset processing.dataset_num_proc) __name__ __module__ __qualname____doc__rr bool__annotations__r strrrintrrU/home/ubuntu/.local/lib/python3.10/site-packages/examples/datasets/tldr_preference.pyrs  rc Cs|d}|ddvr|ddd}d|dd|d }n3|dd d td d DdgvrH|ddd}d|dd|dd|d }n td|d|d}d|}|d|d}|d|d}|||dS)Ninfobatch) batch0_cnndmcnndm0cnndm2articlez  zTITLE: titlez TL;DR:cSsg|]}d|qS)rr).0irrr 8sz!to_preference..edit_b2_eval_testpostz SUBREDDIT: r/ subredditz TITLE: z POST: zUnknown batch: choice summariestext)promptchosenrejected)replacerange ValueError) examplerrr-r' chosen_idx rejected_idxr.r/rrr to_preference3s "" r6a --- tags: [trl] --- # TL;DR Dataset for Preference Learning ## Summary The TL;DR dataset is a processed version of Reddit posts, specifically curated to train models using the [TRL library](https://github.com/huggingface/trl) for preference learning and Reinforcement Learning from Human Feedback (RLHF) tasks. It leverages the common practice on Reddit where users append "TL;DR" (Too Long; Didn't Read) summaries to lengthy posts, providing a rich source of paired text data for training models to understand and generate concise summaries. ## Data Structure - **Format**: [Standard](https://huggingface.co/docs/trl/main/dataset_formats#standard) - **Type**: [Preference](https://huggingface.co/docs/trl/main/dataset_formats#preference) Columns: - `"prompt"`: The unabridged Reddit post. - `"chosen"`: The concise "TL;DR" summary appended by the author. - `"rejected"`: An alternative summary or response that was not selected. This structure enables models to learn the relationship between detailed content and its abbreviated form, enhancing their summarization capabilities. ## Generation script The script used to generate this dataset can be found [here](https://github.com/huggingface/trl/blob/main/examples/datasets/tldr_preference.py). __main__zopenai/summarize_from_feedback comparisons)rr+r)workerrsplitextra)num_procremove_columnsdataset) repo_typeN) dataclassesrrtypingrdatasetsrhuggingface_hubr transformersrrr6 model_cardrparserparse_args_into_dataclasses script_argsr>maprr r rrrrs.