o Ti"@sddlmZmZddlmZddlmZmZddlm Z ddl m Z eGdddZ dd Z e d Zed kre e ZedZed d dZeje ejejddZejZeejdded<eeZejdddZejreejejejdddSdSdS)) dataclassfield)Optional)features load_dataset) ModelCard)HfArgumentParserc@s^eZdZUdZedddidZeed<edddidZe ed <ed dd idZ e e ed <d S) ScriptArgumentsa Arguments for the script. Args: push_to_hub (`bool`, *optional*, defaults to `False`): Whether to push the dataset to the Hugging Face Hub. repo_id (`str`, *optional*, defaults to `"trl-lib/rlaif-v"`): Hugging Face repository ID to push the dataset to. dataset_num_proc (`int` or `None`, *optional*, defaults to `None`): Number of workers to use for dataset processing. Fhelpz4Whether to push the dataset to the Hugging Face Hub.)defaultmetadata push_to_hubztrl-lib/rlaif-vz2Hugging Face repository ID to push the dataset to.repo_idNz0Number of workers to use for dataset processing.dataset_num_proc) __name__ __module__ __qualname____doc__rr bool__annotations__rstrrrintrrM/home/ubuntu/.local/lib/python3.10/site-packages/examples/datasets/rlaif-v.pyr s  r cCsbdddid|ddgdg}dd|d dgdg}dd|d dgdg}||dg||d S) a Convert prompt from "xxx" to [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "xxx"}]}] and chosen and rejected from "xxx" to [{"role": "assistant", "content": [{"type": "text", "text": "xxx"}]}]. Images are wrapped into a list. usertypeimagetextquestion)rr)rolecontent assistantchosenrejected)promptimagesr"r#r)exampler$r"r#rrrto_conversational3sr'a> --- tags: [trl] --- # RLAIF-V Dataset ## Summary The RLAIF-V dataset is a processed version of the [openbmb/RLAIF-V-Dataset](https://huggingface.co/datasets/openbmb/RLAIF-V-Dataset#dataset-card-for-rlaif-v-dataset), specifically curated to train vision-language models using the [TRL library](https://github.com/huggingface/trl) for preference learning tasks. It contains 83,132 high-quality comparison pairs, each comprising an image and two textual descriptions: one preferred and one rejected. This dataset enables models to learn human preferences in visual contexts, enhancing their ability to generate and evaluate image captions. ## Data Structure - **Format**: [Conversational](https://huggingface.co/docs/trl/main/dataset_formats#conversational) - **Type**: [Preference](https://huggingface.co/docs/trl/main/dataset_formats#preference) Columns: - `"prompt"`: The task related to the image. - `"images"`: The image. - `"chosen"`: The preferred answer. - `"rejected"`: An alternative answer that was not preferred. This structure allows models to learn to prefer the _chosen_ response over the _rejected_ one, thereby aligning with human preferences in visual tasks. ## Generation script The script used to generate this dataset can be found [here](https://github.com/huggingface/trl/blob/main/examples/datasets/rlaif-v.py). __main__zopenbmb/RLAIF-V-Datasettrain)split)num_procremove_columnswriter_batch_sizeT)decoder%g{Gz?) test_sizer.dataset) repo_typeN) dataclassesrrtypingrdatasetsrrhuggingface_hubr transformersrr r' model_cardrparserparse_args_into_dataclasses script_argsr1mapr column_namesfSequenceImagecasttrain_test_splitr rrrrrs8