o Ni2@sdZddlmZddlmZddlmZddlZddlmmZ ddl m Z ddl m mZddlmZdZdZd Zd d d d ddddddddddddZddZGdddejjZGdddejjZdS) z>QA4MRE (CLEF 2011/2012/2013): a reading comprehension dataset.)absolute_import)division)print_functionN)loggingaN @InProceedings{10.1007/978-3-642-40802-1_29, author="Pe{\~{n}}as, Anselmo and Hovy, Eduard and Forner, Pamela and Rodrigo, {\'A}lvaro and Sutcliffe, Richard and Morante, Roser", editor="Forner, Pamela and M{\"u}ller, Henning and Paredes, Roberto and Rosso, Paolo and Stein, Benno", title="QA4MRE 2011-2013: Overview of Question Answering for Machine Reading Evaluation", booktitle="Information Access Evaluation. Multilinguality, Multimodality, and Visualization", year="2013", publisher="Springer Berlin Heidelberg", address="Berlin, Heidelberg", pages="303--320", abstract="This paper describes the methodology for testing the performance of Machine Reading systems through Question Answering and Reading Comprehension Tests. This was the attempt of the QA4MRE challenge which was run as a Lab at CLEF 2011--2013. The traditional QA task was replaced by a new Machine Reading task, whose intention was to ask questions that required a deep knowledge of individual short texts and in which systems were required to choose one answer, by analysing the corresponding test document in conjunction with background text collections provided by the organization. Four different tasks have been organized during these years: Main Task, Processing Modality and Negation for Machine Reading, Machine Reading of Biomedical Texts about Alzheimer's disease, and Entrance Exams. This paper describes their motivation, their goals, their methodology for preparing the data sets, their background collections, their metrics used for the evaluation, and the lessons learned along these three years.", isbn="978-3-642-40802-1" } a QA4MRE dataset was created for the CLEF 2011/2012/2013 shared tasks to promote research in question answering and reading comprehension. The dataset contains a supporting passage and a set of questions corresponding to the passage. Multiple options for answers are provided for each question, of which only one is correct. The training and test datasets are available for the main track. Additional gold standard documents are available for two pilot studies: one on alzheimers data, and the other on entrance exams data. z}http://nlp.uned.es/clef-qa/repository/js/scripts/downloadFile.php?file=/var/www/html/nlp/clef-qa/repository/resources/QA4MRE/mainz52011/Training_Data/Goldstandard/QA4MRE-2011-{}_GS.xml)DEENESITRO)_TRACKS_PATH_TMPL_MAIN_GS_LANGUAGES_MAIN)r alzheimerszR2012/Main_Task/Training_Data/Goldstandard/Used_in_Evaluation/QA4MRE-2012-{}_GS.xml)ARBGrrr r r zd2012/Pilot_Tasks/Biomedical_About_Alzheimer/Training_Data/Goldstandard/QA4MRE-2012_BIOMEDICAL_GS.xml)r r r_PATH_ALZHEIMER)rr entrance_examz?2013/Main_Task/Training_Data/Goldstandard/QA4MRE-2013-{}_GS.xml)rrrr r zU2013/Biomedical_About_Alzheimer/Training_Data/Goldstandard/QA4MRE-2013_BIO_GS-RUN.xmlzN2013/Entrance_Exams/Training_Data/Goldstandard/qa4mre-exam-test-withanswer.xml)r r rr_PATH_ENTRANCE_EXAM)201120122013c Cs|jd}|dD]}|j}q t} |dD]} | jd} | j} | | | dd| jvr3| } | }qd||||g}td||||||||| | |d }||fS) a/Gets instance ID and features for every question. Args: topic_id: string topic_name: string test_id: string document_id: string document_str: string question: XML element for question Returns: id_: string. Unique ID for instance. feats: dict of instance features q_idq_stranswera_id answer_id answer_strcorrect_zID: %s topic_id topic_nametest_id document_id document_str question_id question_stranswer_optionscorrect_answer_idcorrect_answer_str)attribitertextlistappendjoinrinfo)r"r#r$r%r&questionr'q_textr(possible_answersrrrr*r+id_featsr8S/home/ubuntu/.local/lib/python3.10/site-packages/tensorflow_datasets/text/qa4mre.py _get_question_s4     r:cs*eZdZdZejjdfdd ZZS) Qa4mreConfigzBuilderConfig for Qa4mre.rrc s|t|dvrtdt|d|dkr&|dkr&tdd}|dkr?|t|dvr?tdt|d||_||_||_|jd|jd|j}t }|d |j|j|j7}t t |j d ||tjd d |d S)aBuilderConfig for Qa4Mre. Args: year: string, year of dataset track: string, the task track from PATHS[year]['_TRACKS']. language: string, Acronym for language in the main task. **kwargs: keyword arguments forwarded to super. r z7Incorrect track. Track should be one of the following: rrzNOnly English documents available for pilot tracks. Setting English by default.rz8Incorrect language for the main track. Correct options: .zDThis configuration includes the {} track for {} language in {} year.z0.1.0)name descriptionversionNr8)lowerPATHS ValueErrorupperrwarnyeartracklang _DESCRIPTIONformatsuperr;__init__tfdscoreVersion)selfrErFlanguagekwargsr=r> __class__r8r9rKs<          zQa4mreConfig.__init__)rr) __name__ __module__ __qualname____doc__rLrMdisallow_positional_argsrK __classcell__r8r8rRr9r;sr;c@seZdZdZeddddeddddeddddeddddeddd ded dd ded dd ded ddded ddded ddded ddded dd ded d ddeddd deddd deddddeddddeddd dedd ddeddddgZddZddZddZdS)Qa4mrez7QA4MRE dataset from CLEF shared tasks 2011, 2012, 2013.rrr)rErFrPrr r r rrrrrrcCstjj|ttjtjtjtjtjtjtjtjtjtjtjdtjtjd ddtdS)Nrr!z7http://nlp.uned.es/clef-qa/repository/pastCampaigns.php)builderr>featuressupervised_keyshomepagecitation) rLrM DatasetInforHr\ FeaturesDictTextSequence _CITATION)rOr8r8r9_infos,z Qa4mre._infoc Cs|j}t}|jdkr#tjtt|jd |j |d |j|j <|jdvr?|jdkr?tjtt|jd|d |j<|jdkrW|jd krWtjtt|jd |d <| |}t j jt jjd |d |j|j|j idgS)zReturns SplitGenerators.rr z {}.main.{})rrrrz{}.alzheimers.ENrrrz2013.entrance_exam.ENfilepathz{}.{}.{})r= gen_kwargs)builder_configdictrFospathr1 _BASE_URLrArErIrGdownload_and_extractrLrMSplitGeneratorSplitTRAIN)rO dl_managercfg download_urlsdownloaded_filesr8r8r9_split_generatorss0  zQa4mre._split_generatorsc cstjj|dM}t|}|}|D]8}|jd}|jd}|D])}|jd} |dD] } | jd} | j } q/|dD] } t ||| | | | Vq?q#qWddS1sYwYdS) zYields examples.rbt_idt_namer_iddocd_idqN) tfiogfileGFileETparsegetrootr,r-r.r:)rOrfftreeroottopicr"r#testr$documentr%r&r3r8r8r9_generate_examples/s(      "zQa4mre._generate_examplesN) rTrUrVrWr;BUILDER_CONFIGSrerurr8r8r8r9rZs+' rZ)rW __future__rrrrjxml.etree.ElementTreeetree ElementTreerabslrtensorflow.compat.v2compatv2r}tensorflow_datasets.public_api public_apirLrdrHrlrAr:rM BuilderConfigr;GeneratorBasedBuilderrZr8r8r8r9s>       0.