o NiD@srdZddlmZddlmZddlmZddlZddlmmZ ddl m Z dZ dZdZGd d d e jjZdS) z PG-19 language modeling dataset.)absolute_import)division)print_functionNa7 @article{raecompressive2019, author = {Rae, Jack W and Potapenko, Anna and Jayakumar, Siddhant M and Hillier, Chloe and Lillicrap, Timothy P}, title = {Compressive Transformers for Long-Range Sequence Modelling}, journal = {arXiv preprint}, url = {https://arxiv.org/abs/1911.05507}, year = {2019}, } aj This dataset contains the PG-19 language modeling benchmark. It includes a set of books extracted from the Project Gutenberg books project (https://www.gutenberg.org), that were published before 1919. It also contains metadata of book titles and publication dates. PG-19 is over double the size of the Billion Word benchmark and contains documents that are 20X longer, on average, than the WikiText long-range language modelling benchmark. Books are partitioned into a train, validation, and test set. Books metadata is stored in metadata.csv which contains (book_id, short_book_title, publication_date, book_link). zgs://deepmind-gutenbergc@s4eZdZdZejdZddZddZ ddZ d S) Pg19zs