The current book corpus dataset is parsed into sentences directly, which is great, but then there is no way to determine document boundaries. Would it be useful to have another bookcorpus dataset that is chunked into books rather than sentences directly?
Shawn Presser went to great lenghts to preserve the structure of the books’ text and it is available at https://github.com/soskek/bookcorpus/issues/27 25 for download.