Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Kristina Toutanova. 2019. BERT: Pre-training of deep
bidirectional transformers for language understanding.
In Proceedings of the 2019 Conference of the North
American Chapter of the Association for Computa-
tional Linguistics: Human Language Technologies,
Volume 1 (Long and Short Papers). Association for
Computational Linguistics.

17

István Endrédy and Attila Novák. 2013. More effec-
tive boilerplate removal – the GoldMiner algorithm. In
Polibits.

Niels Ferguson and Bruce Schneier. 2003. Practical
Cryptography. John Wiley & Sons.

Casey Fiesler, Nathan Beard, and Brian C Keegan.
2020. No robots, spiders, or scrapers: Legal and ethical
regulation of data collection methods in social media
terms of service. In Proceedings of the International
AAAI Conference on Web and Social Media, volume 14,
pages 187–196.
The Pile- An 800GB Dataset of Diverse Text for Language Modeling