C Rosset. 2019. Turing-NLG: A 17-billion-parameter
language model by Microsoft. Microsoft Blog.

S. Russell. 2019. Human Compatible: Artiﬁcial Intelli-
gence and the Problem of Control. Penguin Publishing
Group.

David Saxton, Edward Grefenstette, Felix Hill, and
Pushmeet Kohli. 2019. Analysing mathematical rea-
arXiv preprint
soning abilities of neural models.
arXiv:1904.01557.

Ismaïla Seck, Khouloud Dahmane, Pierre Duthon,
and Gaëlle Loosli. 2018. Baselines and a datasheet
arXiv preprint
for
arXiv:1806.04016.

the Cerema AWP dataset.

Mohammad Shoeybi, Mostofa Patwary, Raul Puri,
Patrick LeGresley, Jared Casper, and Bryan Catanzaro.
2019. Megatron-LM: Training multi-billion parameter
language models using gpu model parallelism. arXiv
preprint arXiv:1909.08053.

Merity Stephen, Xiong Caiming, Bradbury James, and
Richard Socher. 2016.
The Pile- An 800GB Dataset of Diverse Text for Language Modeling