  Recent applications of neural language models have led to an increased
interest in the automatic generation of natural language. However impressive,
the evaluation of neurally generated text has so far remained rather informal
and anecdotal. Here, we present an attempt at the systematic assessment of one
aspect of the quality of neurally generated text. We focus on a specific aspect
of neural language generation: its ability to reproduce authorial writing
styles. Using established models for authorship attribution, we empirically
assess the stylistic qualities of neurally generated text. In comparison to
conventional language models, neural models generate fuzzier text that is
relatively harder to attribute correctly. Nevertheless, our results also
suggest that neurally generated text offers more valuable perspectives for the
augmentation of training data.

Computation and Language