I just can&#x27;t agree that a simple, linear-time operation like &quot;tokenizing words and basic n-gram models out of them&quot; is a tedious problem like you seem to be implying, nor do I feel a solution to this very-solved problem is &quot;compelling&quot;. Word tokenization and n-gram models are simple, unreasonably effective, and very fast. If character-based RNNs do better (albeit far more slowly during training), great, but nothing to see here, let&#x27;s move along.<p>As I&#x27;ve posted here before, people have been training character n-gram models and getting language modeling performances comparable to those from word-based models---without using neural networks---for at least a decade. That it works with RNNs is no surprise because it worked just fine with the much more constrained predecessor technology.