 Training procedure  Model was trained on a single TPUv3 from the official repo with the default parameters.  | Parameter                                        | Value | | ------------------------------------------------ | ----: | | Batch Size                                       | 256   | | Max Sequence Size                                | 512   | | Vocab Size                                       | 30000 |   *Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC)*  