(Liu et al., 2019), and about 33% of that of the world-record, in GPU hours, required to train BERT-large on the same
hardware. It is also 7.9x faster on a CPU, as well as being better performing than other compressed variants of the
architecture, and some of the non-compressed variants: it obtains performance improvements of between 0.3% and 31%,
absolute, with respect to BERT-large, on multiple public natural language understanding (NLU) benchmarks.
Tips:
BORT's model architecture is based on BERT, so one can refer to BERT's documentation page for the
  model's API as well as usage examples.
BORT uses the RoBERTa tokenizer instead of the BERT tokenizer, so one can refer to RoBERTa's documentation page for the tokenizer's API as well as usage examples.
BORT requires a specific fine-tuning algorithm, called Agora ,
