readme: ---
  tags:
    - Multi-exit-BERT
  
  language: en
  
  datasets:
    - wikipedia
    - bookcorpus
    - c4
  ---
  
  # ElasticBERT-LARGE
  
  ## Model description
  
  This is an implementation of the `large` version of ElasticBERT.
  
  [**Towards Efficient NLP: A Standard Evaluation and A Strong Baseline**](https://arxiv.org/pdf/2110.07038.pdf)
  
  Xiangyang Liu, Tianxiang Sun, Junliang He, Lingling Wu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, Xipeng Qiu
  
  ## Code link
  
  [**fastnlp/elasticbert**](https://github.com/fastnlp/ElasticBERT)
  
  ## Usage
  ```python
  >>> from transformers import BertTokenizer as ElasticBertTokenizer
  >>> from models.configuration_elasticbert import ElasticBertConfig
  >>> from models.modeling_elasticbert import ElasticBertForSequenceClassification
  >>> num_output_layers = 1
  >>> config = ElasticBertConfig.from_pretrained('fnlp/elasticbert-large', num_output_layers=num_output_layers )
  >>> tokenizer = ElasticBertTokenizer.from_pretrained('fnlp/elasticbert-large')
  >>> model = ElasticBertForSequenceClassification.from_pretrained('fnlp/elasticbert-large', config=config)
  >>> input_ids = tokenizer.encode('The actors are fantastic .', return_tensors='pt')
  >>> outputs = model(input_ids)
  ```
  
  ## Citation
  
  ```bibtex
  @article{liu2021elasticbert,
    author    = {Xiangyang Liu and
                 Tianxiang Sun and
                 Junliang He and
                 Lingling Wu and
                 Xinyu Zhang and
                 Hao Jiang and
                 Zhao Cao and
                 Xuanjing Huang and
                 Xipeng Qiu},
    title     = {Towards Efficient {NLP:} {A} Standard Evaluation and {A} Strong Baseline},
    journal   = {CoRR},
    volume    = {abs/2110.07038},
    year      = {2021},
    url       = {https://arxiv.org/abs/2110.07038},
    eprinttype = {arXiv},
    eprint    = {2110.07038},
    timestamp = {Fri, 22 Oct 2021 13:33:09 +0200},
    biburl    = {https://dblp.org/rec/journals/corr/abs-2110-07038.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
  }
  ```