readme: ---
  language: 
    - en
  tags:
  - mathematics
  - math-aware
  datasets:
  - MathematicalStackExchange
  ---
  
  # Math-aware RoBERTa
  
  This repository contains our pre-trained RoBERTa-based model for ARQMath 3. It was initialised from RoBERTa-base and further pre-trained on Math StackExchange in only one stage. We also added more LaTeX tokens to the tokenizer to enable a better tokenization of mathematical formulas. This model is not yet fine-tuned on a specific task. 
  
  For further details, please read our paper: http://ceur-ws.org/Vol-3180/paper-07.pdf. 
  
  
  
  # Usage
  
  You can use this model to further fine-tune it on any math-aware task you have in mind, e.g., classification, question-answering, etc. . Please note, that the model in this repository is only pre-trained and not fine-tuned.
  
  
  # Citation
  
  If you find this model useful, consider citing our paper:
  ```
  @article{reusch2022transformer,
    title={Transformer-Encoder and Decoder Models for Questions on Math},
    author={Reusch, Anja and Thiele, Maik and Lehner, Wolfgang},
    year={2022},
    organization={CLEF}
  }
  ```
  