readme: ---
  language: hu
  license: apache-2.0
  datasets:
  - common_crawl
  - wikipedia
  ---
  
  # huBERT base model (cased)
  
  ## Model description
  
  Cased BERT model for Hungarian, trained on the (filtered, deduplicated) Hungarian subset of the Common Crawl and a snapshot of the Hungarian Wikipedia.
  
  ## Intended uses & limitations
  
  The model can be used as any other (cased) BERT model. It has been tested on the chunking and
  named entity recognition tasks and set a new state-of-the-art on the former.
  
  ## Training
  
  Details of the training data and procedure can be found in the PhD thesis linked below. (With the caveat that it only contains preliminary results
  based on the Wikipedia subcorpus. Evaluation of the full model will appear in a future paper.)
  
  ## Eval results
  
  When fine-tuned (via `BertForTokenClassification`) on chunking and NER, the model outperforms multilingual BERT, achieves state-of-the-art results on
  both tasks. The exact scores are
  
  | NER | Minimal NP | Maximal NP |
  |-----|------------|------------|
  | **97.62%** | **97.14%** | **96.97%** |
  
  ### BibTeX entry and citation info
  
  If you use the model, please cite the following papers:
  
  [Nemeskey, Dávid Márk (2020). "Natural Language Processing Methods for Language Modeling." PhD Thesis. Eötvös Loránd University.](https://hlt.bme.hu/en/publ/nemeskey_2020)
  
  Bibtex:
  ```bibtex
  @PhDThesis{ Nemeskey:2020,
    author = {Nemeskey, Dávid Márk},
    title  = {Natural Language Processing Methods for Language Modeling},
    year   = {2020},
    school = {E\"otv\"os Lor\'and University}
  }
  ```
  
  [Nemeskey, Dávid Márk (2021). "Introducing huBERT." In: XVII. Magyar Számítógépes Nyelvészeti Konferencia (MSZNY 2021). Szeged, pp. 3-14](https://hlt.bme.hu/en/publ/hubert_2021)
  
  Bibtex:
  ```bibtex
  @InProceedings{ Nemeskey:2021a,
    author = {Nemeskey, Dávid Márk},
    title = {Introducing \texttt{huBERT}},
    booktitle = {{XVII}.\ Magyar Sz{\'a}m{\'i}t{\'o}g{\'e}pes Nyelv{\'e}szeti Konferencia ({MSZNY}2021)},
    year = 2021,
    pages = {TBA},
    address = {Szeged},
  }
  ```
  