readme: ---
  language: 
    - en
  tags:
  - classics
  - citation mining
  widget:
  - text: "Homer's Iliad opens with an invocation to the muse (1. 1)."
  ---
  
  ### Model and entities
  `roberta_classics_ner` is a domain-specific RoBERTa-based model for named entity recognition in Classical Studies. It recognises bibliographical entities, such as:
  
  
  | id  | label         | desciption                                  | Example               |
  | --- | ------------- | ------------------------------------------- | --------------------- |
  | 0   | 'O'           | Out of entity                               |                       | 
  | 1   | 'B-AAUTHOR'   | Ancient authors                             | *Herodotus*           |
  | 2   | 'I-AAUTHOR'   |                                             |                       |
  | 3   | 'B-AWORK'     | The title of an ancient work                | *Symposium*, *Aeneid* |
  | 4   | 'I-AWORK'     |                                             |                       |
  | 5   | 'B-REFAUWORK' | A structured reference to an ancient work   | *Homer, Il.*          |
  | 6   | 'I-REFAUWORK' |                                             |                       |
  | 7   | 'B-REFSCOPE'  | The scope of a reference                    | *II.1.993a30–b11*     |
  | 8   | 'I-REFSCOPE'  |                                             |                       |
  | 9   | 'B-FRAGREF'   | A reference to fragmentary texts or scholia | *Frag. 19. West*      |
  | 10  | 'I-FRAGREF'   |                                             |                       |
  
  
  ### Example
  
  
  ```
  B-AAUTHOR   B-AWORK                                      B-REFSCOPE
  Homer  's   Iliad opens with an invocation to the muse ( 1. 1).
  ```
  
  
  
  ### Dataset
  
  
  `roberta_classics_ner` was fine-tuned and evaluated on `EpiBau`, a dataset which has not been released publicly yet. It is composed of four volumes of [Structures of Epic Poetry](https://www.epische-bauformen.uni-rostock.de/), a compendium on the narrative patterns and structural elements in ancient epic. 
  Entity counts of the `Epibau` dataset are the following: 
  
  |                | train-set | dev-set | test-set |
  | -------------- | --------- | ------- | -------- |
  | word count     | 712462    | 125729  | 122324   |
  | AAUTHOR        | 4436      | 1368    | 1511     |
  | AWORK          | 3145      | 780     | 670      |
  | REFAUWORK      | 5102      | 988     | 1209     |
  | REFSCOPE       | 14768     | 3193    | 2847     |
  | FRAGREF        | 266       | 29      | 33       |
  | total entities | 13822     | 1415    | 2419     |
  
  
  ### Results
  
  The model was developed in the context of experiments reported [here](http://infoscience.epfl.ch/record/291236?&ln=en).Trained and tested on `EpiBau` with a 85-15 split, the model yields a general F1 score of **.82** (micro-averages). Detailed scores are displayed below. Evaluation was performed with the [CLEF-HIPE-scorer](https://github.com/impresso/CLEF-HIPE-2020-scorer), in strict mode) 
  
  
  | metric    | AAUTHOR | AWORK | REFSCOPE | REFAUWORK |
  | --------- | ------- | ----- | -------- | --------- |
  | F1        | .819    | .796  | .863     | .756      |
  | Precision | .842    | .818  | .860     | .755      |
  | Recall    | .797    | .766  | .756     | .866      | 
  
  
  
  
  Questions, remarks, help or contribution ? Get in touch [here](https://github.com/AjaxMultiCommentary), we'll be happy to chat !  
  
  