readme: ---
  datasets:
  - relbert/semeval2012_relational_similarity_v6
  model-index:
  - name: relbert/relbert-roberta-base-semeval2012-v6-mask-prompt-e-nce-2
    results:
    - task:
        name: Relation Mapping
        type: sorting-task
      dataset:
        name: Relation Mapping
        args: relbert/relation_mapping
        type: relation-mapping
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.8311904761904761
    - task:
        name: Analogy Questions (SAT full)
        type: multiple-choice-qa
      dataset:
        name: SAT full
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.47058823529411764
    - task:
        name: Analogy Questions (SAT)
        type: multiple-choice-qa
      dataset:
        name: SAT
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.47774480712166173
    - task:
        name: Analogy Questions (BATS)
        type: multiple-choice-qa
      dataset:
        name: BATS
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.5630906058921623
    - task:
        name: Analogy Questions (Google)
        type: multiple-choice-qa
      dataset:
        name: Google
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.746
    - task:
        name: Analogy Questions (U2)
        type: multiple-choice-qa
      dataset:
        name: U2
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.4605263157894737
    - task:
        name: Analogy Questions (U4)
        type: multiple-choice-qa
      dataset:
        name: U4
        args: relbert/analogy_questions
        type: analogy-questions
      metrics:
      - name: Accuracy
        type: accuracy
        value: 0.48148148148148145
    - task:
        name: Lexical Relation Classification (BLESS)
        type: classification
      dataset:
        name: BLESS
        args: relbert/lexical_relation_classification
        type: relation-classification
      metrics:
      - name: F1
        type: f1
        value: 0.9136658128672593
      - name: F1 (macro)
        type: f1_macro
        value: 0.9119300574747814
    - task:
        name: Lexical Relation Classification (CogALexV)
        type: classification
      dataset:
        name: CogALexV
        args: relbert/lexical_relation_classification
        type: relation-classification
      metrics:
      - name: F1
        type: f1
        value: 0.8356807511737089
      - name: F1 (macro)
        type: f1_macro
        value: 0.6445552217787743
    - task:
        name: Lexical Relation Classification (EVALution)
        type: classification
      dataset:
        name: BLESS
        args: relbert/lexical_relation_classification
        type: relation-classification
      metrics:
      - name: F1
        type: f1
        value: 0.6598049837486457
      - name: F1 (macro)
        type: f1_macro
        value: 0.6390833044290024
    - task:
        name: Lexical Relation Classification (K&H+N)
        type: classification
      dataset:
        name: K&H+N
        args: relbert/lexical_relation_classification
        type: relation-classification
      metrics:
      - name: F1
        type: f1
        value: 0.9680740070946651
      - name: F1 (macro)
        type: f1_macro
        value: 0.9022447613880005
    - task:
        name: Lexical Relation Classification (ROOT09)
        type: classification
      dataset:
        name: ROOT09
        args: relbert/lexical_relation_classification
        type: relation-classification
      metrics:
      - name: F1
        type: f1
        value: 0.880288310874334
      - name: F1 (macro)
        type: f1_macro
        value: 0.8774948713508829
  
  ---
  # relbert/relbert-roberta-base-semeval2012-v6-mask-prompt-e-nce-2
  
  RelBERT fine-tuned from [roberta-base](https://huggingface.co/roberta-base) on  
  [relbert/semeval2012_relational_similarity_v6](https://huggingface.co/datasets/relbert/semeval2012_relational_similarity_v6).
  Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
  It achieves the following results on the relation understanding tasks:
  - Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/relbert-roberta-base-semeval2012-v6-mask-prompt-e-nce-2/raw/main/analogy.json)):
      - Accuracy on SAT (full): 0.47058823529411764 
      - Accuracy on SAT: 0.47774480712166173
      - Accuracy on BATS: 0.5630906058921623
      - Accuracy on U2: 0.4605263157894737
      - Accuracy on U4: 0.48148148148148145
      - Accuracy on Google: 0.746
  - Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/relbert-roberta-base-semeval2012-v6-mask-prompt-e-nce-2/raw/main/classification.json)):
      - Micro F1 score 