readme: ---
  tags:
  - BERT
  - Text Classification
  - relation
  language:
  - ar
  - en
  license: mit
  datasets:
  - ACE2005
  ---
  
  # Arabic Relation Extraction Model
  - [Github repo](https://github.com/edchengg/GigaBERT)
  - Relation Extraction model based on [GigaBERTv4](https://huggingface.co/lanwuwei/GigaBERT-v4-Arabic-and-English).
  - Model detail: mark two entities in the sentence with special markers (e.g., ```XXXX <PER> entity1 </PER> XXXXXXX <ORG> entity2 </ORG> XXXXX```). Then we use the BERT [CLS] representation to make a prediction.
  - ACE2005 Training data: Arabic
  - [Relation tags](https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/arabic-relations-guidelines-v6.5.pdf) including: Physical, Part-whole, Personal-Social, ORG-Affiliation, Agent-Artifact, Gen-Affiliation
  ## Hyperparameters
  - learning_rate=2e-5
  - num_train_epochs=10
  - weight_decay=0.01
  
  
  ## How to use
  Workflow of a relation extraction model:
  1. Input --> NER model --> Entities
  2. Input sentence + Entity 1 + Entity 2 --> Relation Classification Model --> Relation Type
  
  ```python
  >>> from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer, AuotoModelForSequenceClassification
  
  >>> ner_model = AutoModelForTokenClassification.from_pretrained("ychenNLP/arabic-ner-ace")
  >>> ner_tokenizer = AutoTokenizer.from_pretrained("ychenNLP/arabic-ner-ace")
  >>> ner_pip = pipeline("ner", model=ner_model, tokenizer=ner_tokenizer, grouped_entities=True)
  
  >>> re_model = AutoModelForSequenceClassification.from_pretrained("ychenNLP/arabic-relation-extraction")
  >>> re_tokenizer = AutoTokenizer.from_pretrained("ychenNLP/arabic-relation-extraction")
  >>> re_pip = pipeline("text-classification", model=re_model, tokenizer=re_tokenizer)
  
  def process_ner_output(entity_mention, inputs):
      re_input = []
      for idx1 in range(len(entity_mention) - 1):
          for idx2 in range(idx1 + 1, len(entity_mention)):
              ent_1 = entity_mention[idx1]
              ent_2 = entity_mention[idx2]
  
              ent_1_type = ent_1['entity_group']
              ent_2_type = ent_2['entity_group']
              ent_1_s = ent_1['start']
              ent_1_e = ent_1['end']
              ent_2_s = ent_2['start']
              ent_2_e = ent_2['end']
              new_re_input = ""
              for c_idx, c in enumerate(inputs):
                  if c_idx == ent_1_s:
                      new_re_input += "<{}>".format(ent_1_type)
                  elif c_idx == ent_1_e:
                      new_re_input += "</{}>".format(ent_1_type)
                  elif c_idx == ent_2_s:
                      new_re_input += "<{}>".format(ent_2_type)
                  elif c_idx == ent_2_e:
                      new_re_input += "</{}>".format(ent_2_type)
                  new_re_input += c
              re_input.append({"re_input": new_re_input, "arg1": ent_1, "arg2": ent_2, "input": inputs})
      return re_input
      
  def post_process_re_output(re_output, text_input, ner_output):
      final_output = []
      for idx, out in enumerate(re_output):
          if out["label"] != 'O':
              tmp = re_input[idx]
              tmp['relation_type'] = out
              tmp.pop('re_input', None)
              final_output.append(tmp)
  
      template = {"input": text_input,
                  "entity": ner_output,
                  "relation": final_output}
  
      return template
  
  text_input = """ويتزامن ذلك مع اجتماع بايدن مع قادة الدول الأعضاء في الناتو في قمة موسعة في العاصمة الإسبانية، مدريد."""
  
  ner_output = ner_pip(text_input) # inference NER tags
  
  re_input = process_ner_output(ner_output, text_input) # prepare a pair of entity and predict relation type
  
  re_output = []
  for idx in range(len(re_input)):
      tmp_re_output = re_pip(re_input[idx]["re_input"]) # for each pair of entity, predict relation
      re_output.append(tmp_re_output[0])
  
  
  
  re_ner_output = post_process_re_output(re_output, text_input, ner_output) # post process NER and relation predictions
  print("Sentence: ",re_ner_output["input"])
  print('====Entity====')
  for ent in re_ner_output["entity"]:
    print('{}--{}'.format(ent["word"], ent["entity_group"]))
  print('====Relation====')
  for rel in re_ner_output["relation"]:
    print('{}--{}:{}'.format(rel['arg1']['word'], rel['arg2']['word'], rel['relation_type']['label']))
  
  Sentence:  ويتزامن ذلك مع اجتماع بايدن مع قادة الدول الأعضاء في الناتو في قمة موسعة في العاصمة الإسبانية، مدريد.
  ====Entity====
  بايدن--PER
  قادة--PER
  الدول--GPE
  الناتو--ORG
  العاصمة--GPE
  الاسبانية--GPE
  مدريد--GPE
  ====Relation====
  قادة--الدول:ORG-AFF
  الدول--الناتو:ORG-AFF
  العاصمة--الاسبانية:PART-WHOLE
  ```
  
  ### BibTeX entry and citation info
  
  ```bibtex
  @inproceedings{lan2020gigabert,
    author     = {Lan, Wuwei and Chen, Yang and Xu, Wei and Ritter, Alan},
      title      = {Giga{BERT}: Zero-shot Transfer Learning from {E}nglish to {A}rabic},
      booktitle  = {Proceedings of The 2020 Conference on Empirical Methods on Natural Language Processing (EMNLP)},
      year       = {2020}
    } 
  ```
  