readme: ---
  license: apache-2.0
  ---
  
  # Model description
  
  This model corresponds to the paper "A Domain-adaptive Pre-training Approach for Language Bias Detection in News" (Krieger et al.,2022): https://github.com/Media-Bias-Group/A-Domain-adaptive-Pre-training-Approach-for-Language-BiasDetection-in-News
  
  The model can be used for sequence classification tasks of biased and non-biased language in news and media. It is initialized with *roberta-base* weights and fine-tuned on the *Wiki Neutrality Corpus* (Pryzant et al., 2020). More details on the training setup and experiments can be found in our paper.
  
  # How to use
  You can use the model with the PyTorch framework
  
  ```
  #imports
  !pip install transformers
  !pip install openpyxl
  import torch
  import torch.nn as nn
  import numpy as np
  from transformers import RobertaTokenizer,RobertaModel
  
  #define model class including binary classification layer 
  class RobertaClass(torch.nn.Module):
      def __init__(self):
          super(RobertaClass, self).__init__()
          self.roberta = RobertaModel.from_pretrained("roberta-base")
          self.vocab_transform = torch.nn.Linear(768, 768)
          self.dropout = torch.nn.Dropout(0.2)
          self.classifier1 = torch.nn.Linear(768,2)
  
      def forward(self, input_ids, attention_mask):
          output_1 = self.roberta(input_ids=input_ids, attention_mask=attention_mask)
          hidden_state = output_1[0]
          pooler = hidden_state[:, 0]
          pooler = self.vocab_transform(pooler)
          pooler = self.dropout(pooler)
          output = self.classifier1(pooler)
  
          return output
  
  #load model parameters
  weight_dict = torch.load('DA-Roberta.bin')
                           
  #initialize model with fine-tuned parameters
  model = RobertaClass()
  model.load_state_dict(weight_dict)
  
  #exemplary bias classification with instance extracted from BABE dataset (Spinde et al.,2021)
  tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
  inputs = tokenizer("A cop shoots a Black man, and a police union flexes its muscle", return_tensors="pt")
  outputs = model(**inputs)
  
  if int(torch.argmax(outputs)) == 1:
      print("Biased")
  else:
      print("Non-biased")
  ```
  
  # Cite as 
  
  ```
  @InProceedings{Krieger2022,
    author={Krieger, David and Spinde, Timo and Ruas, Terry and Kulshrestha, Juhi and Gipp, Bela},
    booktitle={2022 ACM/IEEE Joint Conference on Digital Libraries (JCDL)}, 
    title={A Domain-adaptive Pre-training Appraoch for Language Bias Detection in News}, 
    year={2022},
    address = "Cologne,Germany"
    }
  ```