Although much work in NLP has focused on measuring and mitigating stereotypical bias in semantic spaces, research addressing bias in computational argumentation is still in its infancy. In this paper, we address this research gap and conduct a thorough investigation of bias in argumentative language models. To this end, we introduce ABB A , a novel resource for bias measurement specifically tailored to argumentation. We employ our resource to assess the effect of argumentative fine-tuning and debiasing on the intrinsic bias found in transformer-based language models using a lightweight adapter-based approach that is more sustainable and parameter-efficient than full fine-tuning. Finally, we analyze the potential impact of language model debiasing on the performance in argument quality prediction, a downstream task of computational argumentation. Our results show that we are able to successfully and sustainably remove bias in general and argumentative language models while preserving (and sometimes improving) model performance in downstream tasks. We make all experimental code and data available at https://github.com/ umanlp/FairArgumentativeLM . Recently, pre-trained language models (PLMs), e.g., BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), GPT-2 (Radford et al., 2019) and DialoGPT (Zhang et al., 2020) have been shown to encode and amplify a range of stereotypical biases, such as racism, and sexism (e.g., Kurita et al., 2019a; Dev et al., 2020; Nangia et al., 2020; Lauscher et al., 2021a, inter alia ). While such types of biases provide the basis for interesting academic research, e.g., historical analyses (e.g., Garg et al., 2018; Tripodi et al., 2019; Walter et al., 2021, inter alia ), stereotyping constitutes a representational harm (Barocas et al., 2017; Blodgett et al., 2020), and can lead in many concrete socio-technical application scenarios to severe ethical issues by reinforcing societal biases (Hovy and Spruit, 2016; Shah et al., 2020; Mehrabi et al., 2021). But while prior work has focused on how to evaluate and mitigate unfair biases for general-purpose LMs (e.g., Webster et al., 2020) and their applications to specific domains and genre like, for instance, conversational LMs (e.g., Barikeri et al., 2021), there has been little attention to the problem of bias in argumentative language . This is despite previous work from Spliethver and Wachsmuth (2020) pointing out the high potential for harm, due to the high sensitivity of envisioned applications like self-determined opinion formation systems, as well as, crucially, showing that argumentative corpora like those from the online debate portal debate.org (Durmus and Cardie, 2019) do encode unfair biases, which are likely to be captured by argumentative LMs. This is particularly problematic as research in computational argumentation regularly makes use of such corpora for injecting knowledge about argumentative language into PLMs (e.g., Alshomary et al., 2021). Still, to date, there is neither an evaluation resource specifically tailored to argumentative language, nor knowledge on debiasing argumentative LMs or on the effects of debiasing on argumentative downstream tasks. Contributions. We address this research gap with the following contributions: we present AB B A , the first human-annotated resource specifically targeted at English argumentative language, which is annotated for two kinds of social bias that are still under-explored in NLP, namely Queerphobia and Islamophobia . Next, we use AB B A to answer the following four research questions ( RQs ): (RQ1) How does argumentative fine-tuning affect measurable biases in PLMs? typical biases in the LMs, highlighting the importance of bias measurement after injecting argumentative knowledge (4.1). Lauscher et al. (2021a) recently introduced debiasing adapters , a modular and sustainable way of encoding debiasing knowledge in LMs. We con-firm the effectiveness of debiasing adapters with Counterfactual Data Augmentation (Zhao et al., 2018) on two diverse corpora (4.2). (RQ3) Can we obtain an (efficient and robust) fair and argumentative language model given our preexisting set of adapters? We show for the first time how to stack debiasing adapters with argumentation adapters to produce an argumentative and fair language model . Our results indicate that stacking order matters (4.3). (RQ4) What are the effects on argumentative downstream tasks, e.g., argument quality prediction? In a final downstream evaluation encompassing two different datasets for argument quality prediction, we demonstrate that debiasing can have a positive impact on model performance. On one of the corpora, our best results are obtained when combining argumentation and debiasing adapters, hinting at the effectiveness of fair and argumentative language modeling (4.4). We create AB B A , the first annotated corpus of bias in argumentative text following the methodology from Barikeri et al. (2021): (1) specification of the social biases of interest, (2) retrieval of candidates of biased statements, and (3) manual annotation. Bias Specifications. We define the social biases we are interested in using the established notion of explicit bias specifications (Caliskan et al., 2017; Lauscher et al., 2020a). It consists of two sets of target terms ( T 1 and T 2 ) denoting two demographic groups that exhibit different stereotypical perceptions w.r.t. two opposing sets of attribute terms ( A 1 and A 2 ) . Concretely, T 1 consists of target terms referring to a minoritized group (e.g., Muslim ), while T 2 consists of target terms corresponding to a dominant group (e.g., Christian ), i.e., a group in power (D'Ignazio and Klein, 2020). We focus on the bias dimensions Queerphobia and Islamophobia since they have received little attention in NLP research on bias when compared to sexism or other ethnic bias. We view Queerness as an umbrella term for the minority group of the LGBTQI+ community, which includes people of all sexual orientations and gender identities except for heterosexual and cisgender. We compare this to the dominant group of heterosexual cisgender people. The target and attribute terms used for candidate identification are based on the specifications of Barikeri et al. (2021). They include a wide range of attribute terms from the sociological literature and manually compiled target terms. The attribute terms were assembled such that each stereotypical attribute term a 1 forms a loose antonym of an counter-stereotypical attribute term a 2 with a positive or negative sentiment. An exemplary partial term list of the bias specifications can be found in Table 1 and the full set in the Appendix. Candidate Retrieval. We use the dataset from debate.org originally collected by Durmus and Cardie (2019), one of most widely used resources in research on computational argumentation. For retrieving candidates, we compute the Cartesian product of the terms of the minoritized group T 1 with all stereotypical terms of A 1 , giving us a set of stereotyped tuples from T 1  A 1 (e.g., gay and sinful ). Using this set, we extract all sentences and their corresponding arguments that contain both terms from the tuples in a window of size 20 (set during corpus construction to improve the quality of the retrieved passages). We further reduced the compiled comments to those with a maximum number of 500 tokens to allow for a better visualization and to ensure that the annotators attentively read the entire argument. In total, we retrieve 889 candidate sentences from 614 different arguments for Queerphobia and 1,879 candidate sentences from 1,101 different arguments for Islamophobia . Annotating bias. We manually label the candidate sentence and the corresponding argument according to whether a stereotypical bias is present or not. To this end, we hired four annotators, who are all non-native speakers but have excellent English proficiency with academic backgrounds and who hold at least a Bachelor's degree, in slightly different majors (engineering, data science, infor-7842 Dimension Target Term Sets Attribute Term Sets Islamophobia T 1 muslim(s) , islam , quran , koran , ... mation systems, and computer science). They are of diverse gender and cultural background. Annotators were provided with the guidelines found in the Appendix. We initially conducted a pilot study on 90 randomly drawn arguments to iteratively calibrate annotations and refine the guidelines on the basis of the annotators' feedback. Finally, we split the corpus evenly into four independent, equally-sized portions and added further 50 randomly drawn overlapping arguments to analyze annotation quality. In the last step, we merged the annotations on the calibration set using majority voting. The number of annotated and biased instances in the corpus is shown in Table 2. We show examples of biased sentences in Table 3. Analysis of the Annotations. On the overlapping set consisting of 50 arguments, we obtain an inter-annotator agreement (IAA) for Queerphobia on the sentence-level for both Fleiss'  (Fleiss, 1971) and Krippendorff's  (Krippendorff, 2013) of 0 . 65 . The agreement on the argument-level is slightly weaker with 0 . 61 for both measures. For the Islamophobia dimension, we observe a stronger agreement of 0 . 66 on sentence-level and  = 0 . 72 and  = 0 . 73 on the argument-level. Although we are dealing with a rather subjective annotation task, IAA indicates a substantial agreement among the annotators (Viera and Garrett, 2005), suggesting that they are able to reliably identify stereotypes in argumentative sentences and longer text. To determine reasons for disagreement among annotators, we manually conducted a qualitative analysis on the annotated arguments. For Queerphobia , we found that annotators mostly disagreed on statements that referred to the homosexual lifestyle, rather than homosexual people. The following example illustrates one such case: [...] Basically, a gay person is not allowed to engage in sexual acts with an-other man because there is a 0% chance of offspring being produced. This falls into the same category of not using contraceptives, getting abortions, etc. It is not a sin for a gay person to acknowledge their sexuality, or to act in a gay' manner. It is only a sin if he/she gives in to their urges. [...] Here, the annotators disagreed in the annotation of the entire argument. Although the debater clearly states that actually being gay is not a sin, in his opinion, living a homosexual lifestyle is a sin. It appears that for some annotators being homosexual is equivalent to living in a homosexual relationship, while others clearly distinguished these two aspects. For Islamophobia , the disagreements mostly related to arguments that make a distinction between Muslims and the religion Islam, e.g.: [...] I have no issue with Islam, or any religion in general, if you leave me alone I leave you alone, you wondered why so many people hate Islam, its because of the same [...] in your last paragraph, y'all act as if terrorism is 100% okay. That needs to change before Muslims can consider Islam anywhere close to a great religion. Here, the fact that the debater is making an ambiguous statement, expressing no prejudice against Islam but against Muslims caused confusion among the annotators resulting in disagreement. modeling along our two bias dimensions of interest. Instead of full model fine-tuning, we opt for a more sustainable strategy by relying on adapters (Houlsby et al., 2019) to reduce computation time and energy consumption. In addition, the modularity of adapters enables their reuse in further settings and in combination with other pre-trained adapters. Argumentation Adapter. Following Alshomary et al. (2021), we tune general pre-trained models on a large set of arguments to obtain an argumentative language model. In contrast to the original work, we rely on language adapters. Concretely, we adopt the architecture proposed by Pfeiffer et al. (2020), which inserts a single adapter, a two-layer feed-forward network, into each transformer layer. The output of the adapter is computed as A argument ( h , r ) = U ( ReLU ( D ( h ))) + r , with the two matrices D  R h  d and U  R d  h as the adapter's down-projection and up-projection, respectively, h as the transformer's hidden state, and r as the residual. In addition, we inject invertible adapters, which are stacked on top of the embedding layer and the inverses of the invertible adapters are placed in front of the output layer. They perform a similar function to the language adapters, but aim to capture token-level specific transformations (Pfeiffer et al., 2020). Both the language adapters and the invertible adapters are trained on a language modeling task using a causal language modeling loss for auto-regressive models and a masked language modeling loss for auto-encoding models, respectively. Debiasing Adapter. For debiasing, we inject debiasing adapters (Lauscher et al., 2021a) into the models, using the same adapter architecture as before. Following the original work, we use Counterfactual Data Augmentation (Zhao et al., 2018, CDA) and train the adapter parameters on the augmented corpus to break stereotypical associations in the model. To this end, we manually compile pairs of opposing target terms ( t i , t j )  T 1  T 2 , such that t i forms the most suitable antonym of t j in the sense of minority and dominant group (e.g., muslim and christian ) and can be substituted grammatically interchangeably. While this is arguably straightforward with the Islamophobia bias specifications, the target terms of the Queerness dimension are more complex to juxtapose. Therefore, we clustered them into three groups of sexual identity' (e.g., {gay, straight} ), gender identity' (e.g., {transgender, cisgender} ) and biological sex' (e.g., {androgyne, unisexual} ) so as to find the best matching pairs of antonyms (cf. the list in the Appendix). We then replace all occurring target terms from T 1 or T 2 with their opposite term from the set of tuples P = { ( t i , t j ) } N (we randomly select a term from the list if multiple substitutions are possible). We opt for a two-sided application of CDA, keeping both the counterfactual and the original sentences in the training set to avoid over-correction (Webster et al., 2020). We append each counterfactual sentence immediately after its original counterpart and train in two settings, namely using: a) only biased and counterfactual sentences; b) all sentences, i.e., also including neutral ones. Combining Adapters. We investigate three different architectures: first, in 4.3, we study two architectures using AdapterStacking (Pfeiffer et al., 2020), i.e., by stacking the argumentation adapter on top of a debiasing adapter and vice versa (Fig-ure 1). Second, in 4.4, we compare the best architectures from 4.3 with AdapterFusion (Pfeiffer et al., 2020), which requires training additional network layers for interpolating the adapters' outputs. We next describe the experiments to answer the research questions RQ1 through RQ4 (Section 1) that underpin our investigation. computing the LMB score reflecting how much more likely the model is to generate a stereotypically biased argument compared to an inversely biased one. We start with our set of opposing target terms P  T 1  T 2 and we extract the set of all statements S from AB B A (containing instances of term t i such that ( t i , t j )  P ), which have been labelled as stereotypically biased. This results in 279 biased instances for Queerphobia and 465 instances for Islamophobia , respectively. We then create for each instance s ( t i ,a )  S (e.g., All Muslims are terrorists ), a corresponding inversely biased sentence s (cid:48) ( t j ,a ) (e.g., All Christians are terrorists ) to give us a set S (cid:48) of counter-stereotypical statements. In case of multiple pairs for a target term (e.g., {homosexual, heterosexual} and {homosexual, straight} ), we create one counter-stereotypically biased sentence for each possible combination. We then compute the model's perplexity for all statements in the two paired sets S and S (cid:48) with stereotypical and counter-stereotypical statements. Following Barikeri et al. (2021), we compute the mean perplexity for multiple counterfactual instances created from a single biased instance and remove outliers to avoid distorted significance results (Pollet and van der Meij, 2017). The final LMB score corresponds to the t-value obtained by subjecting the paired perplexities to the student's t-test (  = 0 . 05 ). sists of over 380k arguments from over 59k debates. (ii) Considering that it contains mostly arguments retrieved from Debate.org (  87% ), we verify our results using a second corpus: Webis-ChangeMyView-20 (CMV; Al Khatib et al., 2020), which contains over 3.6 million arguments extracted from the ChangeMyView subreddit. For ensuring comparability, we cut each corpus to 300k and perform a train-validation split of 80:20. Models. We experiment with four LMs from Huggingface Transformers (Wolf et al., 2020): BERT ( bert-base-uncased ), GPT-2 ( gpt-2 ), DialoGPT ( microsoft/DialoGPT-medium ) and RoBERTa ( roberta-base ). With the exception of DialoGPT, which contains contains 24 layers with a hidden size of 1 , 024 , all models consist of 12 layers with a hidden size of 768 . Adapter Training and Optimization. We train the argumentative adapters separately on Args.me and CMV for each of the models. Concretely, we train for 10 epochs using the Adam optimizer (Kingma and Ba, 2015) (weight decay = 0 . 01 ,  1 = 0 . 9 ,  2 = 0 . 999 , (cid:15) = 1  10  6 , learning rate= 1  10  4 ) and early stopping based on the perplexity on the validation set (patience: 2 epochs). We set the effective batch size to 32 except for training DialoGPT, for which we employ an effective training batch size of 8 for reasons of computational capacity. The adapter reduction factor is 16 . Results. The LMB scores on AB B A before and after fine-tuning the four PLMs are shown in Figure 2. A negative t-value suggests a stereotypical bias; a positive t-value denotes an counter-stereotypical LMB, respectively. Before fine-tuning, GPT-2 is the only model that exhibits a significant stereotypical bias along the Queerphobia dimension. We show an example sentence pair exhibiting a high difference in model perplexity in Table 4 and provide more examples in the Appendix. For BERT, no significant difference was found between the perplexities on stereotypical and counter-stereotypical sentences along Queer-7845 BERT RoBERTa GPT-2 DialoGPT  15  10  5 0 5 10 15 t v a l u e * * * * * * * * * phobia , whereas RoBERTa and DialoGPT even show a significant counter-stereotypical bias. All PLMs except RoBERTa exhibit a stereotypical bias for the Islamophobia bias, with a significant effect size for DialoGPT and BERT. The findings for DialoGPT are consistent with the results of Barikeri et al. (2021) for conversational text. When adapter-fine-tuning the PLMs on argumentative texts (CMV, Args.me), we notice that the perplexities on AB B A decreased, indicating that we successfully managed to inject argumentative knowledge into the models. However, we also observe that while for RoBERTa, no significant changes in t-values for either bias dimension occur, the sterotypical bias effects of DialoGPT and GPT-2 along the Islamophobia bias dimension are reinforced by argumentative fine-tuning. Most interesting is the effect on DialoGPT along Queerphobia . While the original model exhibited a significant counter-stereotypical bias, fine-tuning results in an opposite bias effect for both CMV and Args.me. Given that the stereotypical bias along the Islamophobia dimension is also reinforced by fine-tuning DialoGPT, it underscores the tendency of the model Args.me Wikipedia Strategy # Train # Val. to pick up and amplify stereotypical biases. All in all, these findings highlight the importance of carefully measuring bias after injecting argumentative knowledge into the models. Debiasing Data. We perform our two CDA strategies from 3 on two corpora: (i) the English Wikipedia ( 20200501.en dump) representing general-purpose encycopledic text. We randomly subsample the corpus, originally consisting of 6,078,422 text blocks, to 500,000 text blocks. (ii) We additionally experiment with the Args.me corpus, which also serves as the source for argumentative text. On both corpora, we perform a train-validation slit of 80:20. The resulting train and test set sizes for both bias types Queerphobia and Islamophobia are listed in Table 5. Models. We focus on two PLMs that exhibited bias along one of the dimensions in the previous experiments and which represent different types of PLMs: BERT as a representative of models trained via masked language modeling and GPT-2 as a model trained via causal language modeling. Adapter Training and Optimization. We train the adapters for 10 epochs on the CDA-augmented data sets which include the neutral sentences, and for 1 epoch on the data sets that exclude the neutral sentences. The rest of the training procedure and all other hyperparameters are the same as for training the argumentaive adapters. Results. We report bias effect size using LMB in Figure 3. The results indicate that, while the original PLMs exhibited significant bias along a dimension, using debiasing adapters we are able to successfully reduce the measurable bias from a significant to a non-significant amount , the only 7846 Queerphobia Islamophobia  15  10  5 0 5 10 15 t v a l u e * * * * * Original Wikipedia w/ N Wikipedia w/o N Args.me w/ N Args.me w/o N exception with the adapters for GPT-2 trained on the CDA-augmented Wikipedia. When we exclude neutral sentences the scores switch into the counter-stereotypical direction: we hypothesize that this indicates the need for a better balancing and sampling of the training data. We see a similar effect for cases in which the original PLM did not exhibit a significant bias  the LMB is likely to switch to the opposite, counter-stereotypical direction. Taking advantage of the modular nature of adapters, we combine argumentation and debiasing adapters (4.1-4.2) to obtain a fair and argumentative language model using AdapterStacking (3). We focus on the bias dimensions for which the original models exhibited a stereotypical effect size. Results. Figure 4 shows the LMB scores of BERT on Islamophobia and GPT-2 along Queerphobia for different stacking orders of the argumentation adapter trained on CMV and the respective debiasing adapters trained on Wikipedia or Args.me (results for the other dimensions and other Argumentation Adapter First Argumentation Adapter Second 15 10 5 0 5 10 15 t v a l ue * * * * * * Argumentation CMV Wikipedia w/ N Wikipedia w/o N Args.me w/ N Args.me w/o N argumentation adapters are found in the Appendix). For BERT, stacking the debiasing adapters for Islamophobia second and the argumentation adapter trained on CMV first (left) reduces the bias to an non-significant amount only in a single case, while stacking the debiasing adapter first (right) removes the bias in three out of four setups. Also for GPT-2, stacking the debiasing adapter first leads to better debiasing results. We hypothesize that the reason for this effect is that both types of adapters are optimized for receiving the input directly from the transformer layers. Thus, the debiasing adapter is more effective when stacked first. In sum, while our results indicate that stacking order matters and debiasing effects are bigger when debiasing adapters are stacked first , we think that this finding warrants future research on the issue. Data and Measures. For testing the influence of our argumentation and debiasing adapters on argument quality prediction, we employ two recently presented data sets: (1) the IBM-Rank-30k (Gretz et al., 2020), an extension of (Toledo et al., 2019), 7847 Dataset Domain # Train # Validation # Test IBM-Rank-30k  20,974 3,208 6,315 GAQCorpus CQA 1,109 476 500 Debates 1,093 469 538 Reviews 700 400 100 Table 6: Number of arguments in training, validation, and test portions of IBM-Rank-30k and GAQCorpus. which consists of short-length arguments (maxi-mum length of 210 characters) annotated by crowd workers. We use the MACE-P aggregations provided by the authors for model training. (2) Additionally, we use the GAQCorpus (Ng et al., 2020; Lauscher et al., 2020b) which covers real-world arguments from three domains, namely community questions and answers (CQA), online debate forums (Debates), and restaurant reviews (Reviews). An overview of the data sets is given in Table 6. On both data sets, we report Pearson's correlation coefficient ( r ). Following Reimers and Gurevych (2017), we report the average of our experiments conducted 50 times with different random seeds (using the best hyperparameter configuration according to the development set results) and additionally conduct an independent t-test. Models. For all AQ models, we rely on a simple linear regression head into which we input the pooled sequence representation. The fine-tuning strategy for the AQ regression is aligned with our previous approaches. Instead of full fine-tuning of the encoder, we add an additional task-specific adapter on top of the already existing adapters and adjust only the task-specific adapter parameters during training. As before, we employ the BERT and GPT-2 base models ( Base ) as well as the adapter-augmented variants. Concretely, we employ the argumentation adapters trained on Args.me and CMV ( Argsme , CMV ), and the debiasing adapters trained on the CDA-augmented Args.me ( DB-Islamo for BERT, DB-Queer for GPT-2). Again, we also study combinations to optimally combine argumentation, debiasing, and task-specific knowledge using either a stacking ( Stacked ) or fusion architecture ( Fusion ). On IBM-Rank-30k, we follow Gretz et al. (2020) and concatenate topic and argument with an additional separator (BERT) or end-of-sequence token (GPT-2). As baselines, we additionally compare with the best results reported by the original works. we optimize our models using Mean Squared Error. We train all task adapters using Adam (Kingma and Ba, 2015) with a batch size of 32 (weight decay = 0 ,  1 = 0 . 9 and  2 = 0 . 999 ). We pad the input sequences to a maximum length of 128. We choose the best hyper-parameters by grid searching for learning rate   { 1  10  4 , 2  10  4 , 3  10  4 } and number of training epochs  { 1 , 2 , 3 , 4 , 5 } based on the performance on the individual dataset's respective validation portion. Results. The results are shown in Table 7. Generally, though the trends are the same, the scores diverge from the results reported in the original works, which can be attributed to our use of task adapters. Interestingly, while injecting argumentation adapters leads to performance improvements on IBM-ArgQ-Rank-30kArgs in 3 out of 4 cases, it seems to hurt the performance on GAQCorpus. On the other hand, the debiasing adapters do not seem to lead to losses: in contrast, in some cases (IBM and GAQDebates for BERT, GAQDebates for GPT-2), we even note performance improvements. For GAQCorpus, the best results are obtained with an argumentative and fair language model  when fusing debiasing and argumentation adapters. We conclude that fair and argumentative language modeling can have a positive impact on argument quality prediction as downstream task . (2020), and Shah et al. (2020). Bolukbasi et al. (2016) were the first to draw attention to the issue of unfair stereotypical bias in NLP, showing that static word embeddings allow for building biased analogies. Later, Caliskan et al. (2017) proposed the well-known Word Embedding Association Test (WEAT), which was extended to more languages by (Lauscher and Glava, 2019; Lauscher et al., 2020c). More works focused on bias evaluation and mitigation in static word embeddings (Gonen and Goldberg, 2019; Dev and Phillips, 2019; Manzini et al., 2019; Lauscher et al., 2020a), and later, the focused shifted towards detecting and attenuating biases in their successors contextualized word embeddings (Dev and Phillips, 2019; Dev et al., 2020; Tan and Celis, 2019). Here, the authors focused on both, bias in general-purpose pretrained language models (May et al., 2019; Kurita et al., 2019b; Zhao et al., 2019; Webster et al., 2020), and bias in particular downstream scenarios (Dev et al., 2020). For instance, Zhao et al. (2018) proposed Counterfactual Data Augmentation (CDA) for the purpose of debiasing coreference resolution systems. Like many other works (Zmigrod et al., 2019; Lu et al., 2020; Webster et al., 2020; Lauscher et al., 2021a) we explore the method for our purposes. Similarly, Vanmassenhove et al. (2018) focused on machine translation and Sheng et al. (2019) on general natural language generation, while Barikeri et al. (2021) specifically target conversational models. In this work, we follow their process for creating AB B A . Bias in Argumentation. It is extremely surprising that given the plethora of works focused on mining, assessing, and generating arguments as well as reasoning over arguments (Lauscher et al., 2021b), to date, Spliethver and Wachsmuth (2020) were the only ones to investigate and quantify social bias in argumentation. They performed a simple co-occurrence analysis for three different argumentation corpora and trained a custom GloVe model (Pennington et al., 2014) based on argumentative text, which they analyzed with WEAT. Our work builds on top of theirs and is the first to examine bias in relation to an argumentative downstream task and also the first to conduct debiasing for computational argumentation models. In this work, we presented an investigation of bias in PLMs and argumentative text. To this end, we created AB B A , the first annotated corpus tailored for measuring bias in computational argumentation models. Using AB B A , we showed that argumentative fine-tuning of language models may lead to an amplification of biases in the models. We then demonstrated how to obtain a fair and argumentative language model by combining argumentation with debiasing knowledge encapsulated in lightweight adapters to ensure higher sustainability and flexibility, and analyzed the effect of stacking orders. An additional downstream evaluation on argument quality prediction indicated that debiasing can even lead in some cases to improved results. We hope that with this work, especially the novel AB B A resource, we will foster further research on fair computational argumentation. The work of Anne Lauscher is funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement No. 949944, INTEGRA-TOR). We thank the anonymous reviewers for their insightful comments. We like to point the reader to the following limitations and ethical considerations: first, following the large body of debiasing research in NLP, we based our evaluation, mitigation, and annotation approach on a fixed set of manually created terms. We are aware that this set is never finite and may be continually revised in subsequent studies. For a recent discussion we refer to Antoniak and Mimno (2021). This is especially the case for the dimension of Queerphobia , where there is increasing openness and understanding toward more diverse forms of sexual orientation and (gender) identity. For instance, our vocabulary does not include the variety of gender-neutral (neo)pronouns (Dev et al., 2021; Lauscher et al., 2022). Further, studies have shown that the perception of prejudice is not only highly subjective, but also largely culture-dependent (Web-ster et al., 2020). Consequently, in order to conduct a thoroughly unbiased annotation study, annotators should be carefully selected and as diverse as possible in terms of cultural heritage, age, ethnicity, and religious affiliation, as well as their gender identity and sexual orientation. While our three annotators were of diverse cultural background such diversity of human resources was not available for this work.
abstain method objective result method result other other other abstain abstain abstain abstain abstain objective method abstain abstain abstain abstain result objective result abstain abstain objective result abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain method method result method method abstain abstain abstain abstain abstain abstain method method method result result abstain abstain abstain result abstain abstain abstain method method result abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain method abstain abstain result objective abstain abstain method abstain abstain abstain method method method abstain result method abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain method abstain method abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result result abstain abstain other abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result method method abstain method abstain abstain abstain abstain abstain abstain other other other other other other abstain other method other other other objective method objective result objective abstain objective other other abstain abstain abstain abstain abstain abstain abstain abstain