The element of repetition in cyberbullying behavior has directed recent computational studies toward detecting cyberbullying based on a social media session . In contrast to a single text, a session may consist of an initial post and an associated sequence of comments. Yet, emerging efforts to enhance the performance of session-based cyberbullying detection have largely overlooked unintended social biases in existing cyberbullying datasets. For example, a session containing certain demographic-identity terms (e.g., gay or black) is more likely to be classified as an instance of cyberbullying. In this paper, we first show evidence of such bias in models trained on sessions collected from different social media platforms (e.g., Instagram). We then propose a context-aware and model-agnostic debiasing strategy that leverages a reinforcement learning technique, without requiring any extra resources or annotations apart from a pre-defined set of sensitive triggers commonly used for identifying cyberbullying instances. Empirical evaluations show that the proposed strategy can simultaneously alleviate the impacts of the unintended biases and improve the detection performance. Cyberbullying has become a prevalent adverse behavior in online social interactions. Recent findings indicate that over 35% of young people have been victims of cyberbullying and roughly 15% have admitted to cyberbullying others (Hinduja and Patchin, 2020; Kim et al., 2021). The detrimental consequences of cyberbullying have motivated considerable efforts in various fields to combat cyberbullying. For example, in computational studies of cyberbullying detection  which have been largely aimed at classifying text posted on social media  Equal contribution     P ( label = bully | z ) ' HQ V L W \  R IW KH  & RQG L W L RQD O  3 U REDE LOL W \ ] ZLWKRXWVHQVLWLYH] VHQVLWLYH platforms with machine learning and natural language processing (NLP)  the primary goal is to improve the overall accuracy and speediness of detection. Partly due to an increased awareness of the repetitive nature of cyberbullying behavior, a number of recent efforts in cyberbullying detection have shifted in focus from classification of a single text to detection in a social media session. A session typically consists of an image/video with a caption, a sequence of comments, and other social content, e.g., number of likes. The promising results, nevertheless, may come from a deeply biased model that captures, uses, and even amplifies the unintended biases embedded in social media data (Zhang et al., 2020). That is, because humans are biased, human-generated language corpora can introduce human social prejudices into model training processes (Caliskan et al., 2017). Evidence of such bias has been found in toxicity detection (Zhang et al., 2020) and hate speech detection (Davidson et al., 2019), revealing that tweets in African-American Vernacular English (AAVE) are more likely to be classified as abusive or offensive. Similarly, a cyberbullying classifier may simply take advantage of sensitive triggers, e.g., demographic-identity information (e.g., gay) and offensive terms (stupid, ni***r), to make decisions. Indeed, we find that in the Instagram data for benchmarking cyberbullying detection released by (Hosseinmardi et al., 2015), 68.4% of sessions containing the word gay are labeled as bullying, 89.4% of sessions containing the word ni***r, and 64.3% of sessions containing the word Mexican. In Figure 1, we showcase differences in the performance of a standard hierarchical attention network (HAN) (Yang et al., 2016)  a commonly used model for session-based cyberbullying detection  and a HAN that was debiased using our proposed strategy in sessions with and without sensitive triggers using the benchmark Instagram data. Specifically, the x -axis represents the probability of the classifier predicting a session as bullying, i.e., the decision scores F : p ( label = bully | Z ) . The y -axis represents the conditional probability densities of the decision scores, i.e., p ( F| Z ) . Figure 1(a) shows that the densities are dependent on Z and the dependencies are largely reduced by our mitigation strategy, as depicted in Figure 1(b). This paper aims to mitigate the unintended bias in cyberbullying detection in social media sessions. Our task poses multi-faceted challenges that render recent model-agnostic research in fair text classification  especially, data manipulation methods (Dixon et al., 2018; Sun et al., 2019)  inapplicable. First, in contrast to a single text (e.g., a tweet), social media sessions with a sequence of comments contain rich contextual information. Bias mitigation cannot be defined without context (Lee et al., 2020). The axiomatic and absolute definitions may render current interventions (e.g., gender-swapping) ineffective and may even misguide cyberbullying classifiers. Second, session-based cyberbullying detection is a sequential decision-making process rather than a one-off operation. Therefore, current decisions made by a cyberbullying classifier can influence its future predictions and debiasing strategies. Third, these data manipulation methods are impractical in our task due to the need for extra data annotation, which is especially time-consuming for sequential social media data with rich context. In addition, these methods consider fairness through a differentiable loss function that may not directly incorporate specific fairness goals or measures. To address these challenges, we propose a context-aware and model-agnostic debiasing training framework for cyberbullying detection. It does not require additional resources, apart from a predefined set of sensitive triggers. In particular, drawing from recent advances in reinforcement learning (RL), we consider a classifier as an agent that interacts with the environment to accumulate experience in cyberbullying detection and bias mitigation. At each timestep, the agent makes decisions based on all comments observed up to that point in time and is updated by the collected feedback. Empirical evaluations on two real-world datasets show that the proposed debiasing framework can effectively mitigate the unintended biases while improving the performance of cyberbullying detection. Cyberbullying Detection. The growing prevalence of social networking sites and convenient access to digital devices and the internet have substantially expedited information-sharing processes. A byproduct of this, however, has been the increased vulnerability of young people, in particular, to one of the most serious online risks  cyberbullying. To help combat cyberbullying, researchers have used various techniques in machine learning and NLP to automate the process of cyberbullying detection. This is also evidenced by a number of recent competitions and workshops for related tasks such as detection of hate speech against immigrants and women (Basile et al., 2019), offensive language identification (Zampieri et al., 2020), and toxic spans detection (Pavlopoulos et al., 2021). Early works simplified the task as text classification, the input of which are content-based features (e.g., cyberbullying keywords) extracted from a single text (e.g., a tweet) and labels denoting whether the text is relevant to cyberbullying, see, e.g., (Di-nakar et al., 2011; Xu et al., 2012). To better leverage the rich information included in social media data, many studies proposed to augment textual features with emotion/sentiment (Dani et al., 2017), social network information such as relational centrality and ego networks (Squicciarini et al., 2015; Huang et al., 2014), and other multi-modal information such as location and time (Cheng et al., 2019b). Extensive experimental results revealed that the improvement of these approaches is significant. From the data perspective, research in cyberbullying detection has shifted from modeling a single text to multi-modal data and social media sessions. Underpinning these transitions is an increased recognition of two distinct characteristics of cyberbullying behavior  repetitiveness and power imbalance (Smith et al., 2008). To address these characteristics, studies such as (Cheng et al., 2019a, 2021) proposed to model the structure of a session and temporal dynamics among the comments using HAN. Yet, whereas numerous studies have focused on achieving better prediction performance, these approaches tend to carry or reinforce the unintended social biases in the datasets (Gencoglu, 2020). Our work thus complements earlier research by examining and mitigating unintended bias in cyberbullying detection models. Fairness in NLP. Humans are inherently biased, and many studies have revealed human biases and discrimination in natural language (Garg et al., 2018; Jentzsch et al., 2019). Evidence has, for instance, emerged in biased pre-trained word embeddings and semantics derived from language corpora. However, in the field of NLP, the question of how to alleviate bias and promote fairness has only more recently begun to be addressed. Using text classification tasks as an example, one predominant method to make the classifiers fairer is to balance training data in a statistical sense. In particular, one can augment original data with external labeled data (Dixon et al., 2018). Similar methods include data oversampling/downsampling, sample weighting (Zhang et al., 2020), and identity term swapping (Park et al., 2018). Dixon et al. (Dixon et al., 2018) added non-toxic samples containing identity terms from Wikipedia articles into training data. A similar strategy was used in (Nozza et al., 2019) for misogyny detection. Badjatiya et al. (Badjatiya et al., 2019) proposed to replace sensitive words with neutral words or tokens. This balancing strategy, while convenient and easy to implement, is not compatible with session-based cyberbullying detection. First, practical considerations impede us from providing additional labeled data with specific sensitive triggers. Data labeling for session-based cyberbullying detection is especially time-consuming and labor-intensive, given that it requires carefully examining a media object and all associated comments in a social media session. Second, because there are potentially many words or tokens sensitive to cyberbullying, identity term swapping is almost impossible. Third, social media sessions contain sequences of comments that provide contextual information important for both cyberbullying detection and bias mitigation. Simple data augmentation can result in the significant loss of such information. Lastly, balancing can introduce additional calibration parameters that can impair classification performance and bias mitigation (Gencoglu, 2020). Cyberbullying is often characterized as a repeated rather than a one-off behavior (Smith et al., 2008). This unique trait has motivated research that focuses on the detection of cyberbullying in entire social media sessions . In contrast to a single text, e.g., a Facebook comment or a tweet, a social media session is typically composed of an initial post (e.g., an image with a caption), a sequence of comments from different users, timestamps, spatial location, user profile information, and other social content such as number of likes (Cheng et al., 2020). Session-based cyberbullying detection presents a number of characteristics such as multi-modality and user interaction (Cheng et al., 2020). In this work, because our goal is to mitigate bias in natural language, we focus on text (i.e., a sequence of comments) in a social media session. We formally define session-based cyberbullying detection as follows: Definition ( Cyberbullying Detection in a Social Media Session ) . We consider a corpus of N social media sessions C = { 1 ,  2 , ...,  N } , in which each session consists of a sequence of comments denoted as { c 1 , ..., c C } . A session is labeled as either y = 1 denoting a bullying session or y = 0 denoting a non-bullying session. Let D be the dimension of extracted textual features (e.g., Bag of Words) x i for c i . Session-based cyberbullying detection aims to learn a binary classifier using a sequence of textual data to identify if a social media session is a cyberbullying instance: F : { x 1 , ..., x C }  RD  { 0 , 1 } . An unbiased model for cyberbullying detection makes decisions based on the semantics in a social media session instead of sensitive triggers potentially related to cyberbullying, such as gay, black, or fat. In the presence of unintended bias, a model may present high performance for sessions with these sensitive triggers without knowing their semantics (Dixon et al., 2018). In this section, we first discuss how to define and assess bias in the context of session-based cyberbullying detection. Bias in a text classification model can be assessed by the False Negative Equality Difference (FNED) and False Positive Equality Difference (FPED) metrics, as used in previous studies such as (Zhang et al., 2020; Gencoglu, 2020; Huang et al., 2020). They are a relaxation of Equalized Odds (Borkan et al., 2019) and defined as FNED = (cid:88) z | FNR z  FNR overall | , (2) FPED = (cid:88) z | FPR z  FPR overall | , (3) where z denotes cyberbullying-sensitive triggers, such as gay, black, and Mexican. The complete list of sensitive triggers can be found in Appendix A. FNR overall and FPR overall denote the False Negative Rate and False Positive Rate over the entire training dataset. Similarly, FNR z and FPR z are calculated over the subset of the data containing the sensitive triggers. An unbiased cyberbullying model meets the following condition: P (  Y | Z ) = P (  Y ) , (4) where  Y stands for the predicted label. By Equation 4, we imply that  Y is independent of the cyberbullying-sensitive triggers Z that is, a debiased model performs similarly for sessions with and without Z . Note that the widely-used non-discrimination evaluation sets  Identity Phrase Templates Test Sets (IPTTS) (Dixon et al., 2018)  are not applicable to our task. IPTTS are generated by predefined templates with slots for specific terms, e.g., I am a boy and I am a girl. They only include examples for single text, whereas a social media session includes a sequence of comments. As we will show in subsection 5.1, the average number of comments in the Instagram dataset is 72, which can pose great challenges for generating synthetic social media sessions and the labeling process. Essentially, a debiasing session-based cyberbullying detection is a sequential decision-making process where decisions are updated periodically to assure high performance. In this debiasing framework, comments arrive and are observed sequentially. At each timestep, two decisions are made Agent Environmnet Session comment comment ... comment Data state (comments[ ]) action Reward Function action state Figure 2: Overview of the proposed model. based on the feedback from past decisions: (1) predicting whether a session is bullying and (2) gauging the performance differences between sessions with and without sensitive triggers. Our debiasing strategy is built on the recent results of RL (Shi et al., 2018; Zou et al., 2019; Mosallanezhad et al., 2019), particularly, the sequential Markov Decision Process (MDP). In this approach, an agent A interacts with an environment over discrete time steps t : the agent selects action a t in response to state s t . a t causes the environment to change its state from s t to s t +1 and returns a reward r t +1 . Therefore, each interaction between the agent and the environment creates an experience tuple M t = ( s t , a t , s t +1 , r t +1 ) . The experience tuple is used to train the agent A through different interactions with the environment. The agent's goal is to excel at a specific task, such as generating text (Shi et al., 2018) or summarizing text (Keneshloo et al., 2019). In this work, we leverage techniques in RL to alleviate the unintended bias when classifying social media sessions into bullying or non-bullying based on user comments. In particular, we consider a standard classifier F (e.g., HAN) as an RL agent and a sequence of comments observed at time { 1 , 2 , ..., t } as state s t . The agent selects an action a t  { non-bullying , bullying } according to a policy function  ( s t ) . ( s t ) indicates the probability distribution of actions a in response to state s t , whereas  ( s t , a t ) shows the probability of choosing action a t in response to state s t . The action can be interpreted as the predicted label  y using the input comments. The reward r t +1 is then calculated for the state-action set ( s t , a t ) and the cumulative discounted sum of rewards G t is used to optimize the policy function  ( s t ) . Below, we provide details of the (1) environment, (2) states, (3) actions, and (4) the reward function for the proposed debiasing approach. Environment is a session comments loader. At each episode, the environment chooses a single session and returns its first t comments as state s t . As such, states are independent from the agent's actions, as they do not affect the next state. When it reaches the maximum number of comments of the selected session C , the process is terminated. State s t is a sequence of comments in a social media session posted by various users from time 1 through time t . Action a t determines a session to be bullying or not, given the input comments or state s t : a t  { bullying , non-bullying } . (5)  Reward function R is used to optimize the policy function  ( s t , a t ) . It is defined based on how successfully the agent predicts the label for the input state s t and how much bias the classifier currently has. We define the bias of a classifier as the harmonic mean of FPED and FNED characterized by the sensitive triggers in cyberbullying. In a debiased classifier, we expect both FPED and FNED to be close to zero. We define the reward function R as R =  l F    2  FPED  FNEDFPED + FNED , (6) where l indicates the prediction error of the classifier and  balances between prediction and the debiasing effect of F . The reward function is calculated based on all sessions in the environment, evaluating the performance and bias of the classifier. Given the environment, state, actions, and the reward function, we aim to learn the optimal action selection strategy  ( s t , a t ) . At each timestep t , the agent classifies a session with t comments and the reward r t +1 is calculated using Equation 6, according to the agent's action a t and state s t . The goal Algorithm 1 The Optimization Algorithm Require: The dataset { x , z, y } , initialized   ( s 0 , a 0 ) , discount rate  , balancing weight  , learning rate lr , number of episode E . of the agent is to maximize its reward according to Equation 6. We use the policy gradient algorithm  REINFORCE (Sutton et al., 1999)  to train the agent. As such, the agent has similar properties to a classifier and the classifier's output distribution can be mapped to the agent's policy function  ( s t , a t ) . We use the following function to update the agent:   = lr   L (  ) , (7) where lr denotes the learning rate,  is the parameter w.r.t. the policy function   ( s t , a t ) , and L (  ) indicates the policy loss: L (  ) = log(   ( s t , a t )  G t ) , (8) where G t = (cid:80) ti =1  i r i +1 is the cumulative sum of rewards with discount rate  . The pseudo-code for the optimization algorithm can be seen in Algorithm 1. 5 Evaluation In this section, we conduct both quantitative and qualitative evaluations to examine the efficacy of our debiasing strategy. 1 In particular, we show that our method can effectively mitigate the impacts of unintended data biases without impairing the model's prediction performance by answering: 1 The source code is publicly available at https://github.com/GitHubLuCheng/MitigateBiasSessionCB Table 1: Statistics of the Instagram and Vine datasets. (1) Can we mitigate the unintended bias of machine learning models for detecting cyberbullying sessions by leveraging techniques in RL? (2) If so, will this debiasing strategy impair the cyberbullying detection performance? and (3) If no' to (2), what is the source of gain? Two benchmark datasets for cyberbullying detection  Instagram (Hosseinmardi et al., 2015) and Vine (Rafiq et al., 2015)  are used for empirical evaluation. The number of sessions in Instagram and Vine is 2,218 and 970, respectively. Both datasets were crawled using a snowball sampling method and manually annotated via the crowd-sourcing platform CrowdFlower. 2 Sessions containing less than 15 comments were removed to ensure data annotation quality. Annotators were asked to examine the image/video, associated caption, and all of the comments in a session before making the final decisions. Instagram: Instagram 3 is a social networking site ranked as one of the top five networks with the highest percentage of users reporting experiences of cyberbullying (the Label Anti Bullying Charity, 2013). Each social media session consists of image content, a corresponding caption, and a sequence of comments in temporal order. In total, this dataset is composed of 2,218 sessions, with an average number of 72 comments in each session. Vine: Vine 4 was a mobile application that allowed users to upload and comment on six-second looping videos. Each social media session consists of video content, the corresponding caption, and a sequence of comments in temporal order. This dataset contains 970 sessions and each session contains, on average, 81 comments. For social media sessions, standard fairness methods, such as identity swapping and data supplementation, are not applicable. We compare our approach with commonly used machine learning 2 models for classification with sequential text data, including HAN, Convolutional Neural Network (CNN), and Gated Recurrent Unit (GRU), as well as a recent model proposed for session-based cyberbullying detection  HANCD (Cheng et al., 2019a). HANCD leverages multi-task learning to jointly model the hierarchical structure of a social media session and the temporal dynamics of its sequence of comments to improve the performance of cyberbullying detection. We also include the state-of-the-art model Constrained (Gencoglu, 2020) that imposes two fairness constraints on cyberbullying detection to mitigate biases. In our implementation, we use the HANCD classifier as the cyberbullying model in Constrained for a fair comparison. The parameter w.r.t. the fairness constraints is set to 0.005, as suggested. Both HAN and HANCD use GRU to extract the context of the input data. We use 1-layer GRUs with a hidden size of 100 and 200 neurons for word and comment attention networks, respectively. As our approach is model-agnostic, for each standard machine learning model, there is a corresponding debiased counterpart. For the proposed method, l F in the reward function (Equation 6) is computed as the cross entropy loss between the true label y and the predicted probability p : l F =  1 2 2 (cid:88) i =1 y i log( p i )+(1  y i ) log(1  p i ) . In Algorithm 1, the classifier F is pre-trained for 5 iterations using loss function l F , learning rate 3 e  3 , and the Adam optimizer (Kingma and Ba, 2014). F is then placed in the RL setting discussed in subsection 4.2. We apply the REINFORCE method with E = 500 episodes, learning rate 1 e  5 ,  = 1 . 0 , and  = 0 . 5 using the Adam optimizer to further update the classifier. Evaluations focus on both the prediction accuracy and the debiasing effect of a model. For prediction performance, we adopt standard metrics for binary classification, including Precision, Recall, F1, and AUC scores. Following (Zhang et al., 2020; Gencoglu, 2020), we use FPED, FNED, and total bias (FPED+FNED) to evaluate how biased a model is w.r.t. sessions with and without sensitive triggers. Lower scores indicate less bias. For all models, pre-trained GloVe word embeddings (Pennington et al., 2014) and 10-fold cross validation with 80/20 split are used for fair comparison. Furthermore, we perform McNemar's test to examine whether a statistically significant difference between baseline and debiased models exists in terms of cyberbullying classification accuracy and equity. The best results are highlighted in bold font. In this section, we show experimental results to answer the first question: Can the proposed framework mitigate unintended bias? As expected, the proposed RL framework can effectively mitigate the impact of the unintended bias embedded in the datasets for cyberbullying detection. We report results for both Instagram and Vine in Table 2. Dedenotes a debiased model, e.g., De-HAN is a HAN debiased by the proposed RL framework. Total stands for the total bias (FPED+FNED). All McNemar's tests resulted in statistical significance with p -values < 0 . 05 . We observe the following: (1) Compared to the standard classifiers, the debiased counterparts sig-nificantly improve FNED and FPED scores, indicating that our proposed debiasing strategy can mitigate the unintended bias in data used for predicting cyberbullying sessions, regardless of the dataset or machine learning model. For example, when tested on Instagram with the HAN model, our debiasing method can decrease FPED, FNED, and total bias by 95.7%, 56.7%, and 57.0%, respectively. For Vine , the improvement with HAN is 71.4%, 3.3%, and 50.5%, respectively. (2) Total biases of standard classifiers come from both the FPRs and FNRs for the Instagram experiments, while the main contributor of biases is the FPRs for the Vine experiments. Our approach mitigates total bias in both scenarios. (3) Our debiasing strategy based on RL techniques is also more effective than the fairness constraints proposed in (Gencoglu, 2020), as indicated by the decreased total biases for both Instagram and Vine . By comparing HANCD, Constrained, and De-HANCD, we see that Constrained decreases FPED by sacrificing FNED, while De-HANCD can decrease both. In addition to the quantitative results, we provide qualitative analyses by visualizing FPED and FNED of both the standard and debiased HANCD models. In an experiment with Instagram for sessions containing ten sensitive triggers, as illustrated in Figure 3, we can observe that compared to De-HANCD, HANCD is more biased toward some sensitive triggers, such as fat and stupid. Demographic-identity related bias is also detected in HANCD. For example, sessions containing identity terms including ne**o, gay, and ni**a are more likely to be falsely identified as bullying, as indicated by FPED. By contrast, De-HANCD mitigates various types of unintended biases and has more consistent performance across all of the sensitive triggers. A dilemma often faced by researchers studying bias and fairness in machine learning is the trade-off between fairness and efficiency (Bertsimas et al., 2012). Under this trade-off theory, forcing cyberbullying classifiers to follow the proposed debiasing strategy would invariably decrease the accuracy. This section shows that, somewhat coun-terintuitively, our approach can outperform biased models w.r.t. overall cyberbullying detection acTable 3: Performance comparisons of different models on the Instagram dataset. Higher AUC, precision (PREC), recall (REC), and F1 scores indicate better performance. p -value < 0 . 05 for all McNemar's tests. the data. Results are presented in Tables 3-4. We see that the proposed debiasing strategy can both alleviate the bias and retain high prediction accuracy. For instance, for Instagram , our approach achieves the highest AUC and F1 score of all evaluated models. For Vine , the improvement of De-HAN over HAN is 9.8% and 41.9% for AUC and F1 score, respectively. The improvement over Constrained is 15.8% and 15.4%, respectively. Biased models present much lower Precision than Recall for Vine . This result is in line with the findings in Table 2, where we observe that the larger bias component is associated with FPRs in Vine . This indicates that when the sample size is small, these models overfit to sensitive triggers for detecting bullying instances. The debiasing strategy effectively reduces models' reliance on those terms and utilizes contextual information for prediction. (b) Performance w.r.t. cyberbullying detection. accuracy? This non-compromising approach may be attributed to the proposed RL framework that effectively captures contextual information. In this section, we examine the impact of parameter  in Equation 6 by varying   { 0 . 0 , 0 . 2 , 0 . 4 , 0 . 6 , 0 . 8 , 1 . 0 } . We show performance w.r.t. bias mitigation (total bias) and cyberbullying detection (F1 score) in Figure 4. The results clearly show the efficacy of the proposed RL framework for bias mitigation. In particular, as we increase  , the RL agent puts more effort toward alleviating biases by minimizing both FPED and FNED simultaneously. Moreover, by interacting with the environment, the RL agent also leverages contextual information in order to minimize the prediction error and receive a larger reward. As a result, the RL agent largely reduces biases while improving the prediction accuracy, as shown by the slight increase in detection performance of the classifier in Figure 4b. In this work, we examined unintended biases in datasets for session-based cyberbullying detection. In contrast to conventional data for bias mitigation in text classification, social media sessions consist of a sequence of comments with rich contextual information. To alleviate these unintended biases, we propose an effective debiasing strategy by leveraging techniques in RL. Our approach is context-aware, model-agnostic, and does not require additional resources or annotations aside from a predefined set of potentially sensitive triggers related to cyberbullying. Empirical evaluations demonstrated that our approach can mitigate unintended bias in the data without impairing a model's prediction accuracy. Other types of decisions in sequential decision-making processes can impact the underlying user population, thereby influencing future comments generated by users. Future research can be directed towards studying the long-term impact of the debiasing strategy, as well as investigating different types of biases in session-based cyberbullying detection, such as gender bias, racial bias, and language bias. Our approach can also benefit from integrating previous studies that use data augmentation or swapping methods to counteract bias. Due to the challenges of data collection and labeling, validating our approach on datasets across different social media platforms is also an important avenue for future work. This work seeks to advance collaborative research efforts aimed at mitigating bias in session-based cyberbullying detection, a topic that has yet to be studied extensively. Here, we provide preliminary solutions, but more work is needed to elucidate ways to build debiased and effective models. While all data used in this study are publicly available, we are committed to securing the privacy of the individuals in our datasets. To this end, we automatically replaced user names with ordered indices in our analysis. The insulting or offensive terms and the figures used in this paper are for illustrative purposes only and do not represent the views or ethical attitudes of the authors. This material is based upon work supported by the National Science Foundation (NSF) Grants 1719722 and 2036127.
abstain abstain abstain abstain objective objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain objective abstain abstain abstain abstain abstain abstain abstain abstain objective abstain method abstain abstain other other other other other other other other other other other other other method other other other other other other other other other other other objective other other other other other abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain method abstain abstain abstain method abstain method method other method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain method method abstain abstain abstain method abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective abstain abstain method abstain abstain method method method abstain other