The problem of detecting psychological stress in online posts, and more broadly, of detecting people in distress or in need of help, is a sensitive application for which the ability to interpret models is vital. Here, we present work exploring the use of a semantically related task, emotion detection, for equally competent but more explainable and human-like psychological stress detection as compared to a black-box model. In particular, we explore the use of multi-task learning as well as emotion-based language model fine-tuning. With our emotion-infused models, we see comparable results to state-of-the-art BERT. Our analysis of the words used for prediction show that our emotion-infused models mirror psychological components of stress. As crises have begun to multiply worldwide, including the COVID-19 pandemic and the resulting economic downturn, psychological stress has risen dramatically 1 . The problem of detecting psychological stress, and more broadly, of detecting people in distress and in need of help, is a sensitive application; therefore, the ability to interpret the results, in order to understand why, is vital. The consequences of blindly trusting a black-box model and mislabeling users' stress levels could be serious in a deployed application such as a therapeutic chatbot, where some users may not receive the immediate help they need. Furthermore, models that make decisions based on psychology theory about factors that impact stress will be easier for humans to understand, and their mistakes will be more obvious. Researchers have recently begun to study psychological stress, but in this work, we propose a new focus on examining the information our models use to make decisions and finding ways to incorporate psychological factors, like emotion, into them. To approach the problem of stress detection, which has much less labeled data than many popular classification tasks, we first note that stress has been shown to interact with emotion (Lazarus, 2006; Thoern et al., 2016; Levenson, 2019), a task that has far more publicly available labeled data. For example, individuals who are stressed are likely to express emotions such as fear, sadness, or anger and unlikely to express emotions such as happiness. Traditional multi-task learning would normally be helpful in this situation, but there are no currently available datasets labeled with both stress and emotion. Even if there were, it would be bene-ficial to incorporate external information without re-labeling new datasets for each new combination of useful tasks. Here, we present work exploring how to use semantically related taskshere, emotion detectionto create emotion-infused models capable of equally competent, but explainable, psychological stress detection as compared to a black-box model. In particular, we explore the use of multi-task learning as well as emotion-infused language model fine-tuning, two existing frameworks which we examine through the lens of interpetability. Our code for this work is available at github. com/eturcan/emotion-infused . Our contributions in this work are as follows: (i) consideration of factors suggested by psychological theory in deep learning methods for predicting stress, with a focus on emotion; (ii) an exploration of three different approaches to emotion-infused models, with experimental results showing comparable results to the state-of-the-art in all cases; and (iii) a framework for interpreting our models to show the impact of emotion and other factors in our models. Researchers who use natural language approaches for stress detection often rely on external resources such as diagnostic questionnaires (e.g., Guntuku et al. (2018)) or techniques like pattern matching (patterns such as I am stressed, e.g., Winata et al. (2018); Lin et al. (2017)) to assign labels. Much of the work that has been done on psychological stress detection focuses either on establishing baseline models with little advancement in computational modeling, or on using external information about the text (e.g., author, time of posting, number of replies), which is usually, but not always available and may differ in meaning or importance across platforms and domains. There has also been a substantial amount of work on detecting related mental health concerns such as anxiety (e.g., Shen and Rudzicz (2017); Gruda and Hasan (2019); Jiang et al. (2020)), but these are distinct from the generalized experience of stress. The most similar work to ours is Turcan and McKeown (2019), our prior work publishing a dataset of psychological stress collected from the social media website Reddit and labeled by crowd workers, and presenting baselines with several basic non-neural and BERT-based models on this data. We use this dataset in our current work; however, we focus on exploring interpretable frameworks for this sensitive task and connecting the stress detection task concretely with emotion detection. The models we propose in this work rely on two types of enhancements to the neural representation learned by models like BERT: multi-task learning and pre-training or fine-tuning. Multi-task learning is an increasingly popular framework in which some parameters in a model are shared between or used to inform multiple different tasks. Hard parameter sharing (Caruana, 1993), the variant we employ, uses some set of parameters as a shared base representation and then allows each task to have some private parameters on top and perform their own separate predictions. Multi-task learning has been successfully applied to many domains across NLP (Sun et al., 2019; Kiperwasser and Ballesteros, 2018; Liu et al., 2019); we are especially interested in instances where it has improved semantic and emotion-related tasks, such as Xu et al. (2018), who perform emotion detection with a suite of secondary semantic tasks including personality classification. Pre-training and fine-tuning are another type of transfer learning where multiple tasks are trained in sequence rather than at the same time. Pre-trained language models are perhaps the most widely used example, where a large neural language model can Dataset Size Dreaddit 3,553 GoEmotions A,E,S 58K GoEmotions FSJ 4,136 Vent 1.6M Table 1: The datasets we use in this work and their relative sizes (in terms of total number of data points). be fine-tuned for many different tasks (Devlin et al., 2019). Additionally, continuing to pre-train the language model itself on language from the target domain has been shown to improve performance (Howard and Ruder, 2018; Chakrabarty et al., 2019; Gururangan et al., 2020) (also note Chronopoulou et al. (2019), who perform this task at the same time as the target task, in a form of multi-task learning). This methodology has been successfully extended to other domains, in which a model is first fine-tuned on some large, broadly useful task and then further fine-tuned for a smaller target task (e.g., Felbo et al. (2017), who first fine-tuned on emoji detection and then fine-tuned on target semantic tasks including emotion and sentiment detection). It should be noted that the psychological stress is much better studied in settings where researchers have access to some physiological signals (e.g., Zuo et al. (2012); Allen et al. (2014); Al-Shargie et al. (2016); Kumar et al. (2020); Jaiswal et al. (2020)). This work is not as relevant to our task, since we have only text data available when detecting stress from online posts. A comparison of all the datasets we use in this work can be seen in Table 1. The primary dataset we use for this work is Dreaddit (Turcan and McKeown, 2019), a dataset of 3,553 segments of Reddit posts from various support communities where the authors believe posters are likely to express stress. The stress detection problem as expressed in this dataset is a binary classification problem, with crowdsourced annotations aggregated as the majority vote from five annotators for each data point. We note that this paper frames the stress classification problem in terms of the author and the timei.e., a post is labeled stressful only if the poster themselves is currently expressing stress. Because this dataset is small for training a deep learning model, we also experiment with larger datasets to provide auxiliary information. We select the GoEmotions dataset (Demszky et al., 2020), which consists of 58,009 Reddit comments labeled by crowd workers with one or more of 27 emotions (or Neutral), for its larger size and genre similarity to Dreaddit. In this paper, we refer to the dataset in this form as GoEmotions all or GoEmotions A . The authors also published two relabelings of this dataset, achieved by agglomerative clustering: one where labels are clustered together into the Ekman 6 basic emotions (anger, disgust, fear, joy, sadness, surprise, neutral) (Ek-man, 1992) (GoEmotions Ekman/E ), and one into simple polarity (positive, negative, ambiguous, neutral) (GoEmotions sentiment/S ). We run our experiments with each version of this dataset. We also explore the use of another social media website, Vent. Vent is a platform more similar to Twitter or Tumblr than Reddit, where users post vents of any length and tag them as they like, and other users react to them or post comments. The benefit of Vent for this purpose is that posters self-identify some emotion they are feeling from a large list of pre-made emotions. The data we use is collected by Malko et al. (2021) 2 . We select Vent data that has been labeled with fear or sadness, which we hypothesize to be related to stress, as well as joy, for a contrast. We note that this dataset is strictly single-class, whereas GoEmotions may have more than one emotion label per data point. In all, there are 1.6M vents in our dataset, much larger than Dreaddit or GoEmotions; we randomly sample this data in a stratified manner to create a training, development, and test set with an 80/10/10 ratio. To examine the effects of domain similarity, we also select a subset of GoEmotions with the corresponding emotion labels  we subsample the existing all dataset to select only data points originally labeled with fear, joy, or sadness, for a final set of 4,136 data points (3,342 of which are the train set). We call this subset GoEmotions FSJ , and we compare it against Vent to see whether genre similarity or data size is more important in this multitask setting. We experiment with three types of emotion-infused models; that is, we present three different ways to incorporate emotion information into our stress detection models, divided into multi-task learning and fine-tuning. 2 Due to license and ethics policy restrictions, we currently do not make this data publicly available. Our first multi-task models, which we refer to as Multi Alt , are simply two single-task models sharing the same base BERT representation layers. The models are alternating in that we train them with two datasets with two different sets of labelsi.e., we train the stress task with the Dreaddit data and the emotion task with the GoEmotions or Vent data. We refer to the variants with a subscript, i.e., Multi AltGoEmotions A (i.e., GoEmotions with all emo-tions), Multi AltGoEmotions E (i.e., the Ekman GoEmotions relabeling), Multi AltVent (i.e., the Vent data), etc. The Multi Alt models can be seen in Figure 1a. One loss step for these models consists of only one dataset and task, so they are trained with the negative log-likelihood (NLL) loss for single-label tasks (Dreaddit, Vent, GoEmotions FSJ ) and the binary cross-entropy (BCE) loss for multi-label tasks (GoEmotions A,E,S ). We also experiment with a multi-task learning setup where we perform the two tasks at the same time on the same input data . We call this architecture Multi. However, because the Dreaddit data is labeled only with stress, we first separately train BERT models on the various versions of GoEmotions and use them to predict emotion labels for Dreaddit. We then take these emotion labels to be silver data and train on them alongside stress. The Multi model can be seen in Figure 1b. Since stress detection is our main task in this work, we focus on this task where we have gold labels for stress, but note that it will be interesting in future work to experiment with other task settings, such as whether stress detection can improve emotion classification. In these models, the losses of the stress task and the emotion task are summed together for each batch using a tunable weight parameter, i.e., L =  L stress + (1   ) L emotion . We experiment with models in which we first endow the BERT representation with knowledge of the emotion task by fine-tuning and then apply it to stress detection (as in Phang et al. (2018)). We perform a sequential version of the Multi Alt models, in which we fine-tune a pre-trained BERT language model on another task, and then extract the language model parameters to initialize a BERT model that we continue to fine-tune on Dreaddit. We denote these models as, e.g., Fine-Tune GoEmotions A (cid:1) Dreaddit for a model that was first trained on GoEmotions all and then on Dreaddit (for space, we will abbreviate Fine-Tune as FT). These fine-tuning models can be seen in Figure 1c. These models are trained with the NLL and BCE losses as in the Multi Alt models. We present a re-implementation of the same BERT-based fine-tuning model used in Turcan and McKeown (2019), where this model performed best on Dreaddit. We report this as an average of 3 runs with distinct random seeds, and our results are, on average, lower than the single model reported, but with high variance. Because of this, we assume that the previously reported performance is from the high end of this variance and use our average score as our baseline in this work. This model is a pre-trained BERT language model (released as bert-base-uncased by Wolf et al. (2019); we use this same pre-trained language model as the basis for all our models) followed by a dropout layer and a dense classification layer. We also report a recurrent neural network (RNN) model, which uses either a long short-term memory network (LSTM) (Hochreiter and Schmidhuber, 1997) or a gated recurrent unit (GRU) (Cho et al., 2014) in place of the transformer from BERT and is otherwise the same. These models are trained with the NLL and BCE losses as with the Multi Alt models. We train all of our models with minibatch gradient descent using the Adam optimizer (Kingma and Ba, 2015) with a batch size of 16, given GPU space constraints. We perform gradient clipping to 1.0 to prevent exploding gradients. When training any model, we perform early stopping based on the F1 score on the Dreaddit development set and select the model parameters from the epoch that achieved the best development score for our final evaluated model. We tune hyperparameters for all our models using Bayesian Optimization from the Python library ax 3 . All models train the initial learning rate of the Adam optimizer and the dropout probability before the final classification layer; the Multi models also tune the loss weight parameter  , and we also note that the RNN model tunes additional parameters such as the type of RNN, hidden dimension, etc. For all models, we tune parameters based on the F1 score on the Dreaddit development set; we train an ensemble of three models with three different, fixed random seeds and average their performance for a given parameter setting. We report the mean and standard deviation of three models, with three 3 https://github.com/facebook/Ax Model Binary F1 Accuracy RNN 67.58  1.22 68.86  1.10 BERT 78.88  1.09 79.11  1.32 Multi AltGE A 79.02  0.35 79.72  0.69 Multi AltGE E 80.24  1.39 81.07  1.13 Multi AltGE S 79.46  1.05 79.86  0.50 Multi AltGE FSJ 79.17  0.61 78.69  1.86 Multi AltVent 80.34  1.39 79.67  2.03 Multi Dr S 78.97  0.24 78.55  0.07 Multi Dr FSJ 78.90  0.59 78.55  0.07 Table 2: Results of our multitask models. different random seeds, trained with the best hyperparameters. More details about hyperparameter tuning can be found in the appendix. We report the results of our multi-task models in Table 2 4 . In general, our Multi Alt models perform similarly, and outperform the Multi models; we assume this is due to the introduction of noise in labeling the silver emotion data. Of these models, Multi AltVent performs best. With regards to GoEmotions, the 28-way classification of GoEmotions A naturally leads to lower numerical performance than the tasks with smaller numbers of classes, and we expect that GoEmotions S may group too many distinctly labeled emotions together under the same emotion labels; it seems GoEmotions E is the happy medium for this model. We also note that the Multi AltVent and Multi AltGoEmotions E models perform equally well, which indicates that the genre mismatch is not an issue for this problem, or that Vent has a similar enough genre to Reddit that it does not affect the results. Somewhat surprisingly, Multi AltGoEmotions FSJ does not do as well as 4 We did compute statistical significance by calculating the majority vote of each of the models' 3 runs and using the approximate randomization test, but no model is significantly different from BERT. Multi AltVent ; however, the GoEmotions data is much smaller than Vent, especially when subsampled to select specific emotions. We further report the results of our fine-tuning models in Table 3. Because we expect that genre similarity should play a larger role when the secondary task can offer no direct training signal during the primary task fine-tuning, we evaluate on GoEmotions here and not Vent. Here, we observe that our best model, Fine-Tune GoEmotions FSJ (cid:1) Dreaddit , scores at least one standard deviation above BERT. We see higher increases in performance for the simpler classification problems in GoEmotions S and GoEmotions FSJ and worsened performance for GoEmotions A , suggesting that in the sequential paradigm, more complex tasks are not able to interact appropriately with the main task and instead interfere. We also report the performance of the fine-tuning BERT models we trained on GoEmotions in order to label Dreaddit with emotion in Table 4; these results track well with the fine-tuning results reported by Demszky et al. (2020). Because these models are intermediates used for labeling, we report the F1 scores of the single model we actually used for labeling, although we tuned their parameters with an average of 3 different instances as with all other models. Many-way classification problems have much more opportunity for error and noise in an already-noisy process of labeling unlabeled data, so we use only the two best-performing GoEmotions models, which are those trained on the fewest-label datasets, GoEmotions S and GoEmotions FSJ , for our Multi models. Overall, the inclusion of emotion information results in modest improvements, even though not statistically significant, as compared to BERT. However, our true goal in this work is to analyze the explainability of all of these models, to which we turn next. We perform three different analyses to probe our trained models and discover what information they learn to use. For our Multi Alt models, we investigate the usefulness of the emotion prediction layers in explaining stress classifications, and for all models, we use Local Interpretable Model-agnostic Explanations (LIME) (Ribeiro et al., 2016) to show that our emotion-infused models rely on meaningfully different types of words than BERT in order to make their predictions. We perform an analysis of our Multi Alt models to see what information they learn about emotion 5 We take the development sets of each of the datasets (Dreaddit and GoEmotions) and predict their labels under the other task (i.e., emotion for Dreaddit and vice-versa). We report the correlation of these predicted labels with the gold labels in Table 5 6 . In this case, the GoEmotions FSJ variant is a single-label three-way classification problem, so we report the correlation ratio  (Fisher, 1938). The other GoEmotions variants are multi-label, so we report the coefficient of determination R 2 (Cohen et al., 2015). We further present breakdowns of the correlations per emotion category for the polarity and 5 We did perform an equivalent analysis on the Multi models, which shows similar trends, but as Multi Alt shows better performance, we omit it for space. 6 We also note the possibility that different combinations of emotions are relevant to stress; however, not enough of our data is labeled with multiple emotion labels (4% of Dreaddit's silver labels from GoEmotions S , 9% of GoEmotions E ) to test this hypothesis in this work. FSJ subsets of GoEmotions in Table 6 and include the All and Ekman sets as well as the Vent data in the appendix. We observe that our multi-task models generally learn a moderate correlation between the stress labels and the emotion labels; they learn that negative emotions like fear and sadness are linked to stress and neutral or positive emotions are linked to non-stress, which makes intuitive sense. These emotion predictions can help explain the stress clas-sifier's predictions; imagine, for example, showing a patient or clinician that the patient's social media shows a strong pattern of fear and anger as a more detailed explanation for places a stress classifier detects stress. From a machine learning perspective, this correlation also suggests the potential for using emotion data as distantly-labeled stress data to supplement the small extant stress datasets. We also investigate the types of information each model is using to make its decisions. In this section, we use the Linguistic Inquiry and Word Count (LIWC) (Pennebaker et al., 2015), a hand-crafted lexicon which collects words belonging to psychologically meaningful categories like positive emotion and cognitive processes, to categorize the information our different models use to predict stress. We first analyze the unigrams our various models use to perform stress classification using LIME. LIME accepts an input from our development set, perturbs it in the bag-of-unigrams space, and runs one of our classifiers on each perturbation to calculate the importance of various unigrams; through LIWC BERT Multi AltGE E Multi AltVent Multi Dr FSJFTGEFSJ (cid:1) Dr Affective Processes 19% 22% 19% 16% 22% Positive Emotion 8% 10% 9% 9% 12% Anger 31% 40% 30% 25% 31% Cognitive Processes 16% 17% 17% 17% 17% Certainty 8% 13% 12% 16% 11% Perceptual Processes 17% 15% 14% 14% 15% Biological Processes 15% 19% 17% 16% 17% Achievement 17% 19% 19% 13% 17% Table 7: A comparison of how often several of our models rely on words from several LIWC categories to make their decisions, according to LIME. this process, we acquire the 10 unigrams with the highest magnitude output by LIME for each development example and consider them explanations. We thus have 2,760 individual unigram explanations for the entire development set to analyze. We then use the word lists from LIWC 2015's 72 psychological categories to see what types of words each classifier tends to use to make decisions of stress vs. non-stress. An abbreviated list of results showing our best models from each category is shown in Table 7 7 . We observe small but consistent effects suggesting that, in comparison to the basic BERT model, our emotion-enhanced models broadly learn to use the following information: Affective information . Most emotion-infused models except for Multi learn to use affective information, which includes both positive and negative emotion words, more often. We see the largest increase in anger, one of the emotions we had identified as relevant to stress, for Multi AltGoEmotions E , which makes intuitive sense because anger is one of the Ekman six basic emotions and thus, is explicitly predicted by this model. Cognitive processes . All models show some increase in using words related to cognitive processes as compared to BERT; however, its subcategory Certainty, which includes words about absoluteness such as never , obvious , and clearly , shows larger changes. For example, Multi Dreaddit FSJ uses Certainty twice as often as BERT. These cognitive words seem to target the mental aspects of stress. Rumination and a focus on absoluteness are known signs of anxiety disorders, an extreme form of chronic stress (Nolen-Hoeksema et al., 2008; Miranda and Mennin, 2007). Additional differences . We observe other, 7 More detail on the full table is available in the appendix. smaller patterns among LIWC usage for these models. For example, the Multi Alt models use the most achievement-oriented words (although most models show modest increases), suggesting that this information, which includes words about success and failure, is relevant to emotion and to stress. This makes sense, since failing to achieve (e.g., failing a class) can be a major stressor. We also see larger proportions of biological process words used by all emotion-infused models. We suggest this is because Dreaddit includes posts taken from Reddit communities about anxiety and PTSD, where posters are likely to describe their physical and mental symptoms while seeking help. We then investigate the data itself for highly significant words using the measure of relative salience proposed by Mohammad (2012), RelativeSalience ( w | T 1 , T 2 ) = f 1 N 1  f 2 N 2 . That is, it measures the importance of a token w in two different corpora T 1 , T 2 by subtracting their two relative frequencies (where f 1 , f 2 are the counts of token w in each corpus and N 1 , N 2 are the total tokens in each corpus). We compute this measure for all words in the Dreaddit training data, taking our two corpora to be the subsets labeled stress and not-stress. We take the top 200 unigrams for each label (stress as opposed to non-stress and vice-versa) and provide some examples in Table 8 with the full list of words available in the appendix. We examine the words and divide them into related groups in order to understand what types of information should theoretically be most important to classifying the data. For example, we see that different sets of function words are actually among the most important for both classes, with words like conjunctions typically appearing more indicative of stress Category Example Words Stress Function Words and, but, how, like, no, not, or, where, why Negative Sentiment awful, bad, cry, fear, hate, stress, stupid Helplessness alone, can't, nothing, nowhere, trying Non-Stress Function Words a, for, if, some, the, was, who, will, would Positive Sentiment amazing, best, good, great, hope, nice Support email, helped, support, thank, together, we Table 8: Some examples of words identified by relative salience on the Dreaddit training data as indicative of stress or non-stress. (which echoes Turcan and McKeown (2019)'s finding that stressful data is typically longer with more clauses), while non-stress includes words expressing future-thinking like if , will , and would . We also naturally find negative words for stress and positive words for non-stress, as well as a dichotomy of isolation and helplessness for stress vs. support and community for non-stress which is supported by psychological literature (Grant et al., 2009). We then look at the intersection between relative salience and LIME explanations, counting how many LIME explanations are highly salient words for stress or non-stress; abbreviated results are shown in Table 9 and the full table is available in the appendix. We see that our emotion-infused models learn to rely more often on words identified as indicative of non-stress, the minority class, instead of stress, the majority class. We note that the presented models do sometimes make some new errors when incorporating emotional information, and that while these methods successfully incorporate such information with no feature crafting, some further innovation may be needed in order to use this information optimally. For example, we reproduce an example from our development set, with profanity censored: And everyone was passive aggressive. The manager tried to peg down my salary multiple times like a f**king haggler at a market. Anyway, I decided to go get some antidepressants and the bottle fell out of my pocket, a coworker noticed and reported it to my boss. Who smiled and asked if there was anything I'd like to tell her. The passive aggressive s**t really got to me, and then I realized that I was being illegally paid. The annotators for Dreaddit label this post not stress, presumably because there is not enough context for how the poster feels about this story presently, and the poster conveys more anger than anything else. The LIME explanations for the BERT model, which labels this correctly, include some profanity, but largely focus on function words. However, all four of our Multi AltGoEmotions models misclassify this example as stressed and rely on words like aggressive (from passive aggressive ) and the profanity to do so. Meanwhile, the emotion classifiers of our Multi AltGoEmotions models are misled by words like smiled and label this example joy or positive . This is a difficult example; without noticing that the event happened in the past, it is easy to assume the poster is presently stressed. We believe examples like this require some grounding for example, an understanding of what passive aggressive means and some representation of the time-line involved, that language models simply cannot express in the traditional classification setup. We also reproduce an anonymized example where our emotion-infused models improve upon BERT: She comes crying to me and formulates a plan to break up. She talks to <name> about their issues and her will to leave him wilts. She stays with him. Rinse and repeat, except it gets worse over time. How can I break the cycle, or help her break the cycle? BERT misclassifies this example, where the author is stressed about a friend's situation, as non-stressful, relying on words like break and help , while our Multi AltGoEmotions models successfully use the word crying to predict stress. We notice that crying or worse is the highest-ranked explanation for most of our emotion-infused models. These results are promising for the development of models that focus on information that humans consider intuitive. In this work, we present a suite of emotion-enhanced models that incorporate emotional information in various ways to enhance the task of binary stress prediction. All three types of our models achieve comparable performance to a state-of-the-art fine-tuning BERT baseline, and, more importantly, we show that they result in more explainable models. We also introduce a new framework for model interpretation using LIME and show that our emotion-enhanced multi-task models offer a new dimension of interpetability by using the predictions of auxiliary tasks to explain the primary task. In our future work, we hope to expand these analyses to tasks in other domains and devise model architectures that can make more direct use of multitask learning to make and explain their predictions. Our intended use of stress detection is to help those in distress. We envision systems such as therapeutic chatbots or assistants that can understand users' emotions and identify those in need so that a person can intervene. We would urge any user of stress detection technology to carefully control who may use the system. Currently, the presented models may fail in two ways: they may either misclassify stress, or they may use the wrong information to make their predictions. Obviously, there is some potential harm to a person who is truly in need if a system based on this work fails to detect them, and it is possible that a person who is not truly in need may be irritated or offended if someone reaches out to them because of a mistake. In terms of explanations, we note that previous work has shown that focusing on incorrect rationales can unfairly target some groups of people (Zhong et al., 2019), although in this work we see that function words truly differ across the stressed and non-stressed populations and we do not observe any language that we know to be representative of minority groups in our explanations. We emphasize our intention that emotional systems such as this be used responsibly, with a human in the loopfor example, a guidance counselor who can look at the predicted labels and offered explanations for their students' stress levels and decide whether or not they seem sensible. We note that because most of our data was collected from Reddit, a website with a known overall demographic skew (towards young, white, American men 8 ), our conclusions about what stress looks like and how to detect it cannot necessarily be applied to broader groups of people. We also note that we have no way to determine the demographic information of the specific posters in any of our datasets and whether they differ from the overall Reddit statistics. We hope that we, and other researchers, can find ways to consider the specific ways in which minority groups express stress as well. We thank our reviewers, as well as the members of our Natural Language Processing research group at Columbia University, for their insightful and constructive comments.
abstain objective objective result result abstain abstain abstain abstain objective abstain abstain abstain abstain objective objective abstain abstain objective abstain abstain result other other other other method objective objective other method abstain other method other other other other abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain method method method method method abstain method method method method method method other abstain abstain method method abstain method method method abstain method method method method method method method abstain method method abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method result objective abstain abstain method method abstain abstain abstain abstain method abstain abstain other