Identifying changes in individuals' behaviour and mood, as observed via content shared on online platforms, is increasingly gaining importance. Most research to-date on this topic focuses on either: (a) identifying individuals at risk or with a certain mental health condition given a batch of posts or (b) providing equivalent labels at the post level. A disadvantage of such work is the lack of a strong temporal component and the inability to make longitudinal assessments following an individual's trajectory and allowing timely interventions. Here we define a new task, that of identifying moments of change in individuals on the basis of their shared content online. The changes we consider are sudden shifts in mood (switches) or gradual mood progression (escalations). We have created detailed guidelines for capturing moments of change and a corpus of 500 manually annotated user timelines (18.7K posts). We have developed a variety of baseline models drawing inspiration from related tasks and show that the best performance is obtained through context aware sequential modelling. We also introduce new metrics for capturing rare events in temporal windows. Linguistic and other content from social media data has been used in a number of different studies to obtain biomarkers for mental health. This is gaining importance given the global increase in mental health disorders, the limited access to support services and the prioritisation of mental health as an area by the World Health Organization (2019). Studies using linguistic data for mental health focus on recognising specific conditions related to mental health (e.g., depression, bipolar disorder) (Hus-seini Orabi et al., 2018), or identifying self-harm ideation in user posts (Yates et al., 2017; Zirikly et al., 2019). However, none of these works, even when incorporating a notion of time (Lynn et al., Figure 1: Example of an Escalation (with a darker peak) and a Switch within a user's timeline. 2018; Losada et al., 2020), identify how an individual's mental health changes over time. Yet being able to make assessments on a longitudinal level from linguistic and other digital content is important for clinical outcomes, and especially in mental health (Velupillai et al., 2018). The ability to detect changes in individual's mental health over time is also important in enabling platform moderators to prioritise interventions for vulnerable individuals (Wadden et al., 2021). Users who currently engage with platforms and apps for mental health support (Neary and Schueller, 2018) would also benefit from being able to monitor their well-being in a longitudinal manner. Motivated by the lack of longitudinal approaches we introduce the task of identifying Moments of Change' (MoC) from individuals' shared online content . We focus in particular on two types of changes: Switches  mood shifts from positive to negative, or vice versa  and Escalations  gradual mood progression (see Fig. 1, detailed in  3). Specifically we make the following contributions:  We present the novel task of identifying moments of change in an individual's mood by analysing linguistic content shared online over time, along with a longitudinal dataset of 500 user timelines (18.7K posts, English language) from 500 users of an online platform. We propose a number of baseline models for 4647 automatically capturing Switches/Escalations, inspired by sentenceand sequence-level state-of-the-art NLP approaches in related tasks. We introduce a range of temporally sensitive evaluation metrics for longitudinal NLP tasks adapted from the fields of change point detection (van den Burg and Williams, 2020) and image segmentation (Arbelaez et al., 2010). We provide a thorough qualitative linguistic analysis of model performance. Social Media and Mental Health Online user-generated content provides a rich resource for computational modelling of wellbeing at both population and individual levels. Research has examined mental health conditions by analysing data from platforms such as Twitter and Reddit (De Choudhury et al., 2013; Coppersmith et al., 2014; Cohan et al., 2018) as well as peer-support networks such as TalkLife (Pruksachatkun et al., 2019). Most such work relies on proxy signals for annotations (e.g., self-disclosure of diagnoses, posts on support networks) and is characterised by a lack of stan-dardisation in terms of annotation and reporting practices (Chancellor and De Choudhury, 2020). We have provided thorough annotation guidelines for Moments of Change that can aid mental health monitoring over time irrespective of the underlying condition. Moments of Change (MoC) Little work has specifically focused on automatically capturing changes in user behaviour based on their social media posts. Within the health domain, Guntuku et al. (2020) showed that a user's language on Facebook becomes more depressed and less informal prior to their visit to an emergency department. With respect to mental health, De Choudhury et al. (2016) proposed to identify shifts to suicide ideation by predicting (or not) a transition from posting on a regular forum to a forum for suicide support. Pruk-sachatkun et al. (2019) examined moments of affective change in TalkLife users by identifying positive changes in sentiment at post-level with respect to a distressing topic earlier in a user's thread. In both cases MoC are overly specific and modelled through binary classification without any notion of temporal modelling. conditions from textual data, including self-harm, suicide ideation, eating disorders, and depression (Benton et al., 2017; Kshirsagar et al., 2017; Yates et al., 2017; Husseini Orabi et al., 2018; Jiang et al., 2020; Shing et al., 2020). Researchers are increasingly adopting sequential modelling to capture temporal dynamics of language use and mental health. For example, Cao et al. (2019) encode microblog posts using suicide-oriented embeddings fed to an LSTM network to assess the suicidality risk at post level. Sawhney et al. (2020b, 2021) improves further on predicting suicidality at post-level by jointly considering an emotion-oriented post representation and the user's emotional state as reflected through their posting history with temporally aware models. The recent shared tasks in eRisk also consider sequences of user posts in order to classify a user as a positive (wrt self-harm or pathological gambling) or control case (Losada et al., 2020; Parapar et al., 2021). While such work still operates at the postor user-level it highlights the importance of temporally aware modelling. Related Temporal NLP Tasks Semantic change detection (SCD) aims to identify words whose meaning has changed over time. Given a set of word representations in two time periods, the dominant approach is to learn the optimal transformation using Orthogonal Procrustes (Schnemann, 1966) and measure the level of semantic change of each word via the cosine distance of the resulting vectors (Hamilton et al., 2016). A drawback of this is the lack of connection between consecutive windows. Tsakalidis and Liakata (2020) addressed this through sequential modeling by encoding word embeddings in consecutive time windows and taking the cosine distance between future predicted and actual word vectors. Both approaches are considered as baselines for our task. First story detection (FSD) aims to detect new events reported in streams of textual data. Having emerged in the Information Retrieval community (Allan et al., 1998), FSD has been applied to streams of social media posts (Petrovic et al., 2010). FSD methods assume that a drastic change in the textual content of a document compared to previous documents signals the appearance of a new story. A baseline from FSD is considered in 4.2. We describe the creation of a dataset of individuals' timelines annotated with Moments of Change. A 4648 user's timeline P ( u ) s : e is a subset of their history, a series of posts [ p 0 , ..., p n ] shared by user u between dates s and e . A Moment of Change (MoC) is a particular point or period (range of time points) within [ s, e ] where the behaviour or mental health status of an individual changes. While MoC can have different definitions in various settings, in this paper we are particularly interested in capturing MoC pertaining to an individual's mood. Other types of MoC can include life events, the onset of symptoms or turning points (e.g., moments of improvement, difficult moments or moments of intervention within therapy sessions). 1 We address two types of Moments of Change: Switches (sud-den mood shifts from positive to negative, or vice versa) and Escalations (gradual mood progression from neutral or positive to more positive or neutral or negative to more negative). Capturing both sudden and gradual changes in individuals' mood over time is recognised as important for monitoring mental health conditions (Lutz et al., 2013; Shalom and Aderka, 2020) and is one of the dimensions to measure in psychotherapy (Barkham et al., 2021). Individual's timelines are extracted from Talklife 2 , a peer-to-peer network for mental health support. Talklife incorporates all the common features of social networks  post sharing, reacting, commenting, etc. Importantly, it provides a rich resource for computational analysis of mental health (Pruk-sachatkun et al., 2019; Sharma et al., 2020; Saha and Sharma, 2020) given that content posted by its users focuses on their daily lives and well-being. A complete collection between Aug'11Aug'20 (12.3M posts, 1.1M users) was anonymised and provided to our research team in a secure environment upon signing a License Agreement. In this environment, 500 user timelines were extracted (3.2) and an additional anonymisation step was performed to ensure that usernames were properly hashed when present in the text. The 500 timelines were subsequently annotated using our bespoke annotation tool (3.3) to derive the resulting longitudinal dataset (3.4). 1 A limitation of our work stems from the fact that MoC are revealed to us by the user's shared content (i.e., we cannot identify changes in a user's well-being unless these are expressed online). We provide details on the limitations of our work in the Ethics Statement (7). 2 https://www.talklife.com/ 3.2 Timeline Extraction Existing work extracts user timelines either based on a pre-determined set of timestamps (e.g., considering the most recent posts by a user) (Sawhney et al., 2020b) or by selecting a window of posts around mentions of specific phrases (e.g., around self-harm) (Mishra et al., 2019). The latter introduces potential bias into subsequent linguistic analysis (Olteanu et al., 2019), while the former could result into selecting timelines from a particular time period  hence potentially introducing temporally-dependent linguistic or topical bias (e.g., a focus on the COVID-19 pandemic). Here we instead extract timelines around points in time where a user's posting behaviour has changed. Our hypothesis is that such changes in a user's posting frequency could be indicative of changes in their lives and/or mental health. Such association between changes in posting behaviour on mental health fora and changes in mental health has been assumed in prior literature (De Choudhury et al., 2016). Identifying changes in posting frequency We create a time series of each user's daily posting frequency based on their entire history. We then employ a change-point detection model to predict the intensity of daily post frequency by the given user. Bayesian Online Change-point Detection (Adams and MacKay, 2007) with a Poisson-Gamma underlying predictive model (Zachos, 2018) was chosen as our model, due to its highly competitive performance (van den Burg and Williams, 2020) and the fact that extracted timelines using this method had the highest density of MoC compared to a number of different timeline extraction (anomaly detection and keyword-based) methods for the same dataset. Extracting timelines around change-points Upon detecting candidate MoC as change-points in posting frequency, we generated candidate timelines for annotation by extracting all of the user's posts within a seven-day window around each change-point. We controlled for timeline length (between 10 and 150 posts, set empirically) so that they were long enough to enable annotators to notice a change but not so long as to hinder effective annotation. This control for timeline length means that our subsequent analysis is performed (and models are trained and evaluated) on time periods during which the users under study are quite active; however, the upper bound of 150 posts in 15 days set for each timeline also ensures that we 4649 do not bias (or limit) our analysis on extremely active users. Finally, to ensure linguistic diversity in our dataset, 500 timelines extracted in this way were chosen for annotation at random, each corresponding to a different individual. The resulting dataset consists of 18,702 posts (  =35, SD =22 per timeline; range of timeline length=[10,124], see Fig. 2(a)). Annotation Interface An annotation interface was developed to allow efficient viewing and annotation of a timeline (see snippet in Fig. 3). Each post in a timeline was accompanied by its timestamp, the user's self-assigned emotion and any associated comments (color-coded, to highlight recurrent users involved within the same timeline). Given the context of the entire timeline, annotations for MoC are performed at post level: if an annotator marks a post as a MoC , then they specify whether it is (a) the beginning of a Switch or (b) the peak of an Escalation (i.e., the most posi-tive/negative post of the Escalation). Finally, the range of posts pertaining to a MoC (i.e., all posts in the Switch/Escalation) need to be specified. Data annotation After a round of annotations for guideline development with PhD students within the research group (co-authors of the paper), we recruited three external annotators to manually label the 500 timelines. They all have University degrees in humanities disciplines and come from three different countries; one of them is an English native speaker. Annotators were provided with a set of annotation guidelines containing specific examples, which were enriched and extended during iterative rounds of annotation. 3 Annotators completed 2 hands-on training sessions with a separate set of 10 timelines, where they were able to ask questions and discuss opinions to address cases of disagreement. Following the initial training phase, we performed spot checks to provide feedback and answer any questions while they labelled the full dataset (n=500 timelines). Annotators were encouraged to take breaks whenever needed, due to the nature of the content. On average, each annotator spent about 5 minutes on annotating a single timeline. The annotation of MoC is akin to assessment of anomaly detection methods since MoC (Switches and Escalations) are rare, with the majority of posts not being annotated (label None'). Measuring the agreement in such settings is therefore complex, as established metrics such as Krippendorff's Alpha and Fleiss' Kappa would generally yield a low score. This is due to the unrealistically high expected chance agreement (Feinstein and Cicchetti, 1990), which cannot be mitigated by the fact that annotators do agree on the majority of the annotations (especially on the None' class). For this reason, we have used as the main indicator the per label positive agreement computed as the ratio of the number of universally agreed-upon instances (the intersection of posts associated with that label) over the total number of instances (the union of posts associated with that label). As highlighted 3 Guidelines are available at https:// github.com/Maria-Liakata-NLP-Group/Annotation-guidelines . in Table 1, while perfect agreement for None' is at 69%, perfect agreement on Escalations and Switches is at 19% and 8%, respectively. However, if instead of perfect agreement we consider majority agreement (where two out of three annotators agree), these numbers drastically increase (30% for Switches and 50% for Escalations). Moreover, by examining the systematic annotation preferences of our annotators we have observed that the native speaker marked almost double the amount of Switches compared to the other two annotators, in particular by spotting very subtle cases of mood change. We have thus decided to generate a gold standard based on majority decisions, comprising only cases where at least two out of three annotators agree with the presence of a MoC. The rare cases of complete disagreement have been labelled as None'. We thus have 2,018 Escalations and 885 Switches from an overall of 18,702 posts (see Fig. 2(b) for the associated lengths in #posts). In future work we plan to consider aggregation methods based on all annotations or approaches for learning from multiple noisy annotations (Paun and Simpson, 2021). Our aim is to detect and characterise the types of MoC based on a user's posting activity. We therefore treat this problem as a supervised classification task (both at post level and in a sequential/timeline-sensitive manner, as presented in 4.2) rather than an unsupervised task, even though we also consider effectively baselines with unsupervised components (FSD, SCD in 4.2). Contrary to traditional sentence or document-level NLP tasks, we incorporate timeline-sensitive evaluation metrics that account for the sequential nature of our model predictions (4.1). Given a user's timeline, the aim is to classify each post within it as belonging to a Switch (IS), an Escalation (IE), or None (O). At this point we don't distinguish between beginnings of switches/peaks of escalations and other posts in the respective ranges. While the task is sequential by definition, we train models operating both at the post level in isolation and sequential models at the timeline-level (i.e., accounting for user's posts over time), as detailed in 4.2. We contrast model performance using common post-level classification metrics as well as novel timeline-level evaluation approaches (4.1). This allows us to investigate the impact of (a) accounting for severe class imbalance and (b) longitudinal modelling. We have randomly divided the annotated dataset into 5 folds (each containing posts from 100 timelines) to allow reporting results on all of the data through cross-validation. Post-level We first assess model performance on the basis of standard evaluation metrics at the post level (Precision, Recall, F1 score). These are obtained per class and macro-averaged, to better emphasize performance in the two minority class labels (IS & IE). However, post-level metrics are unable to show: (a) the expected accuracy at the timeline level (see example in Fig. 4) and (b) model suitability in predicting regions of change. These aspects are particularly important since we aim to build models capturing MoC over time. Timeline-level Our first set of timeline-level evaluation metrics are inspired from work in changepoint detection (van den Burg and Williams, 2020) and mirror the post-level ones, albeit operating on a window and timeline basis. Specifically, working on each timeline and label type independently, we calculate Recall R ( l ) w (Precision P ( l ) w ) by counting as correct a model prediction for label l if the prediction falls within a window of w posts around post labelled l in the gold standard. Formally: R ( l ) w = | TP w ( M ( l ) ,GS ( l ) ) | | GS ( l ) | , P ( l ) w = | TP w ( M ( l ) ,GS ( l ) ) | | M ( l ) | , where T P w denotes the true positives that fall within a range of w posts and M ( l ) / GS ( l ) are the predicted/actual labels for l , respectively. Note that each prediction can only be counted once as correct. R ( l ) w and P ( l ) w are calculated on every timeline and are then macro-averaged. The second set of our timeline-level evaluation metrics is adapted from the field of image segmentation (Arbelaez et al., 2010). Here we aim at evaluating model performance based on its ability to capture regions of change (e.g., in Fig 4, GS' shows a timeline with three (two) such regions of Escalations (Switches)). For each such true region R ( l ) GS , we define its overlap O ( R ( l ) GS , R ( l ) M ) with each predicted region R ( l ) M as the intersection over union between the two sets. This way, we can get recall and precision oriented coverage metrics as follows: C ( l ) r ( M  GS ) = 1 (cid:80) R ( l ) GS | R ( l ) GS | (cid:80) R ( l ) GS | R ( l ) GS |  max R ( l ) M { O ( R ( l ) GS ,R ( l ) M ) } , 4651 Figure 4: Actual ( GS , shown twice) vs Predicted labels for each post (square) of a single timeline, by two models ( M1 , M2 ). C ( l ) p ( M  GS ) = 1 (cid:80) R ( l ) M | R ( l ) M | (cid:80) R ( l ) M | R ( l ) M |  max R ( l ) GS { O ( R ( l ) GS ,R ( l ) M ) } . The coverage metrics are calculated on the timeline basis and macro-averaged similarly to R ( l ) w and P ( l ) w . Using a set of evaluation metrics, each capturing a different aspect of the task, ensures assess to model performance from many different angles. We have considered different approaches to addressing our task: (i) Nave methods, specifically a Majority classifier (predicting always None) and a Random predictor, picking a label based on the overall label distribution in the dataset. It has been shown that comparisons against such simple baselines is essential to assess performance in computational approaches to mental health (Tsakalidis et al., 2018). (ii) Post-level supervised models operating on posts in isolation (i.e., ignoring post sequence in a user's timeline): (a) Random Forest (Breiman, 2001) on tfidf post representations ( RF-tfidf ); (b) BiLSTM (Huang et al., 2015) operating on sequences of word embeddings ( BiLSTM-we );(c) BERT(ce) (Devlin et al., 2019) using the cross-entropy loss; and (d) BERT(f) trained using the alpha-weighted focal loss (Lin et al., 2017), which is more appropriate for imbalanced datasets. (iii) Emotion Classification We used DeepMoji ( EM-DM ) (Felbo et al., 2017) and Twitter-roBERTa-base ( EM-TR ) from TweetEval '20 (Barbieri et al., 2020) operating on the post-level, to generate soft-max probabilities for each emotion (64 for EM-DM , 4 for EM-TR ). These provide meta-features to a BiLSTM to obtain timeline-sensitive models for identifying MoC. (iv) First Story Detection (FSD) . We have used two common approaches for comparing a post to the n previous ones: representing the previous posts as (i) a single centroid or (ii) the nearest neighbour to the current post among them (Allan et al., 1998; Petrovic et al., 2010). In both cases, we calculate the cosine similarity of the current and previous posts. The scores are then fed into a BiLSTM as meta-features for a sequential model. Results are reported for the best method only. (v) Semantic Change Detection (SCD) . Instead of the standard task of comparing word representations in consecutive time windows, we consider a user being represented via their posts at particular points in time. We follow two approaches. The first is an Orthogonal Procrustes approach (Schne-mann, 1966) operating on post vectors ( SCD-OP ). Our aim here is to find the optimal transformation across consecutive representations, with higher errors being indicative of a change in the user's behaviour. In the second approach ( SCD-FP ) a BiLSTM is trained on the user's k previous posts in order to predict the next one (Tsakalidis and Liakata, 2020). Errors in prediction are taken to signal changes in the user. In both cases, we calculate the dimension-wise difference between the actual and the transformed/predicted representations (post vectors) and use this as a meta-feature to a BiLSTM to obtain a time-sensitive model. (vi) Timeline-sensitive . From our (ii) post-level classifiers, BERT(f) tackles the problem of imbalanced data but fails to model the task in a longitudinal manner. To remedy this, we employ BiLSTM-bert , which treats a timeline as a sequence of posts to be modelled, each being represented via the [CLS] representation of BERT(f) . To convert the post-level scores/representations from (iii)-(v) above into time-sensitive models we used the same BiLSTM from (vi), operating at the timeline-level. Details for each model and associated hyperparameters are in the Appendix. Model Comparison Table 2 summarises the results of all models; Fig. 5 further shows the P w / R w metrics for IE/IS for the best-performing models. BiLSTM-bert confidently outperforms all com-4652 Post-level Evaluation Coverage-based Metrics IS IE O macro-avg IS IE O macro-avg P R F1 P R F1 P R F1 P R F1 C p C r C p C r C p C r C p C r N a  v e Majority  .000 .000  .000 .000 .845 1.000 .916 .282 .333 .305  .000  .000 .619 .559 .206 .186 Random .047 .047 .047 .108 .108 .108 .845 .845 .845 .333 .333 .333 .031 .045 .033 .096 .386 .452 .150 .198 P o s t l e v e l RF-tfidf .294 .006 .011 .568 .087 .151 .852 .991 .917 .571 .361 .360 .250 .005 .152 .087 .632 .602 .345 .231 BiLSTM-we .245 .119 .160 .416 .347 .378 .878 .923 .900 .513 .463 .479 .173 .091 .138 .330 .557 .606 .289 .342 BERT(ce) .285 .186 .222 .454 .368 .406 .883 .921 .901 .540 .492 .510 .247 .163 .172 .344 .578 .621 .332 .376 BERT(f) .260 .321 .287 .401 .478 .436 .898 .864 .881 .520 .554 .534 .227 .269 .160 .423 .503 .567 .297 .420 T i m e li n e l e v e l FSD  .000 .000  .000 .000 .845 1.000 .916 .282 .333 .305  .000  .000 .619 .559 .206 .186 EM-TR .344 .036 .065 .444 .248 .318 .865 .957 .909 .551 .414 .431 .297 .024 .273 .104 .639 .589 .403 .239 EM-DM .533 .118 .193 .479 .351 .405 .880 .948 .913 .631 .472 .504 .347 .023 .363 .177 .646 .592 .452 .264 SCD-OP .200 .005 .009 .478 .408 .440 .882 .947 .913 .520 .453 .454 .167 .001 .344 .180 .663 .609 .391 .263 SCD-FP .270 .082 .126 .503 .370 .426 .880 .944 .911 .551 .465 .488 .227 .039 .317 .254 .649 .611 .398 .301 BiLSTM-bert .397 .264 .316 .568 .461 .508 .898 .936 .917 .621 .553 .580 .331 .197 .345 .340 .664 .656 .447 .398 Table 2: Post-level and Coverage-based evaluation for each model ( first and second highest scores are highlighted). peting models in terms of post-level macro-F1. It provides a 8.6% relative improvement (14% for the IS/IE labels) against the second best performing model ( BERT(f) ). Furthermore, it achieves a great balance between precisionand recall-oriented timeline-level metrics, being consistently the second-best performing model. This performance is largely attributed to two factors, which are studied further below: (a) the use of the Focal loss on BERT, generating [CLS] representations that are much more focused on the minority classes (IE/IS), and (b) its longitudinal aspect. Post-level The BERT variants perform better than the rest in all metrics. Their coverage metrics though suggest that while they manage to predict better the regions compared to most timeline-level methods (i.e., high C r ), they tend to predict more regions than needed (i.e., low C p )  partially due to their lack of contextual (temporal-wise) information. Finally, as expected, BERT(f) achieves much higher recall for the minority classes (IE/IS), in exchange for a drop in precision compared to BERT(ce) and in recall for the majority class (O). Models from Related Tasks EM-DM achieves very high precision ( P , P w ) for the minority classes, showing a clear link between the tasks of emotion recognition and detecting changes in a user's mood  indeed, emotionally informed models have been successfully applied to post-level classification tasks in mental health (Sawhney et al., 2020a); however, both EM models achieve low recall ( R , R w ) for IE/IS compared to the rest. For the SCD inspired models, SCD-FP outperforms SCD-OP on most metrics. This is largely due to the fact that the former uses the previous k =3 posts to predict the next post in a user's timeline (instead of aligning it based on the previous post only. Thus SCD-FP benefits from its longitudinal component  a finding consistent with work in semantic change detection (Tsakalidis and Liakata, 2020). Representation vs Fine-tuning vs Focal Loss While BiLSTM-bert yields the highest macro-F1 and the most robust performance across all metrics, it is not clear which of its components contributes the most to our task. To answer this, we perform a comparison against the exact same BiLSTM, albeit fed with different input types: (a) average word embeddings as in BiLSTM-we , (b) Sentence-BERT representations (Reimers and Gurevych, 2019) and (c) fine-tuned representations from BERT(ce) . As shown in Table 3, fine-tuning with BERT(ce) outperforms Sentence-BERT representations. While the contextual nature of all of the BERT-based models offers a clear improvement over the static word embeddings, it becomes evident that the use of the focal loss during training 4653 Post Timeline Coverage P R F1 P 1 R 1 C p C r Word emb. the initial BERT(f) is vital, offering a relative improvement of 6% in post-level macro-F1 (13.7% for IS/IE). Calibrating the parameters in the focal loss could provide further improvements for our task in the future (Mukhoti et al., 2020). Timelinevs Post-level Modelling The importance of longitudinal modelling is shown via the difference between the BERT and BiLSTM variants when operating on single posts vs on the timeline-level (e.g., see the post-level results of BERT(ce) / Word emb. in Table 3 vs BERT(ce) / BiLSTM-we in Table 2, respectively). We further examine the role of longitudinal modelling in the rest of our best-performing models from Table 2. In particular, we replace the timeline-level BiLSTM in EM-DM and SCD-FP with a two-layer feed-forward network, operating on post-level input representations  treating each post in isolation. The differences across all pairwise combinations with and without the longitudinal component are shown in Fig. 6. Timeline-level models achieve much higher precision (6.1%/6.9%/11.1% for P / P 1 / C p , respectively) in return for a small sacrifice in the timeline-level recall-oriented metrics (-2.8%/1.9%/2.3% for R / R 1 / C r ), further highlighting the longitudinal nature of the task. Here we analyse the cases of Switches/Escalations identified or missed by our best performing model ( BiLSTM-bert ). Switches (IS) are the most challenging to identify, largely due to being the smallest class with the lowest inter-annotator agreement. However, the EM -based models achieve high levels of precision on Switches, even during post-level evaluation (see Table 2). We therefore employ EM-TR (Barbieri et al., 2020), assigning probability scores for anger/joy/optimism/sadness to each post, and use them to characterise the predictions made by BiLSTM-bert . Fig. 7 and Table 4 show that our model predicts more often (in most cases, correctly) a Switch' when the associated posts express positive emotions (joy/optimism), but misses the vast majority of cases when these emotions are absent. The reason for this is that TalkLife users discuss issues around their well-being, with a negative mood prevailing. Therefore, BiLSTM-bert learns that the negative tone forms the users' baseline and thus deviations from this constitute cases of Switches' (see example in Table 5). We plan to address this in the future by incorporating transfer learning approaches to our model (Ruder et al., 2019). Escalations (IE) are better captured by our models. Here we examine more closely the cases of Peaks' in the escalations (i.e., the posts indicating the most 4654 Text True Pred. Oh, forgot :) Stay safe you lovely people all around the world! negative/positive state of the user within an escalation  see 3.3). As expected, the post-level recall of BiLSTM-bert in these cases is much higher than its recall for the rest of IE cases (.557 vs .408). In Fig. 8 we analyse the recall of our model in capturing posts denoting escalations, in relation to the length of escalations. We can see that our model is more effective in capturing longer escalations. As opposed to the Switch class, we found no important differences in the expressed emotion between TP and FN cases. By carefully examining the cases of Peaks in isolation, we found that the majority of them express very negative emotions, very often including indication of self-harm. A Logistic Regression trained on bigrams at the post-level to distinguish between identified vs missed cases of Peaks showed that the most positively correlated features for the identified cases were directly linked to self-harm (e.g., kill myself , to die , kill me ). However, this was not necessarily the case with missed cases. Nevertheless, there were several cases of self-harm ideation that were missed by BiLSTM-bert , as well as misses due to the model ignoring the user's baseline, as is the case with Switches (see Table 6). Transfer learning and domain adaptation strategies as well as self-harm detection models operating at the post level could help in mitigating this problem. When my parents go out, I am gonna cut. I feel so horrible. I really don't want to be here anymore. Someone please text me... I swear I am about to harm myself... Please, anyone!' Had an awesome day with my gf and she tagged me! I am not alone! :) Have not cut for the past year!! Yay!! We present a novel longitudinal dataset and associated models for personalised monitoring of a user's well-being over time based on linguistic online content. Our dataset contains annotations for: (a) sudden shifts in a user's mood (switches) and (b) gradual mood progression (escalations). Proposed methods are inspired by state-of-the-art contextual models and longitudinal NLP tasks. Importantly, we have introduced temporally sensitive evaluation metrics, adapted from the fields of change-point detection and image segmentation. Our results highlight the importance of considering the temporal aspect of the task and the rarity of mood changes. Future work could follow four main directions: (a) integrating longitudinal models of detecting changes, with post-level models for emotion and self-harm detection (see 5.2); (b) incorporating transfer learning methods (Ruder et al., 2019) to adapt more effectively to unseen users' timelines; (c) adjusting our models to learn from multiple (noisy) annotators (Paun and Simpson, 2021) and (d) calibrating the parameters of focal loss and testing other loss functions suited to heavily imbalanced classification tasks (Jadon, 2020). Ethics institutional review board (IRB) approval was obtained from the corresponding ethics board of the University of Warwick prior to engaging in this research study. Our work involves ethical considerations around the analysis of user generated content shared on a peer support network (Talk-Life). A license was obtained to work with the user data from TalkLife and a project proposal was submitted to them in order to embark on the project. The current paper focuses on the identification of moments of change (MoC) on the basis of content shared by individuals. These changes involve recognising sudden shifts in mood (switches or es-4655 calations). Annotators were given contracts and paid fairly in line with University payscales. They were alerted about potentially encountering disturbing content and were advised to take breaks. The annotations are used to train and evaluate natural language processing models for recognising moments of change as described in our detailed guidelines. Working with datasets such as TalkLife and data on online platforms where individuals disclose personal information involves ethical considerations (Mao et al., 2011; Keklloglu et al., 2020). Such considerations include careful analysis and data sharing policies to protect sensitive personal information. The data has been de-identified both at the time of sharing by TalkLife but also by the research team to make sure that no user handles and names are visible. Any examples used in the paper are either paraphrased or artificial. Potential risks from the application of our work in being able to identify moments of change in individuals' timelines are akin to those in earlier work on personal event identification from social media and the detection of suicidal ideation. Potential mitigation strategies include restricting access to the code base and annotation labels used for evaluation. Limitations Our work in this paper considers moments of change as changes in an individual's mood judged on the basis of their self-disclosure of their well-being. This is faced by two limiting factors: (a) users may not be self-disclosing important aspects of their daily lives and (b) other types of changes related to their mental health (other than their mood/emotions, such as important life events, symptoms etc.) may be taking place. Though our models could be tested in cases of non-self-disclosure (given the appropriate ground truth labels), the analysis and results presented in this work should not be used to infer any conclusion on such cases. The same also holds for other types of moments of change' mentioned in 2 (e.g., transition to suicidal thoughts), as well as other types of changes, such as changes in an individual in terms of discussing more about the future, studied in Althoff et al. (2016), or changes in their self-focus (Pyszczynski and Greenberg, 1987) over time, which we do not examine in this current work. This work was supported by a UKRI/EPSRC Turing AI Fellowship to Maria Liakata (grant EP/V030302/1) and the Alan Turing Institute (grant EP/N510129/1). The authors would like to thank Dana Atzil-Slonim, Elena Kochkina, the anonymous reviewers and the meta-reviewer for their valuable feedback on our work, as well as the three annotators for their invaluable efforts in generating the longitudinal dataset.
abstain abstain abstain abstain abstain objective method method objective objective abstain abstain abstain abstain abstain abstain abstain abstain objective result objective objective abstain method other other other method other other other other other other other other other other other other other other other method other other other other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method other abstain method abstain abstain method method method method method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method other abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain abstain abstain method other other other