Throughout a conversation, participants make choices that can orient the flow of the interaction. Such choices are particularly salient in the consequential domain of crisis counseling, where a difficulty for counselors is balancing between two key objectives: advancing the conversation towards a resolution, and empathetically addressing the crisis situation. In this work, we develop an unsupervised methodology to quantify how counselors manage this balance. Our main intuition is that if an utterance can only receive a narrow range of appropriate replies, then its likely aim is to advance the conversation forwards, towards a target within that range. Likewise, an utterance that can only appropriately follow a narrow range of possible utterances is likely aimed backwards at addressing a specific situation within that range. By applying this intuition, we can map each utterance to a continuous orientation axis that captures the degree to which it is intended to direct the flow of the conversation forwards or backwards. This unsupervised method allows us to characterize counselor behaviors in a large dataset of crisis counseling conversations, where we show that known counseling strategies intuitively align with this axis. We also illustrate how our measure can be indicative of a conversation's progress, as well as its effectiveness. Participants in a conversation constantly shape the flow of the interaction through their choices. In psychological crisis counseling conversations, where counselors support individuals in mental distress, these choices arise in uniquely complex and high-stakes circumstances, and are reflected in rich conversational dynamics (Sacks, 1992). As such, counseling is a valuable context for computationally modeling conversational behavior (Atkins when i tell my mom about the bullies she just ignores me Have you confided to anyone else about this? yeah there's my sister she just tells me to get over it That sounds so frustrating, you deserve to be listened to. t 0 t 1 t 2 c 1 c 2 Figure 1: Two possible exchanges in a counseling conversation, illustrating key objectives that a counselor must balance: c 1 aims to advance the conversation towards a discussion of possible confidants; c 2 aims to address the emotion underlying the preceding utterance. et al., 2014; Althoff et al., 2016; Perez-Rosas et al., 2018; Zhang et al., 2019). Modeling the conversational choices of counselors in this endeavor is an important step towards better supporting them. Counselors are driven by several objectives that serve the broader goal of helping the individual in distress; two key objectives are exemplified in Figure 1. 1 The counselor must advance a conversation towards a calmer state where the individual is better equipped to cope with their situation (Mishara et al., 2007; Sandoval et al., 2009): in c 1 , the counselor prompts the individual to brainstorm options for social support. The counselor must also empathetically address what was already said, coming to an empathic understanding of the individual (Rogers, 1957; Hill and Nakayama, 2000): in c 2 , the counselor validates feelings that the individual has just shared. Balancing both objectives is often challenging, and overshooting in one direction can be detrimental to the conversation. A counselor who leans too much on advancing forwards could rush the conversation at the expense of establishing an empathetic connection; a counselor who leans too much backwards , on addressing what was already said, may fail to make any progress. 1 These examples are derived from material used to train counselors in our particular setting, detailed in Section 2. 5277 In this work, we develop a method to examine counselor behaviors as they relate to this balancing challenge. We quantify the relative extent to which an utterance is aimed at advancing the conversation, versus addressing existing content. We thus map each utterance onto a continuous backwards-forwards axis which models the balance of these objectives, and refer to an utterance's position on this axis as its orientation . At an intuitive level, our approach considers the range of content that is expected to follow or precede a particular utterance. For an utterance like c 1 that aims to advance the conversation towards an intended target, we would expect a narrow range of appropriate replies, concentrated around that target (e.g., suggestions of possible confidants). We would likewise expect an utterance like c 2 that aims to address a previously-discussed situation to only be an appropriate reply for a narrow range of possible utterances, concentrated around that specific type of situation (e.g., disclosures of negative feelings). Starting from this intuition, we develop an unsupervised method to quantify and compare these expected forwards and backwards ranges for any utterance, yielding our orientation measure. Using this measure, we characterize counselor behaviors in a large collection of text-message conversations from a crisis counseling service, which we accessed in collaboration with the service and with the participants' consent. We show how orientation meaningfully distinguishes between key conversational strategies that counselors are taught during their training. We also show that our measure tracks a conversation's progress and can signal its effectiveness, highlighting the importance of balancing the advancing and addressing objectives, and laying the basis for future inquiries in establishing potential causal effects. In summary, we develop an unsupervised methodology that captures how counselors balance the conversational objectives of advancing and addressing (Section 4), apply and validate it in a large dataset of counseling conversations (Section 5), and use it to investigate the relation between a counselor's conversational behavior and their effectiveness (Section 5.4). While our method is motivated by a salient challenge in counseling, we expect similar balancing problems to recur in other conversational settings where participants must carefully direct the flow of the interaction, such as court trials and debates (Section 6). We develop our method in the context of Crisis Text Line, a crisis counseling platform which provides a free 24/7 service for anyone in mental crisishenceforth texters to have one-on-one conversations via text message with affiliated counselors. We accessed a version of this collection, with over 1.5 million conversations, in collaboration with the platform and with the consent of the participants. The data was scrubbed of personally identifiable information by the platform. 2 These conversations are quite substantive, averaging 25 messages with 29 and 24 words per counselor and texter message, respectively. In each conversation, a crisis counselor's high-level goal is to guide the texter towards a calmer mental state. In service of this goal, all counselors first complete 30 hours of training provided by the platform, which draws on past literature in counseling to recommend best practices and conversational strategies. The first author also completed the training to gain familiarity with the domain. While the platform offers guidance to counselors, their task is inevitably open-ended, given the emotional complexity of crisis situations. As such, the counselors are motivated by an explicit goal that structures the interaction, but they face a challenging flexibility in choosing how to act. We now describe the conversational challenge of balancing between advancing the conversation forwards or addressing what was previously said. Our description of the challenge and our computational approach to studying it are informed by literature in counseling, on the platform's training material and on informal interviews with its staff. A conversational balance. A crisis counselor must fulfill multiple objectives in their broader goal of helping a texter. One objective is guiding the texter through their initial distress to a calmer mental state (Mishara et al., 2007; Sandoval et al., 2009), as in Figure 1, c 1 . Various strategies that aim to facilitate this advancing process are taught to counselors during training: for instance, a counselor may prompt a texter to identify a goal or cop-2 The data can be accessed by applying at https:// www.crisistextline.org/data-philosophy/data-fellows/ . The extensive ethical and privacy considerations, and policies accordingly implemented by the platform, are detailed in Pisani et al. (2019). ing mechanism (Rollnick and Miller, 1995). As such, they attempt to move the conversation forwards , towards its eventual resolution. The counselor must also engage with the texter's concerns (Rogers, 1957; Hill and Nakayama, 2000), as in c 2 , via strategies that empathetically address what the texter has already shared (Roll-nick and Miller, 1995; Weger et al., 2010; Bodie et al., 2015). For instance, counselors are taught to reflect , i.e., reframe a texter's previous message to convey understanding, or draw on what was said to affirm the texter's positive qualities. In doing so, the counselor looks backwards in the conversation. Past work has posited the benefits of mixing between strategies that aim at either objective (Mishara et al., 2007). However, as the training acknowledges, striking this balance is challenging. Overzealously seeking to advance could cut short the process of establishing an empathetic connection. Conversely, focusing on the conversation's past may not help with eventual problem solving (Bodie et al., 2015), and risks stalling it. A texter may start to counterproductively rehash or ruminate on their concerns (Nolen-Hoeksema et al., 2008; Jones et al., 2009); indeed, prior psychological work has highlighted the thin line between productive reflection and rumination (Rose et al., 2007; Landphair and Preddy, 2012). Orientation. To examine this balancing dynamic, we model the choices that counselors make at each turn in a conversation. Our approach is to derive a continuous axis spanned by advancing and addressing. We refer to an utterance's position on this axis, representing the relative extent to which it aims at either objective, as its orientation  . We interpret a forwards-oriented utterance with positive  as aiming to advance the conversation, and a backwards-oriented utterance with negative  as aiming to address what was previously brought up. In the middle, the axis reflects the graded way in which a counselor can balance between aimsfor instance, using something the texter has previously said to help motivate a problem-solving strategy. Related characterizations. While we develop orientation to model a dynamic in counseling, we view it as a complement to other characterizations of conversational behaviors in varied settings. Prior work has similarly considered how utterances relate to the preceding and subsequent discourse (Webber, 2001). Frameworks like centering theory (Grosz et al., 1995) aim at identifying referenced entities, while we aim to more abstractly model interlocutor choices. Past work has also examined how interlocutors mediate a conversation's trajectory through taking or ceding control (Walker and Whittaker, 1990) or shifting topic (Nguyen et al., 2014); Althoff et al. (2016) considers the rate at which counselors in our setting advance across stages of a conversation. While these actions can be construed as forwards-oriented, we focus more on the interplay between forwardsand backwards-oriented actions. A counselor's objectives may also cut across these concepts: for instance, the training stresses the need for empathetic reflecting across all stages and topics. Orientation also complements prior work on dialogue acts, which consider various roles that utterances play in discourse (Mann and Thompson, 1988; Core and Allen, 1997; Ritter et al., 2010; Bracewell et al., 2012; Rosenthal and McKeown, 2015; Prabhakaran et al., 2018; Wang et al., 2019). In counseling settings, such approaches have highlighted strategies like reflection and question-asking (Houck, 2008; Gaume et al., 2010; Atkins et al., 2014; Can et al., 2015; Tanana et al., 2016; Perez-Rosas et al., 2017, 2018; Park et al., 2019; Lee et al., 2019; Cao et al., 2019). Instead of modeling a particular taxonomy of actions, we model how counselors balance among the underlying objectives; we later relate orientation to these strategies (Section 5). Most of these approaches use annotations or predefined labeling schemes, while our method is unsupervised. We now describe our method to measure orientation, discussing our approach at a high level before elaborating on the particular operationalization. The code implementing our approach is distributed as part of the ConvoKit library (Chang et al., 2020), at http://convokit.cornell.edu . Orientation compares the extent to which an utterance aims to advance the conversation forwards with the extent to which it looks backwards. Thus, we must somehow quantify how the utterance relates to the subsequent and preceding interaction. Naive attempt: direct comparison. As a natural starting point, we may opt for a similarity-based approach: an utterance that aims to address its preceding utterance, or predecessor , should be similar 5279 sounds frustrating confided to anyone ignoresjudges laughs doesn't because just problem ignore nothing sister friend counselor expected predecessors: expected replies: Figure 2: Words representative of replies and predecessors for utterances with two example phrasings, as observed in training data. to it; an utterance that aims to advance the conversation should be similar to the reply that it prompts. In practice, having to make these direct comparisons is limiting: an automated system could not characterize an utterance in an ongoing conversation by comparing it to a reply it has yet to receive. This approach also has important conceptual faults. First, addressing preceding content in a conversation is different from recapitulating it. For instance, counselors are instructed to reframe rather than outright restate a texter's message, as in Figure 1, c 2 . Likewise, counselors need not advance the conversation by declaring something for the texter to simply repeat; rather than giving specific recommendations, counselors are instructed to prompt the texters to come up with coping strategies on their own, as in c 1 . Further, texters are not bound to the relatively formal linguistic style counselors must maintain, resulting in clear lexical differences. Measuring orientation is hence a distinct task from measuring similarity. Second, an utterance's intent to advance need not actually be realized. A counselor's cues may be rebuffed or misunderstood (Schegloff, 1987; Thomas, 1983): a texter could respond to c 1 by continuing to articulate their problem with t 2 . Likewise, a counselor may intend to address a texter's concerns but misinterpret them. To model the balance in objectives that a counselor is aiming for, our characterization of an utterance cannot be contingent on its actual reply and predecessor. Our approach: characterizing expectations. We instead consider the range of replies we might expect an utterance to receive, or the range of predecessors that it might follow. Intuitively, an utterance with a narrow range of appropriate replies aims to direct the conversation towards a particular target, moreso than an utterance whose appropriate replies span a broader range. 3 Likewise, an utterance that is an appropriate reply to only a narrow range of possible predecessors aims to address a particular situation. We draw on unlabeled data of past conversations to form our expectations of these ranges, and build up our characterizations of utterances from their constituent phrasings , e.g., words or dependency-parse arcs. The intuition for our approach is sketched in Figure 2. From our data, we observe that utterances containing confided to anyone generally elicited replies about potential confidants (e.g., sister , friend ), while the replies that followed utterances with sounds frustrating span a broader, less well-defined range. As such, we have a stronger expectation of what a reply prompted by a new utterance with confided to anyone might contain than a reply to a new utterance with sounds frustrating . More generally, for each phrasing w , we quantify the strength of our expectations of its potential replies by measuring the range spanned by the replies it has already received in the data, which we refer to as its forwards-range (cid:0)! (cid:27) w . We would say that confided to anyone has a smaller (cid:0)! (cid:27) w than sounds frustrating , meaning that its observed replies were more narrowly concentrated; this is represented as the relative size of the red regions on the right side of Figure 2. In the other direction, we observe in our data that sounds frustrating generally followed descriptions of frustrating situations (e.g., ignores , judges ), while the range of predecessors to confided to anyone is broader. We thus have a stronger expectation of the types of situations that new utterances with sounds frustrating would respond to, compared to new utterances with confided to anyone . For a phrasing w , we quantify the strength of our expectations of its potential predecessors by measuring its backwards-range (cid:0) (cid:27) w , spanned by the predecessors we've observed. As such, sounds frustrating has a smaller (cid:0) (cid:27) w than confided to anyone , corresponding to the relative size of the blue regions on the left side of Figure 2. 3 Consider leading versus open-ended questions. When people ask leading questions, they intend to direct the interaction towards specific answers they have in mind; when people ask open-ended questions, they are more open to what answers they receive and where the interaction is headed. The relative strengths of our expectations in either direction then indicate the balance of objectives. If we have a stronger expectation of w 's replies than of its predecessorsi.e., smaller (cid:0)! (cid:27) w than (cid:0) (cid:27) w we would infer that utterances with w aim to advance the conversation towards a targeted reply more than they aim to address a particular situation. Conversely, if we have stronger expectations of w 's predecessorsi.e., smaller (cid:0) (cid:27) w  we would infer that utterances with w aim to address the preceding interaction, rather than trying to drive the conversation towards some target. We thus measure orientation by comparing a phrasing's forwardsand backwards-range. The expectation-based approach allows us to circumvent the shortcomings of a direct comparison; we may interpret it as modeling a counselor's intent in advancing and addressing at each utterance (Moore and Paris, 1993; Zhang et al., 2017). We now detail the steps of our method, which are outlined in Figure 3. Formally, our input consists of a set of utterances from counselors f c i g , and a set of utterances from texters f t i g , which we've observed in a dataset of conversations (Figure 3A). We note that each texter utterance can be a reply to, or a predecessor of, a counselor utterance (or both). We use this unlabeled training data to measure the forwards-range (cid:0)! (cid:27) w , the backwards-range (cid:0) (cid:27) w (Figures 3B-D), and hence the orientation  w of each phrasing w used by counselors (Figure 3E). We then aggregate to an utterance-level measure. For each counselor phrasing w , let (cid:0)! T w denote the subset of texter utterances which are replies to counselor utterances containing w (Figure 3A). As described above, the forwards-range (cid:0)! (cid:27) w quantifies the spread among elements of (cid:0)! T w ; we measure this by deriving vector representations of these utterances (cid:0)! U w (Figure 3B, detailed below), and then comparing each vector in (cid:0)! U w to a central reference point (cid:0)! u w (Figures 3C and 3D). 4 Likewise, (cid:0) (cid:27) w quantifies the similarity among elements of (cid:0) T w , the set of predecessors to counselor utterances with w ; we compute (cid:0) (cid:27) w by comparing each corresponding vector in (cid:0) U w to a central point (cid:0) u w . 4 Using a central reference point to calculate the forwards-range, as opposed to directly computing pairwise similarities among replies in (cid:0)! U w , allows us to account for the context of w in the utterances that prompted these replies (via tf-idf weighting, as subsequently discussed). To obtain vectors for each texter utterance, we construct X , a tf-idf reweighted term-document matrix where rows represent texter utterances and columns represent phrasings used by texters. To ensure that we go beyond lexical matches and capture conceptual classes (e.g., possible confidants, frustrating situations), we use singular value decomposition to get X (cid:25) USVT . Each row of U is a vector representation u i of utterance t i in the induced low-dimensional space T . (cid:0)! U w then consists of the corresponding subset of rows of U (high-lighted in Figure 3B). Deriving central points (Figure 3C). For each w , we take the central point (cid:0)! u w to be a weighted average of vectors in (cid:0)! U w . Intuitively, a texter utterance t i with vector u i should have a larger contribution to (cid:0)! u w if w is more prominent in the counselor utterance c i that preceded it. We let w iw denote the normalized tf-idf weight of w in c i , and use w iw as the weight of the corresponding vector u i . To properly map the resultant weighted sum  w iw u i into T , we divide each dimension by the corresponding singular value in S . As such, if w w is a vector of weights w iw , we can calculate the central point (cid:0)! u w 5281 of (cid:0)! we likewise compute u w = w Tw U w S (cid:0) 1 . Forwardsand backwards-ranges (Figure 3D). We take the forwards-range (cid:0)! (cid:27) w of w to be the average cosine distance from each vector in (cid:0)! U w to the center point (cid:0)! u w . Likewise, we take (cid:0) (cid:27) w as the average distance from each vector in (cid:0) U w to (cid:0) u w . Phrasing-level orientation (Figure 3E). Importantly, since we've computed the forwardsand backwards-ranges (cid:0)! (cid:27) w and (cid:0) (cid:27) w using distances in the same space T , their values are comparable. We then compute the orientation of w as their difference:  w = (cid:0) (cid:27) w (cid:0) (cid:0)! (cid:27) w . Utterance-level orientation. To compute the orientation of an utterance c i , we first compute the orientation of each sentence in c i as the tf-idf weighted average  w of its constitutent phrasings. Note that a multi-sentence utterance can orient in both directionse.g., a counselor could concatenate c 2 and c 1 from Figure 1 in a single utterance, addressing the texter's previous utterance before moving ahead. To model this heterogeneity, we consider both the minimum and maximum sentence-orientations in an utterance:  min captures the extent to which the utterance looks backwards, while  max captures the extent to which it aims to advance forwards. We apply our method to characterize messages from crisis counselors on the platform. We compute the orientations of the phrasings they use, represented as dependency-parse arcs. We use a training set of 351,935 texter and counselor messages each, from a random sample of conversations omitted in subsequent analyses. 5 Table 1 shows representative phrasings and sentences of different orientations. 6 Around two-thirds of phrasings and sentences have  < 0 , echoing the importance of addressing the texter's previous remarks. In what follows, we analyze counselor behaviors in terms of orientation, and illustrate how the measure can be useful for examining conversations. We start by validating our method via two complementary approaches. In a subset of sentences manually annotated with the counseling 5 Further implementation details are listed in the appendix. 6 Example sentences are derived from real sentences in the data, and modified for readability. The examples were chosen to reflect common situations in the data, and were vetted by the platform to ensure the privacy of counselors and texters. strategies they exhibit, we show that orientation meaningfully reflects these strategies (Section 5.1). At a larger scale, we show that the orientation of utterances over the course of a conversation aligns with domain knowledge about counseling conversation structure (Section 5.2). We also find that other measures for characterizing utterances are not as rich as orientation in capturing counseling strategies and conversation structure (Section 5.3). Finally, we show that a counselor's orientation in a conversation is tied to indicators of their effectiveness in helping the texter (Section 5.4). Even though it is computed without the guidance of any annotations, we expect orientation to meaningfully reflect strategies for advancing or addressing that crisis counselors are taught. The first author hand-labeled 400 randomly-selected sentences with a set of pre-defined strategies derived from techniques highlighted in the training material. We note example sentences in Table 1 which exemplify each strategy, and provide more extensive descriptions in the appendix. Figure 4A depicts the distributions of orientations across each label, sorted from most backwardsto most forwards-oriented. We find that the relative orientation of different strategies corroborates their intent as described in the literature. Statements reflecting or affirming what the texter has said to check understanding or convey empathy (characterized by phrasings like totally normal ) tend to be backwards-oriented; statements prompting the texter to advance towards problem-solving (e.g., [what] has helped ) are more forwards-oriented. Exploratory queries for more information on what the texter has already said (e.g., happened to make ) tend to have middling orientation (around 0). The standard deviation of orientations over messages within most of the labels is significantly lower than across labels (bootstrapped p < : 05 , solid circles), showing that orientation yields interpretable groupings of messages in terms of important counseling strategies. The measure also offers complementary information. For instance, we find sentences that aren't accounted for by pre-defined labels, but still map to interpretable orientations, such as backwards-oriented examples assuaging texter concerns about the platform being a safe space to self-disclose. We also show that orientation tracks with the structure of crisis counseling conversations as described in the training material. Following Althoff et al. (2016), we divide each conversation with at least ten counselor messages into five equal-sized segments and average  max and  min over messages in each segment. Figure 4B (black lines) shows that over the course of a conversation, messages tend to get more forwards-oriented (higher  max and  min ). This matches a standard conversation structure taught in the training: addressing the texter's existing problems before advancing towards problem-solving. While this correspondence holds in aggregate, orientation also captures complementary information to advancement through stagese.g., while problem-solving, counselors may still address and affirm a texter's ideas (Table 1, row 3). We also consider a subset of conversations where we expect a different trajectory: for potentially suicidal texters, the training directs counselors to immediately start a process of risk assessment in which actively prompting the texter to disclose their level of suicidal ideation takes precedence over other objectives. As such, we expect more forwards-oriented messages at the starts of conversations involving such texters. Indeed, in the 30% of conversations which are risk-assessed, we find significantly larger  max in the first segment (Figure 4B, orange line; Wilcoxon p < 0 : 01 in the first stage, comparing within-counselor). min is smaller at each stage, suggesting that counselors balance actively prompting these critical disclosures with addressing them. capturing a counselor's balancing decisions: Naive distance. We conside the naive approach in Section 4, taking a difference in cosine distances between tf-idf representations of a message and its reply, and a message and its predecessor. Backwards-range. We consider just the mes-sage's backwards-range. For each sentence we take tf-idf weighted averages of component (cid:0) (cid:27) w and take minimum (cid:0) (cid:27) for each message. 7 7 We get qualitatively similar results with maximum (cid:0)! (cid:27) . Question-asking. We consider whether the message has a question. This was used in Walker and Whittaker (1990) as a signal of taking control, which could be construed as forwards-oriented. Within-label standard deviations of each alternative measure are generally not significantly smaller than across-label (Figure 4A), indicating that these measures are poorer reflections of the counseling strategies. Label rankings under the measures are also arguably less intuitive. For instance, reflection statements have relatively large (naive) cosine distance from their predecessors. Indeed, the training encourages counselors to process rather than simply restate the texter's words. These measures also track with the conversation's progress differentlynotably, none of them distinguish the initial dynamics of risk-assessed conversations as reflected in  max (see appendix). Past work on counseling has extensively discussed the virtues of addressing a client's situation (Rogers, 1957; Hill and Nakayama, 2000). Some studies also suggest that accounting for both addressing and advancing is important, such that effective counselors manage to mix backwardsand forwards-oriented actions (Mishara et al., 2007). We use orientation to examine how these strategies are tied to conversational effectiveness in crisis counseling at a larger scale, using our measures to provide a unified view of advancing and addressing. To derive simple conversation-level measures, we average  max and  min over each counselor message in a conversation. Adjudicating counseling conversation quality is known to be difficult (Tracey et al., 2014). As a starting point, we relate our conversation-level measures to two complementary indicators of a conversation's effectiveness: 8 Perceived helpfulness. We consider responses from a post-conversation survey asking the texter whether the conversation was helpful, following Althoff et al. (2016). Out of the 26% of conversations with a response, 89% were rated as helpful. 9 Conversation length. We consider a conversation's length as a simple indicator of the pace of its progress: short conversations may rush the texter, while prolonged conversations could suggest 8 We perform all subsequent analyses on a subset of 234,433 conversations, detailed in the appendix. Figure 5A compares  min and  max in conversations rated as helpful and unhelpful by texters. Both measures are significantly smaller in conversations perceived as helpful, suggesting that texters have a better impression of relatively backwards-oriented interactions where the counselor is inclined towards addressing their situation. As such, this result echoes past findings relating addressing to effectiveness. Figure 5B compares  min in conversations of varying lengths, showing that  min increases with length, such that counselors exhibit less propensity for addressing in longer conversations. Anecdotal observations cited in interviews with the platform's staff suggest one interpretation: conversations in which a texter feels their concerns were not satisfactorily addressed may be prolonged when they circle back to revisit these concerns. Figure 5C relates  max to conversation length. We find that  max is smaller in the lengthiest conversations, suggesting that such prolonged in-10 As the training material notes, conversation length and texter perception may signal complementary or even conflicting information about a texter's experience of a conversation and its effectiveness: Some texters resist the end of the conversation. They ruminate [...] causing the conversation to drag on without any progress. 5284 teractions may be stalled by a weaker impulse to advance forwards. Extremely short conversations have smaller  max as well, such that premature endings may also reflect issues in advancing. As such, we add credence to the previously-posited benefits of mixing addressing and advancing: forwards-oriented actions may be tied to making progress, while a weaker propensity to advance may signal a suboptimal pace. Counselor-level analysis. These findings could reflect various confoundsfor instance, a counselor's choice of orientation may have no bearing on the rating they receive from a particularly difficult texter. To address this, we compute similar correspondences between orientation and our effectiveness indicators at the level of counselors rather than conversations; this analysis is detailed in the appendix. Our conversation-level results are replicated under these controls. In this work, we sought to examine a key balance in crisis counseling conversations between advancing forwards and addressing what has already been said. Realizing this balance is one of the many challenges that crisis counselors must manage, and modeling the actions they take in light of such challenges could point to policies to better support them. For instance, our method could assist human supervisors in monitoring the progress of ongoing conversations to detect instances of rushing or stalling, or enable larger-scale analyses of conversational behaviors to inform how counselors are trained. The unsupervised approach we propose could circumvent dif-ficulties in getting large-scale annotations of such sensitive content. Future work could bolster the measure's usefulness in several ways. Technical improvements like richer utterance representations could improve the measure's fidelity; more sophisticated analyses could better capture the dynamic ways in which the balance of objectives is negotiated across many turns. The preliminary explorations in Section 5.4 could also be extended to gauge the causal effects of counselors' behaviors (Kazdin, 2007). We expect balancing problems to recur in conversational settings beyond crisis counseling, such as court proceedings, interviews, debates and other mental health contexts like long-term therapy. In these settings, individuals also make potentially consequential choices that span the backwards-forwards orientation axis, such as addressing previous arguments (Tan et al., 2016; Zhang et al., 2016) or asking leading questions (Leech, 2002). Our measure is designed to be broadly applicable, requiring no domain-specific annotations; we provide exploratory output on justice utterances from the Supreme Court's oral arguments in the appendix and release code implementing our approach at http://convokit.cornell.edu to encourage experiments in other domains. However, the method's efficacy in the present setting is likely boosted by the relative uniformity of crisis counseling conversations; and future work could aim to better accomodate settings with less structure and more linguistic variability. With such improvements, it would be interesting to study other domains where interlocutors are faced with conversational challenges. We thank Jonathan P. Chang, Caleb Chiam, Liye Fu, Dan Jurafsky, Jack Hessel, and Lillian Lee for helpful conversations, and the anonymous reviewers for their thoughtful comments. We also thank Ana Smith for collecting and processing the Supreme Court oral argument transcripts we used in the supplementary material. This research, and the counseling service examined herein, would not have been possible without Crisis Text Line. We are particularly grateful to Robert Filbin, Christine Morrison, and Jaclyn Weiser for their valuable insights into the experiences of counselors and for their help with using the data. The research has been supported in part by NSF CAREER Award IIS1750615 and a Microsoft Research PhD Fellowship. The collaboration with Crisis Text Line was supported by the Robert Wood Johnson Foundation; the views expressed here do not necessarily reflect the views of the foundation.
abstain abstain objective objective abstain method result abstain abstain abstain other abstain abstain abstain abstain abstain abstain method objective objective objective method objective objective objective method result objective objective objective objective abstain abstain abstain abstain abstain abstain abstain abstain abstain method other other other other other other other other other other other other other other other other abstain abstain objective objective other other objective other abstain method method other other other objective method method other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain method method abstain method abstain method abstain abstain method abstain method abstain method abstain abstain abstain abstain abstain method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain abstain other other other other other other