Task-oriented dialogue systems are increasingly prevalent in healthcare settings, and have been characterized by a diverse range of architectures and objectives. Although these systems have been surveyed in the medical community from a non-technical perspective, a systematic review from a rigorous computational perspective has to date remained noticeably absent. As a result, many important implementation details of healthcare-oriented dialogue systems remain limited or under-specified, slowing the pace of innovation in this area. To fill this gap, we investigated an initial pool of 4070 papers from well-known computer science, natural language processing, and artificial intelligence venues, identifying 70 papers discussing the system-level implementation of task-oriented dialogue systems for healthcare applications. We conducted a comprehensive technical review of these papers, and present our key findings including identified gaps and corresponding recommendations. Dialogue systems 1 have a daily presence in many individuals' lives, acting as virtual assistants (Hoy, 2018), customer service agents (Xu et al., 2017), or even companions (Zhou et al., 2020). While some systems are designed to conduct unstructured conversations in open domains ( chatbots ), others ( task-oriented dialogue systems ) help users to complete tasks in a specific domain (Jurafsky and Martin, 2009; Qin et al., 2019). Task-oriented dialogue systems can potentially play an important role in health and medical care (Laranjo et al., 2018), and they have been adopted by growing numbers of patients, caregivers, and clinicians (Kearns et al., 2019). Nonetheless, there remains a translational 1 We follow an inclusive definition of dialogue systems , encompassing any intelligent systems designed to converse with humans via natural language. gap (Newman-Griffis et al., 2021) between cutting-edge, foundational work in dialogue systems and prototypical or deployed dialogue agents in healthcare settings. This limits the proliferation of scientific progress to real-world systems, constraining the potential benefits of fundamental research. We move towards closing this gap by conducting a comprehensive, scientifically rigorous analysis of task-oriented healthcare dialogue systems. Our underlying objectives are to (a) explore how these systems have been employed to date, and (b) map out their characteristics, shortcomings, and subsequent opportunities for follow-up work. Importantly, we seek to address the limitations of prior systematic reviews by extensively investigating the included systems from a computational perspective. Our primary contributions are as follows: 1. We systematically search through 4070 papers from well-known technical venues and identify 70 papers fitting our inclusion criteria. 2 2. We analyze these systems based on many factors, including system objective, language, architecture, modality, device type, and evaluation paradigm, among others. 3. We identify common limitations across systems, including an incomplete exploration of architecture, replicability concerns, ethical and privacy issues, and minimal investigation of usability or engagement. We offer practical suggestions for addressing these as an on-ramp for future work. In the long term, we hope that the gaps and opportunities identified in this survey can stimulate more rapid advances in the design of task-oriented healthcare dialogue systems. We also hope that the survey provides a useful starting point and synthesis of prior work for NLP researchers and practi-2 A full listing of these papers is provided in the appendix. Dialogue systems in healthcare have been the focus of several recent surveys conducted by the medical and clinical communities (Vaidyam et al., 2019; Laranjo et al., 2018; Kearns et al., 2019). These surveys have investigated the real-world utilization of deployed systems, rather than examining their design and implementation from a technical perspective. In contrast, studies examining these systems through the lens of AI and NLP research and practice have been limited. Zhang et al. (2020) and Chen et al. (2017) presented surveys of recent advances in general-domain task-oriented dialogue systems. Although they provide an excellent holistic portrait of the subfield, they do not delve into aspects of particular interest in healthcare settings (e.g., system objectives doubling as clinical goals), limiting their usefulness for this audience. Vaidyam et al. (2019), Laranjo et al. (2018), and Kearns et al. (2019) conducted systematic reviews of dialogue systems deployed in mental health (Vaidyam et al., 2019) or general healthcare (Laranjo et al., 2018; Kearns et al., 2019) settings. Vaidyam et al. (2019) examined 10 articles, and Laranjo et al. (2018) and Kearns et al. (2019) examined 17 and 46 articles, respectively. All surveys were written for a medical audience and focused on healthcare issues and impact, covering few articles from AI, NLP, or general computer science venues. Montenegro et al. (2019) and Tudor Car et al. (2020) recently reviewed 40 and 47 articles, respectively, covering conversational agents in the healthcare domain. These two surveys are the closest to ours, but differ in important ways. First, our focus is on a specific class of conversational agents: task-oriented dialogue systems. The surveys by Montenegro et al. (2019) and Tudor Car et al. (2020) used a wider search breading their ability to provide extensive technical depth. We also reviewed more papers (70 articles), which were then screened using a more thorough taxonomy as part of the analysis. Some aspects that we considered that differ from these prior surveys include the overall dialogue system architecture, the dialogue management architecture, the system evaluation methods, and the dataset(s) used when developing and/or evaluating the system. We designed search criteria in concert with our goal of filling a translational information gap between fundamental dialogue systems research and applied systems in the healthcare domain. To do so, we retrieved articles from well-respected computer science, AI, and NLP databases and screened them for focus on task-oriented dialogue systems designed for healthcare settings. Our target databases were: (1) ACM, 3 (2) IEEE, 4 (3) the ACL Anthology, 5 and (4) the AAAI Digital Library. 6 ACM and IEEE are large databases of papers from prestigious conferences and journals across many CS fields, including but not limited to robotics, human-computer interaction, data mining, and multimedia systems. The ACL Anthology is the premier database of publications within NLP, hosting papers from major conferences and topic-specific venues (e.g., SIGDIAL , organized by the Special Interest Group on Discourse and Dialogue). The AAAI Digital Library hosts papers not only from the AAAI Conference on Artificial Intelligence , but also from other AI conferences, AI Magazine , and the Journal of Artificial Intelligence Research . We applied the following inclusion criteria when identifying papers:  The main focus must be on the technical design or implementation of a task-oriented dialogue system. The system must be designed for health-related applications. The article must not be dedicated to one specific module of the system's architecture (e.g., 3 https://dl.acm.org/ 4 https://ieeexplore.ieee.org/ 5 https://www.aclweb.org/anthology/ 6 https://aaai.org/Library/library.php 6639 the natural language understanding component of a health-related dialogue system). Although a narrower scopee.g., developing improved methods for slot-fillingis common when publishing in the dialogue systems community, these papers tend to place more emphasis on technical design irrespective of application context, offering less coverage of the system-level characteristics that are the target of this survey. We followed four steps in our screening process. First ( Initial Search) , we applied a predefined search query to the databases to populate our initial list of papers. To generate the query, we used the keywords task-oriented, dialogue system, conversational agent, health, and healthcare, and synonyms and abbreviations of these keywords. We short-listed papers using these keywords individually as well as in combination with one another. Next ( Title Screening ), we performed a preliminary screening through the initial list of papers by reading the titles, keeping those that satisfied the inclusion criteria. Then ( Abstract Screening ), we went through the list of papers remaining after the title screening and read the abstracts, keeping those that satisfied the inclusion criteria. Lastly ( Final Screening ), we read the body of the papers remaining after the abstract screening and kept those that satisfied the inclusion criteria. These funnel filtering processes were conducted by a computer science graduate student (a fluent L2 English speaker) using predefined search and screening guidelines. Questions or uncertainties regarding a paper's compliance with inclusion criteria were forwarded along to the senior project lead (a computer science professor and fluent L1 English speaker with expertise in NLP) and final consensus was reached via discussion among the two parties. We detail the number of papers remaining after each screening step in Table 1. Overall, this screening process combined with our subsequent surveying methods spanned eight months, covering papers published prior to January 2021. In total, 70 papers (21 from ACM, 31 from IEEE, 16 from ACL, and 2 from AAAI 7 ) satisfied the inclusion criteria. We survey papers meeting our inclusion criteria according to a wide range of parameters, and present our findings in the following 7 Papers about task-oriented dialogue systems published at AAAI often focus on one specific component of the system from a technical perspective, rather than proposing a conversational agent as a whole. Therefore, only two papers from the AAAI Digital Library satisfied the inclusion criteria. subsections, grouped into thematic categories: ontology (4), system architecture (5), system design (6), dataset (7), and system evaluation (8). We map each paper to its domain of research (4.1), system objective (4.2), target audience (4.3), and language (4.4), and present our findings. Task-oriented dialogue systems can potentially impact many facets of healthcare in society (Bick-more and Giorgino, 2004). We define a domain of research as the healthcare area in which the system operates. We identify both broad domains and more specific subcategories thereof based on the systems surveyed, outlined in Figure 1. Broad domain categories include mental health , physical health , health information , patient assistance , physician assistance , cognitive or developmental health , and other (comprising subcategories not easily classifiable to one of the broader domains). Systems in the mental health domain supported individuals with mental or psychological health conditions, and systems in the cognitive or developmental health domain were a close analogue for individuals with conditions impacting memory, executive, or other cognitive function. Systems in the physical health domain were targeted towards individuals with specific physical health concerns, including infectious (e.g., Covid-19), noninfectious (e.g., cancer), and temporary (e.g., preg-6640 System Objective # Papers Diagnosis 7 Monitoring 8 Intervention 13 Counseling 5 Assistance 12 Multi-Objective 25 Table 2: Distribution of system objectives across the surveyed papers. Additional details regarding multi-objective papers are provided in the appendix. nancy) conditions. Systems providing health information performed general-purpose actions such as offering advice or suggesting disease diagnoses. Finally, systems performing patient assistance or physician assistance supported specific patientor physician-focused healthcare tasks. Dialogue systems designed for mental health , physical health , and health information were the most prevalent, covering 51 of the 70 included papers. Task-oriented dialogue systems define value relative to the goals of a target task. We define the system objective as the healthcare task for which a system is designed. Some system objectives may be closely aligned with a single domain, whereas others may occur in many different domains (e.g., monitoring mental, physical, or cognitive condi-tions). Thus, although the domain of research and system objective may frequently correlate, there is not by necessity a direct association. Included systems were categorized as being designed to: diagnose a health condition (e.g., by predicting whether the user suffers from cognitive de-cline); monitor user states (e.g., by tracking their diets or periodically checking their mood); intervene by addressing users' health concerns or improving their states (e.g., by teaching children how to map facial expressions to emotions); counsel users without providing any direct intervention (e.g., by listening to users' concerns and empathizing with them); or assist users by providing information or guidance (e.g., by answering questions from users who are filling out forms). Many systems were also categorized as multi-objective , meaning that they were designed for more than one of those goals. signed for more than one target objective. Among multi-objective systems, those that were designed for both diagnosis and assistance had the highest frequency (7/25); we provide additional details regarding these systems in Table 8 of the appendix. Separately, we also considered the role of engagement as an objective of each system. We define this as a goal of engaging target users in interaction, irrespective of underlying health goals. Engagement may be of particular interest in healthcare settings since it can be critical in encouraging adoption or adherence with respect to healthcare outcomes (Montenegro et al., 2019). Surprisingly, almost 60% of the papers (41 of the 70 surveyed) did not mention any goals pertaining to engaging users in more interactions. The final consumers of healthcare systems often fall into three groups: patients , caregivers , and clinicians . Table 3 shows the number of systems surveyed that focus on each category. We find that out of 70 task-oriented dialogue systems, 59 are designed specifically for patients. Most general-domain dialogue systems research has been conducted in English and other high-resource languages (Artetxe et al., 2020). Expanding language diversity may extend the benefits of health-related dialogue systems more globally. As shown in Figure 2, among the systems included in our review a majority (56%) are designed for English speakers. Encouragingly, several of the included systems did focus on lower-resource languages, including Telugu (Duggenpudi et al., 2019), Bengali (Rahman et al., 2019), and Setswana (Grover et al., 2009). We investigate both the general architecture of the system (5.1), and if applicable, the dialogue man-6641 Task-oriented dialogue systems are generally designed using pipeline or end-to-end architectures. Pipeline architectures typically consist of separate components for natural language understanding, dialogue state tracking, dialogue policy, and natural language generation. The ensemble of the dialogue state tracker and dialogue policy is the dialogue manager (Chen et al., 2017). End-to-end architectures train a single model to produce output for a given input, often interacting with structured external databases and requiring extensive training data (Chen et al., 2017). As shown in Table 4, only 2.85% of papers (2 of the 70 surveyed) implemented an end-to-end system; this is unsurprising given the limited training data available in most healthcare domains. We also found that 14% (10 papers) did not directly specify the architecture of their developed system. Unlike other pipeline components that impact user experience and engagement but not fundamental decision-making, the dialogue manager is central to overall functionality (Zhao et al., 2019); thus, we afford it special attention. In rule-based approaches, the system interacts with users based on a predefined set of rules, with success conditioned upon coverage of all relevant cases (Siangchin and Samanchuen, 2019). Intent-based approaches seek to extract the user's intention from the dialogue, and then perform the relevant action (Jurafsky and Martin, 2009). In hybrid dialogue management architectures, the system leverages a combination of rule-based and intent-based approaches, and fi-nally corpus-based approaches mine the dialogues of human-human conversations and produce responses using retrieval methods or generative methods (Jurafsky and Martin, 2009). As shown in Table 5, among papers reporting on dialogue management architecture, we observe a fairly even mix of rule-based, intent-based, and hybrid architectures. Modality, the channel through which information is exchanged between a computer and a human (Karray et al., 2008), can play an important role in dialogue quality and user satisfaction (Bilici et al., 2000). Unimodal systems use a single modality for information exchange, whereas multimodal systems use multiple modalities (Karray et al., 2008). Systems reviewed in this survey operated using one or more of several modalities. In text-based or spoken interaction, users interact with the system by typing or speaking, respectively. In interaction via graphical user interface (GUI) , users interact with the system through the use of visual elements. In general, multimodal dialogue systems can be flexible and robust, but especially challenging to implement in the medical domain (Sonntag et al., 2009). We find that 49 papers describe unimodal systems and 21 describe multimodal systems. Ta-6642 Unimodal Multimodal Category # Papers Category # Papers Text 23 Spoken + Text 14 Spoken 25 Spoken + GUI 4 GUI 1 Text + GUI 3 Table 6: Distribution of modality type across the unimodal (49 total, left) and multimodal (21 total, right) systems surveyed. ble 6 provides more details regarding their distribution across modalities. Dialogue systems may facilitate interaction using a variety of devices (Arora et al., 2013), ranging from telephones (Garvey and Sankaranarayanan, 2012) to computers (McTear, 2010) to any other technology that allows interaction (e.g., VR-based avatars (Brinkman et al., 2012b; McTear, 2010)). We categorized the included systems as mobile , telephone , desktop/laptop , in-car , PDA , robot , virtual environment , or virtual reality (including virtual agents and avatars) systems, considering systems as multi-device if they leveraged multiple devices for interaction. As shown in Figure 3, we found that multi-device and mobile-based dialogue systems were most popular. Table 9 in the appendix provides additional details regarding multi-device systems. Data is crucial for effective system development (Serban et al., 2015), but many datasets for training dialogue systems are smaller than those used for other NLP tasks (Lowe et al., 2017). This is even more pronounced in the healthcare domain, in part due to the risk of data misuse by others or the lack of data sharing incentives (Lee and Yoon, 2017). We reviewed each paper for information regarding the data used during system development, focusing on dataset size, availability, and privacy-preserving measures. Only 20 papers provide details about the data used (two papers provided a link to the dataset, and the remaining 18 discussed the dataset size). Unfortunately, the remaining papers did not provide rationale for their lack of data or other replicability information. Our assumption is that often the data contained sensitive information, preventing authors from releasing specific details, but only 19 of the 70 included papers provided information about data-related privacy or ethical considerations. Only 10 mentioned Institutional Review Board (IRB) approval for their dataset and/or task, despite IRB (or equivalent) review being a crucial step towards ensuring that research is conducted ethically and in such a way that protects human subjects to the extent possible (Amdur and Biddle, 1997). We examined the means through which systems were evaluated both qualitatively and quantitatively (Deriu et al., 2019; Hastie, 2012). We defined human evaluation , often implemented in prior work through questionnaires (Grover et al., 2009; Holmes et al., 2019; Parde and Nielsen, 2019; Wang et al., 2020) or direct feedback from real-world users (Deriu et al., 2019), as an evaluation that relies on subjective, first-hand, human user experience. In contrast, automated evaluation provides an objective, quantitative measurement of one or more dimensions of the system from a mathematical perspective (Finch and Choi, 2020). Some metrics used for automated evaluation of the reviewed systems include measures of task performance (Ali et al., 2020) and completion rates (Holmes et al., 2019), response correctness (Ros-ruen and Samanchuen, 2018), and response time (Grover et al., 2009). In Table 7, we observe that nearly half of the papers conducted human evaluations; however, a large percentage (37%) also did not discuss evaluation at all. We further analyzed papers conducting human evaluations and found that they included an average of 26 (mode = 12) participants. More details regarding the human and automated evaluations are provided in Tables 10, 11, and 12 of the appendix. In a follow-up analysis of system usability , defined as the degree to which users are able to engage with a system safely, effectively, efficiently, and enjoyably (Lee et al., 2019), we observed that 33 papers explicitly evaluated the usability of their system. We identify common limitations across many surveyed systems, accompanied by recommendations for addressing them in future work. 9.1 Incomplete Exploration of System Design We observed little system-level architectural diversity across the surveyed systems, with most (83%) having a pipeline architecture. This architectural homogeneity limits our understanding of good design practice within this domain. Recent studies demonstrate that end-to-end architectures for task-oriented dialogue systems could compete with pipeline architectures given sufficient high-quality data (Hosseini-Asl et al., 2020; Ham et al., 2020; Bordes et al., 2017; Wen et al., 2016). However, the external knowledge sources often leveraged in end-to-end systems are notoriously complex in many healthcare sub-domains (Campillos-Llanos et al., 2020). Additionally, for healthcare applications interpretability is highly desired (Ham et al., 2020), but explanations are often obfuscated in end-to-end systems (Ham et al., 2020; Wen et al., 2016). Finally, users of these systems may seek guidance on sensitive topics, which can exacerbate privacy concerns (Xu et al., 2021). Any system trained on large, weakly curated datasets may also learn unpleasant behaviors and amplify biases in the training data, in turn producing harmful consequences (Dinan et al., 2021; Bender et al., 2021). We recommend further experimentation with architectural design, in parallel with work towards developing high-quality healthcare dialogue datasets, which to date remain scarce (Farzana et al., 2020). interaction. However, it is well-established that individuals from certain demographic groups are more comfortable conversing with dialogue systems via speech (Tudor Car et al., 2020). Text-based systems may also be more likely to violate privacy considerations (Tudor Car et al., 2020). Thus, we recommend that researchers engage in further exploration of multimodal or spoken dialogue systems when applicable and appropriate. Many of the surveyed systems were also implemented on mobile phones. Although an advantage of mobile-based systems is that they are readily available using a technology familiar to most users, Lee et al. (2018) found that users significantly reduced their usage over time when engaging long-term with mobile health applications. Tudor Car et al. (2020) suggest that one way to overcome this limitation in mobile-based systems is by directly embedding them in applications or platforms with which users already engage habitually (e.g., Facebook Messenger). This more ambient dissemination approach may facilitate easier and more lasting integration of system use in individuals' daily lives. Finally, we identified that most systems (84%) target only patients, with research on systems targeted towards clinicians and caregivers remaining limited. We recommend further exploration of systems targeted towards these critical audiences. This may offer broad, high-impact support in understanding, diagnosing, and treating patients' health issues (Valizadeh et al., 2021; Kaelin et al., 2021). Data accessibility restrictions reduce the capacity of public health research (Strongman et al., 2019), and these limitations may be partially responsible for the imbalance of pipeline versus end-to-end architectures (9.1). Only a small percentage of papers surveyed (29%) ventured to discuss the quantity or characteristics of the data used during system development in any way. A lack of data transparency hinders scientific progress and severely impedes replicability. We call upon researchers to publish data when permissible by governing protocol, and descriptive statistics to the extent allowable when circumstances prevent data release. We also view the development of high-quality, publicly available datasets as an important frontier in translational dialogue systems research (9.1). ods (34%). This prevents the research community from replicating developed systems and generalizing study findings more broadly (Walker et al., 2018). Well-established guidelines exist and are being increasingly enforced within the NLP community to prevent reproducibility issues (Dodge et al., 2019). The disregard of reproducibility best practices observed with many healthcare dialogue systems may be partially attributed to the most common target venues for this work, which may place less emphasis on replication. This validates a central motivator for publishing this surveywithout adequate inclusion of target domain and technical stakeholders in interdisciplinary, translational research, progress will remain constrained. We strongly urge researchers in this domain to provide implementation details in their publications. Real-world medical data facilitates the development of high-quality healthcare applications (Bertino et al., 2005; Di Palo and Parde, 2019; Farzana et al., 2020), but protecting the rights and privacy of contributors to the data is critical for ensuring ethical research conduct (Institute of Medicine, 2009), as is proper treatment of copyright protections. We screened all included papers for coverage of privacy and ethical concerns, and observed that only 27% of the surveyed papers considered participant or patient privacy in the design of their system. Moreover, only 14% of the surveyed papers documented any evidence of Institutional Review Board (or IRB-equivalent) approval. Research involving healthcare dialogue systems is unquestionably human-centered, and as such the absence of ethical oversight in the design of such systems is a grave concern. Although technical researchers entering this space may be unfamiliar with human subjects research and protocol, we urge all dialogue systems researchers to submit their experimental design and protocol for review by an appropriate external review board. We also ask that researchers consider the potential harms from use or misuse of their systems, following guidelines established by the ACM Code of Ethics. 8 9.4 Room for Increased Language Diversity We observed that most systems (56%) targeted English speakers. Developing multilingual dialogue systems or systems for speakers of low-resource 8 https://www.acm.org/code-of-ethics languages brings up various challenges (Lpez-Czar Delgado and Araki, 2005), but solving this problem could have have tremendous benefit for individuals in non-English speaking communities with minimal or unreliable healthcare access. The systems developed by Duggenpudi et al. (2019), Rahman et al. (2019), and Grover et al. (2009) provide case examples for how such systems may be implemented. We also note that while troubling, a 56% share of systems targeted towards English speakers is consistent with linguistic homogeneity in the field in general, and actually slightly low relative to many other NLP tasks (Mielke, 2016; Bender, 2009). Healthcare dialogue systems may on some level offer a case example for how applications originally designed for high-resource (i.e., English-language) settings can be adapted and re-engineered to provide better coverage of the diverse, real-world potential user base. Finally, more than 50% (37/70) of the included papers did not evaluate system usability or general user experience. Usability testing can improve productivity and safeguard against errors (Rogers et al., 2005), both of which are critical in healthcare tasks. Therefore, we urge the research community to consider and assess usability when designing for this domain. The systems among those surveyed that do this already (e.g., those developed by Wang et al. (2020), Lee et al. (2020b), Wei et al. (2018), or Demasi et al. (2020)) provide case examples for how it might be done. Almost 60% of the surveyed systems were not explicitly designed to engage users, despite this being a common objective in the general domain (Ghazarian et al., 2019). Healthcare dialogue systems may stand to benefit particularly well from such measures (Parde, 2018), since patient engagement is predictive of adoption and adherence to healthcare outcomes (Montenegro et al., 2019). To increase user satisfaction and system performance, we recommend that the research community more purposefully consider engagement when designing their healthcare-oriented dialogue systems. In this work, we conducted a systematic technical survey of task-oriented dialogue systems used for health-related purposes, providing much-needed analyses from a computational perspective and narrowing the translational gap between basic and applied dialogue systems research. We comprehensively searched through 4070 papers in computer science, NLP, and AI databases, finding 70 papers that satisfied our inclusion criteria. We analyzed these papers based on numerous technical factors including the domain of research, system objective, target audience, language, system architecture, system design, training dataset, and evaluation methods. Following this, we identified and summarized gaps in this existing body of work, including an incomplete exploration of system design, replicability concerns, potential ethical and privacy issues, room for increased language diversity, and minimal investigation of usability or user engagement. Finally, we presented evidence-based recommendations stemming from our findings as a launching point for future work. It is our hope that interested researchers find the information provided in this survey to be a unique and helpful resource for developing task-oriented dialogue systems for healthcare applications. Beyond the concrete changes suggested during the discussion, it is important to consider the broader ethical implications of task-oriented dialogue systems in healthcare settings. Although the goal of such systems may not be to replace human healthcare providers, it is likely that deployed systems would support clinicians, defraying workload for overburdened individuals. In doing so, these systems may have significant impact on healthcare decision-making. Machines are imperfect, and thus a possible harm is that these systems may misinterpret user input or make incorrect predictions a mistake that in high-stakes healthcare settings could prove detrimental or even dangerous. Researchers and developers should be cognizant of possible harms stemming from the use and misuse of task-oriented dialogue systems for healthcare settings, and should implement both automated (e.g., strict thresholds for diagnostic suggestions) and human (e.g., training to ensure staff awareness of potential system fallibilities) safeguards. Moreover, a potential benefit of these systems is their potential to meaningfully and beneficially extend healthcare access to underserved populations. As such, it is important to ensure that automated systems do not fall prey to the same biases often observed among human healthcare providers (FitzGerald and Hurst, 2017). Systems trained to perform healthcare tasks using datasets that are not representative of the target population may exhibit poorer performance with users who already experience marginalization or are otherwise vulnerable, impeding or even reversing benefits. We call upon researchers to examine, debias, and curate their training data such that task-oriented dialogue systems for healthcare applications elevate, rather than diminish, outcomes for the historically underserved users which they are best poised to benefit. This material is based upon work supported by the National Science Foundation under Grant No. 2125411, and by a start-up grant from the University of Illinois at Chicago. Any opinions, findings, and conclusions or recommendations are those of the authors and do not necessarily reflect the views of the National Science Foundation. We thank the anonymous reviewers for their insightful suggestions, which further strengthened this work.
abstain abstain abstain objective method abstain abstain abstain method abstain abstain method objective abstain abstain objective objective method objective method abstain method method other other other other other other other other other other objective other method objective abstain method abstain abstain abstain abstain method abstain other abstain method abstain method method method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain other abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method result method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain method abstain result abstain abstain abstain method result other abstain abstain abstain abstain abstain method abstain abstain abstain method objective abstain result objective objective method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain other other other