## Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work

WESLEY HANWEN DENG ∗ , Carnegie Mellon University, USA SOLON BAROCAS † , Microsoft Research, USA JENNIFER WORTMAN VAUGHAN † , Microsoft Research, USA

Recent years have witnessed increasing calls for computing researchers to grapple with the societal impacts of their work. Tools such as impact assessments have gained prominence as a method to uncover potential impacts, and a number of publication venues now encourage authors to include an impact statement in their submissions. Despite this recent push, little is known about the way researchers go about assessing, articulating, and addressing the potential negative societal impact of their work - especially in industry settings, where research outcomes are often quickly integrated into products and services. In addition, while there are nascent efforts to support researchers in this task, there remains a dearth of empirically-informed tools and processes. Through interviews with 25 industry computing researchers across different companies and research areas, we identify four key factors that influence how they grapple with (or choose not to grapple with) the societal impact of their research: the relationship between industry researchers and product teams; organizational dynamics and cultures that prioritize innovation and speed; misconceptions about societal impact; and a lack of sufficient infrastructure to support researchers. To develop an effective impact assessment template tailored to industry computing researchers' needs, we conduct an iterative co-design process with these 25 industry researchers, along with an additional 16 researchers and practitioners with prior experience and expertise in reviewing and developing impact assessments or responsible computing practices more broadly. Through the co-design process, we develop 10 design considerations to facilitate the effective design, implementation, and adaptation of an impact assessment template for use in industry research settings and beyond, as well as our own 'Societal Impact Assessment' template with concrete scaffolds. We explore the effectiveness of this template through a user study with 15 industry research interns, revealing both its strengths and limitations. Finally, we discuss the implications for future researchers, organizations, and policymakers seeking to foster more responsible research practices.

Additional Key Words and Phrases: Responsible Computing, Ethics, Impact Assessment

## ACMReference Format:

Wesley Hanwen Deng, Solon Barocas, and Jennifer Wortman Vaughan. 2025. Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work. Proc. ACM Hum.-Comput. Interact. 9, 2, Article CSCW178 (April 2025), 37 pages. https://doi.org/10.1145/3711076

∗ This work was conducted while the author was an intern at Microsoft Research. † Both authors contributed equally to this research.

Authors' addresses: Wesley Hanwen Deng, hanwend@cs.cmu.edu, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA, 15213, USA; Solon Barocas, solon@microsoft.com, Microsoft Research, 300 Lafayette St., New York, NY, 10012, USA; Jennifer Wortman Vaughan, jenn@microsoft.com, Microsoft Research, 300 Lafayette St., New York, NY, 10012, USA.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

2573-0142/2025/4-ARTCSCW178 $15.00

© 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM.

## 1 INTRODUCTION

In recent years, many organizations across the academy [2, 9, 18, 32, 35, 61, 68, 73, 80, 98], government [96, 97, 138, 140], civil society [3, 88, 99, 113, 128, 138], and industry [61, 89, 90, 100, 143] have published reports and explored tools and processes to better support computing researchers in grappling with the potential negative societal impact of their research. Among other tools, impact assessments have become popular as a way to support both researchers and practitioners in assessing, articulating, and addressing the impact of their work [33, 56, 88, 93, 138]. At a high level, an impact assessment is a systematic process used to identify, assess, and address the potential impact of a proposed project, program, or policy on people, organizations, and society as a whole [37, 88, 90].

To date, very little of this work has focused on the way industry computing researchers in particular go about assessing and addressing the potential negative societal impacts of their work. Despite a growing line of research on building tools to support practitioners (e.g., product teams) in reflecting and anticipating the societal impact of their work [13, 46, 95, 146, 154], prior work from CSCW and the broader field of HCI suggests that resources for responsible computing in industry are not as readily available as one might hope [40, 62, 64, 82, 111, 145, 152], raising the prospect that industry researchers are similarly lacking in support. Given the proximity that industry research has to practice - and thus its increased likelihood of being integrated into products and services there is an acute need to better understand how industry researchers are currently approaching this challenge. This need is especially urgent in light of the recent rise of generative AI and the risks it poses [16, 76, 147], since much of the research on generative AI takes place in industry contexts.

Despite these efforts, the path forward remains unclear [3, 10, 17, 23, 44, 71, 81, 94, 118]. A small but growing line of work has begun to empirically investigate computing researchers' practices around attempting to confront the negative societal impact of their work. This research has revealed that current efforts are not fully effective [10, 23, 81, 94], due in part to a general lack of incentives and support for self-reflection, including from academic institutions [3, 18, 44].

In addition, while several tools and processes have been proposed to help computing researchers better grapple with the potential negative societal impacts of their work [18, 33, 56, 99, 113, 138, 140], few of these are informed by empirical research. As HCI research has repeatedly demonstrated, such support is unlikely to be effective unless it is informed by an understanding of the on-theground challenges faced by its intended users [12, 40, 72, 77, 84, 115, 153, 156].

To this end, our paper explores the following research questions:

Prior work has pointed out that fostering more responsible research practices in computing will require significant changes across a range of institutions and sectors [18, 44, 68, 80, 88, 113, 116, 138]. While we endorse many of these more ambitious proposals, change can be costly and slow, progressing in fits and starts. Our goal is thus to develop an effective impact assessment tool that researchers can adopt immediately and that can be adapted to different organizational settings and arrangements as these broader changes in practices, policies, and norms ideally unfold.

- · RQ1 : What are industry computing researchers' current perceptions, practices, and challenges around assessing, articulating, and addressing the potential negative societal impact of their research?
- · RQ3 : How might we fulfill these design considerations to develop an effective societal impact assessment template for industry computing researchers?
- · RQ2 : What are some design considerations for designing and implementing an impact assessment template to support industry computing researchers in effectively assessing, articulating, and addressing potential negative societal impacts?

Fig. 1. An overview of the first two contributions or our work. On the left side, we present the empirical findings on current perceptions, practices, and practical challenges (Section 5). On the right side, we present the design considerations for the impact assessment template content and structure (in blue; see Section 6.1) and using the template within an organization (in green; see Section 6.2).

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01057v3-figure-8de67f0c-95ed-45c3-84fe-c03bc3b8ae11.jpg)

To investigate our research questions, we first conducted study sessions with 25 industry computing 1 researchers across different organizations and research areas. These researchers are formally employed by private companies, but also publish and present their research findings and innovations at peer-reviewed conferences or journals in the field of computing. Each study session was broken into two components: a semi-structured interview and a co-design activity. The semi-structured interview was aimed at understanding industry researchers' perceptions of the need to grapple with the potential negative societal impact of their research as well as their current practices for doing so, including the challenges they commonly encounter. The co-design activity with researchers was focused on soliciting their needs for effectively conducting impact assessment through the iterative co-design of a research-specific impact assessment, which we call the Societal Impact Assessment (SIA) template. To complement the feedback from industry computing researchers, who represent the intended users of the template, we also involved 16 researchers and practitioners with prior experience in developing and reviewing impact assessments or responsible computing practices more broadly (whom we refer to as 'impact assessment experts' throughout this paper) in the co-design process. These participants represent potential administrators or reviewers of the template and are included to ensure that the template fulfills its purpose of effective self reflection. Finally, to further understand the usability and usefulness of the SIA template, we recruited 15 interns from a range of industry research teams at a large U.S.-based technology company.

Our paper ultimately makes three main contributions:

- · An empirical understanding of industry computing researchers' perceptions, practices, and challenges around assessing, articulating, and addressing the potential negative societal impact of their research. Our findings reveal four factors that influence how industry computing researchers grapple with (or choose not to grapple with) the societal impact of

1 We adopt a broad definition of 'computing.' We cover all areas traditionally viewed as part of computer science, as well as social science and design research that examines the socio-technical aspects of technology along with research at the intersection of computer science and other fields, like biology. In adopting this broad definition, our aim is to be inclusive of the types of research that typically occur in industry research labs.

their research: the relationship between industry researchers and product teams (Section 5.1), organizational cultures that prioritize innovation and speed (Section 5.2), disagreements and misconceptions about societal impact (Section 5.3), and the lack or inadequacy of infrastructure to support impact assessment in industry settings (Section 5.4).

- · A version of the Societal Impact Assessment (SIA) template iteratively co-designed with industry computing researchers and impact assessment experts across different organizations and research areas (Section 7), along with results from a one-week user study that reveal the strengths and limitations of the SIA template (Section 8).
- · A set of 10 empirically-informed design considerations to facilitate the proper design, implementation, and adaptation of impact assessment templates in industry research settings and beyond (Section 6). These design considerations are formulated though the co-design with both industry computing researchers and the impact assessment experts. Figure 1 contains an overview of these design considerations, color-coded to indicate which pertain to template content and structure and which to the use of the template within an organization.

## 2 BACKGROUND

Recent calls for computing researchers to grapple with the societal impacts of their work have ushered in a sea-change in thinking about what it means to engage in responsible research in computing. Traditionally, the scope of responsible research practices was thought to include threats to research integrity (such as fraud or a failure to disclose limitations) [8, 127] and the dangers posed by the process of conducting research (such as the risks to the human subjects enlisted in a study or the mishandling of their data) [8, 18]. Recent calls expand the scope to include the risks of downstream harm to society more broadly due to the dissemination of research findings and their associated artifacts. Notably, while Institutional Review Boards address ethical issues that arise in the process of conducting research, they purposefully (and are often legally obligated to) avoid considering the dangers that research findings might pose to society overall [18, 49, 151]. As a result, while there is growing concern around downstream harms, there are no well-established processes for dealing with them. In what follows, we offer an overview of how concerns with societal impact have worked their way through the research community over the past few years and the mechanisms that have been proposed or adopted to address them.

A number of computer science conferences, especially in AI and machine learning, have responded to these calls with changes to their paper submission requirements or instructions and their peer review processes. In 2020, the Neural Information Processing Systems (NeurIPS) conference

In March 2018, Hecht et al. suggested that computer science reviewers should critically examine both the potential positive and negative impacts of research and called for the development of formal guidelines to aid this process [61]. In May 2021, the Partnership on AI issued recommendations for the responsible publication of AI research, encouraging researchers to consider the downstream consequences of research early in the research process and disclose relevant information about the risks of downstream harms [99]. The National Academies of Sciences, Engineering, and Medicine followed in May 2022 with a report that likewise called on computer science as a field to integrate concerns with the societal implications of research into the research and publication process [96]. In July 2022, the Ada Lovelace Institute, the Canadian Institute For Advanced Research (CIFAR), and the Partnership on AI released a joint report on "A Culture of Ethical AI" offering many similar recommendations targeted at the organizers of AI conferences, with the goal of encouraging greater reflection by researchers on the societal impacts of their work [30]. Later that year, in December 2022, the Ada Lovelace Institute also issued its own, much longer report, offering more detailed recommendations for Ethics Review Committees that could help anticipate and address the societal impacts of AI research [3].

introduced a requirement that authors 'include a section in their submissions discussing the broader impact of their work, including possible societal consequences - both positive and negative' [80]. A workshop took place at NeurIPS that same year on 'Navigating the Broader Impacts of AI Research' [9]. The subsequent year, in an effort to provide more guidance and flexibility to authors, this requirement was relaxed and a prompt about negative societal impact was incorporated into a broader 'paper checklist' designed to encourage responsible research practices [19]. Other major AI conferences, such as the International Conference on Machine Learning (ICML), the Conference on Computer Vision and Pattern Recognition (CVPR), and the Annual Meetings of the Association for Computational Linguistics (ACL) have introduced similar requirements [2, 35, 68], as has the International AAAI Conference on Web and Social Media (ICWSM) [1]. In 2024, organizers of the ACM Conference on Fairness, Transparency, and Accountability (FAccT) encouraged researchers to include an 'adverse impact statement' in their submissions, arguing that researchers working on ethical issues in computing need to consider the potential negative societal impact of their own work, too [98]. To aid researchers in fulfilling these new requirements or recommendations, conference organizers and researchers in related fields have written informal guides to support computing researchers in writing impact statements [2, 7, 60, 98].

Finally, funding bodies have also begun to incorporate these considerations into their review processes. In 2020, a cross-disciplinary group of researchers at Stanford developed an Ethics and Society Review Process to 'facilitate ethical and societal reflection as a requirement to access funding,' focusing on early assessment and mitigation of negative societal impacts in computing research [18, 32]. In January 2023, the National AI Research Resource Task Force - a group convened by the National Science Foundation and White House Office of Science and Technology Policy at the behest of the United States Congress - issued a report offering recommendations for how the United States government should go about setting up a public resource to support AI research [97, 140]. The report suggested that consideration of societal impacts should be a part of the process of reviewing proposals to use public resources.

## 3 RELATED WORK

## 3.1 Challenges in Considering Potential Negative Societal Impact

Despite the aforementioned growing calls for computing researchers to attend to the potential negative societal impacts of their work, it is still far from clear how this can be done effectively [3, 10, 23, 44, 81, 94]. For example, examining the broader impact statements written in NeurIPS 2020 papers, Ashurst et al. found that researchers have struggled to fulfill these requirements and to follow these suggestions, and that the quality of the resulting reflections and mitigations is spotty at best [10]. It appears that the guidelines provided by conference organizers and researchers are not sufficiently effective. Other work also suggested that it is unclear if researchers devoted sufficient effort to reflect on the societal impact of their research beyond simply completing these requirements [81, 94].

A small but growing line of empirical work has started to surface practical challenges faced by computing researchers when grappling with the potential negative societal impact of their work [3, 18, 44, 116]. To start with, there are limited tools and systemic guidelines to support assessing, articulating, and addressing potential negative societal impact, leaving researchers to do much of the reflection and possible mitigation on their own [18, 44]. A report from the Ada Lovelace Institute suggested that traditional research ethics processes, such as IRBs, are often not well-suited for computing research such as AI and ML [3]. Moreover, through a series of interviews, Do &amp; Pang et al. found that academic computing researchers often lack sufficient training, experience, as well as opportunities to collaborate with experts from diverse backgrounds in grappling with

unintended consequences of their research [44]. This work suggests that there is a lack of incentives for researchers to consider the negative societal impact of their work, as the process might be at odds with some researchers' goals to publish quickly [3, 44]. In contrast, Rubambiza et al. find that computing researchers may care quite a lot about the societal impact of their work, but are forced to compromise on their vision of societal impact or place less emphasis on societal impact in order to satisfy different audiences (e.g., collaborators, funders, peer reviewers, etc.) whose buy-in is required to execute the research project and publish the findings [116].

Although a small body of prior work has engaged industry computing researchers as part of their study participants in relevant tasks [40, 46, 83], it does not empirically examine the current practices and practical challenges around assessing negative societal impact faced by industry researchers . Understanding these aspects is critical and timely, considering the close relationship between industry research and product and the emerging discussions around the difficulties of assessing impact at later stages of the research-to-practice pipeline [144]. Although there is a growing body of CSCW and broader HCI research literature aimed at understanding the current practices, challenges, and needs of industry developer teams in building responsible technology [40, 41, 46, 62, 64, 82, 84, 106, 111, 145, 150, 156], there has been little focus on industry researchers . Therefore, our work extends the empirical findings from prior studies to uncover factors influencing how industry computing researchers grapple with (or choose not to grapple with) societal impact. These insights also lay the empirical groundwork for developing tools and processes that support computing researchers in industry and beyond.

## 3.2 Tools and Processes to Support Anticipating Potential Harms

CSCW, design, and HCI research more broadly have a rich history of creating tools and processes aimed at helping designers and developers anticipate potential failures, risks, and harms of the technology they are creating. This work often seeks to foster a greater appreciation of the issues that might arise across various technology deployments and real-world usage scenarios and thus to help developers better anticipate and address them [11, 13, 26, 34, 36, 50, 51, 53, 65, 69, 75, 95, 131, 158]. For example, value sensitive design is a theoretically-grounded design paradigm that can be applied to develop technologies that uphold the values of all potentially impacted stakeholders by identifying and addressing those values early in the technology development process [51-53]. Envisioning Cards , a practical tool derived from value sensitive design, can stimulate reflection and dialogue about the long-term impacts of new technology on different stakeholders [95]. Through a series of workshops with industry technology developers, Elsayed-Ali et al. iteratively designed and evaluated Responsible &amp; Inclusive Cards aimed at promoting critical reflection on the potential impacts of industry technology work [46]. Other common methods such as design fiction [21, 50, 85] and speculative design [11, 47, 75] are often used by CSCW and design researchers to imagine and create future scenarios and objects, in order to help researchers and practitioners explore, reflect, and anticipate the potential societal impact of technology. For example, Timelines is a design activity that assists technology developers in reflecting on the ethical concerns related to technical developments by creating fictional stories from news headlines [154]. More recently, HCI researchers have explored leveraging generative AI in scaffolding computing researchers and technology developers in anticipating the potential harms caused by their work [28, 102, 146]. With few exceptions (such as the work of Pang et al. [102], which targets researchers within the academy), these tools are primarily targeted at practitioners, rather than researchers. Our work draws inspiration from these works to better understand the unique challenges faces by industry computing researchers specifically and to develop empirically-informed recommendations and tools for this distinct community.

Recently, impact assessments have emerged as a particularly common approach to encouraging proactive anticipation and potential mitigation of harms in computing innovations [93]. These

emerging impact assessments for computing innovations often draw inspiration from impact assessments that have already been deployed to attempt to capture various aspects of responsible technology creation, such as fiscal impact [74], environmental impact [31, 92], data protection [37], and privacy [57]. For example, the Chief Information Officers Council of the U.S. and the Government of Canada have each developed a questionnaire-based algorithmic impact assessment as a resource to guide organizations and institutions in 'assess[ing] and mitigat[ing] the impacts associated with deploying an automated decision system' [33, 56]. In collaboration with the Ada Lovelace Institute, the UK National Health Service AI lab piloted an algorithmic impact assessment to help developers 'maximise the benefits and mitigate the harms of AI technologies in healthcare' [139]. Similarly, in the private sector, Microsoft recently began requiring product teams to complete a responsible AI impact assessment prior to the launch of any AI-based features [89, 90].

Existing impact assessments for computing innovations are designed for technology developers, often for deployed technologies that have clear use cases in mind [22, 93]. These tools are not immediately appropriate for assessing the impact of research publications or other research artifacts, which may be harder to predict since the potential use cases are more open-ended. Furthermore, there is limited empirical knowledge on designing usable and effective tools for computing researchers to assess societal impacts. [44]. In this study, we aim to bridge this gap by iteratively designing and developing an empirically-informed impact assessment template for industry computing researchers, drawing from existing tools and grounded in empirical evidences from our co-design sessions.

## 3.3 Understanding and Supporting Responsible Computing in Industry Settings

Within the CSCW and broader HCI communities, there has been a significant push to enhance our understanding of industry practitioners' current practices, challenges, and needs around responsible technology design, and to develop tools and guidelines to support them [40, 41, 62, 64, 84, 105, 106, 108, 111, 117, 123, 124, 145, 150, 152, 156, 157]. This research has underscored how organizational dynamics and culture influence responsible and ethical computing practices. For example, a culture that predominantly values moving fast and scaling up can obstruct the use of tools designed to encourage thoughtful reflection [41, 82, 84, 148]. Moreover, power dynamics in the workplace can significantly affect responsible innovations and technology developments. For example, recent works by Rakova et al. [111] and Wong et al. [152] each revealed how 'ethics workers' within companies frequently encountered pushback from leadership when advocating for more responsible technologies. Researchers have also developed tools and processes to support workers' reflexive and anticipation work in industry practice, including the Responsible &amp; Inclusive Cards mentioned above [13, 46, 146]. On the practical side, works have further emphasized the importance of integrating support for responsible development into teams' existing tools and workflows [40, 62, 78, 84, 145, 146, 157].

However, the vast majority of this prior research has focused on teams creating and deploying products as opposed to industry researchers producing academic publications or other research artifacts like data or code. In our study, we specifically investigate how organizational dynamics and company culture might influence industry researchers' capacity and willingness to assess, articulate, and address the potential negative societal impacts of their work. Through interviews and co-design with industry researchers, we develop an impact assessment template tailored specifically for industry research rather than product design and development.

## 4 METHODS

To investigate our research questions, we conducted a series of interviews with 25 industry computing researchers, an iterative co-design process with the same 25 industry computing researchers and 16 impact assessment experts, and a user study with 15 interns working in industry research. We

Iterative co-design with both researchers and experts

RQ3: Develop of a usable and useful impact assessment template (Section 7, 8)

Fig. 2. Overview of our methods and how each study component maps to our research questions and results.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01057v3-figure-311c2821-788d-4619-8533-32c3c37853aa.jpg)

illustrate our method in Figure 2. Each activity is described below, and full protocols are provided in the supplementary material. 2

All studies were conducted over a video conferencing platform between June and August 2023. Sessions were recorded except in cases in which participants opted out and were transcribed using transcription software. Participant quotes were anonymized at the individual, team, and organization levels. Participation in the studies was voluntary and participants were told that they were free to skip any questions they were uncomfortable answering and to leave the study at any time for any reason. Participants in each study were compensated with a $50 gift card. All studies were approved by our institution's IRB.

## 4.1 Creating the Initial Prototype of the Impact Assessment Template

To create the initial prototype of the societal impact assessment template that we used as a basis for iteration in the co-design sessions, we began by synthesizing insights from prior academic literature and other reports on assessing the impact of research in areas such as AI, HCI, and medical research [3, 5, 88, 98, 138]. From these, we formulated a list of initial design considerations. We also drew inspiration from publicly available impact assessment templates such as the responsible AI impact assessment template from Microsoft [89, 90], the algorithmic impact assessments from the UK NHS AI lab [138], the CIO Council of the U.S. [33], and the Government of Canada [56], and the privacy impact assessments from the Government of Canada [57] and introduced by the EU with the General Data Protection Regulation (GDPR) [37]. When designing the template scaffolding content, we also drew inspiration from artifacts in value sensitive design [6, 51, 95, 120, 154, 158], tools and guidelines for conducting responsible computing research [2, 19, 65, 68, 127], as well as survey papers discussing the taxonomy of socio-technical harms [119, 147]. The resulting template, included in the supplemental material, was intended to guide industry researchers through a step-by-step process to consider the intended uses and applications of their work, identify potential stakeholders, anticipate positive societal impact, anticipate potential negative societal impact, and formulate possible mitigations.

2 The supplementary material will be available with the full published version of this paper.

| ID   | Years   | Research Area                       | ID   | Years   | Research Area                |
|------|---------|-------------------------------------|------|---------|------------------------------|
| P01  | 0-3     | Augmented Reality (AR)              | P14  | 3-10    | VR, AR                       |
| P02  | 3-10    | HCI, Responsible AI (RAI)           | P15  | 0-3     | Computer Vision              |
| P03  | 0-3     | Reinforcement Learning              | P16  | 3-10    | VR, AR                       |
| P04  | 3-10    | Accessibility                       | P17  | 10+     | Reinforcement Learning       |
| P05  | 10+     | Hardware security                   | P18  | 3-10    | HCI, Responsible AI          |
| P06  | 0-3     | AI, Philosophy                      | P19  | 3-10    | Sociology, CSCW              |
| P07  | 0-3     | Virtual Reality (VR), Accessibility | P20  | 0-3     | NLP, RAI                     |
| P08  | 10+     | AI, Accessibility                   | P21  | 3-10    | AI, HCI, Healthcare          |
| P09  | 3-10    | Computational Biology               | P22  | 3-10    | NLP, Computer Vision         |
| P10  | 3-10    | Psychology, CSCW                    | P23  | 0-3     | HCI, NLP                     |
| P11  | 0-3     | Computer Vision, Accessibility      | P24  | 0-3     | Computer Graphics            |
| P12  | 3-10    | Natural Language Processing (NLP)   | P25  | 10+     | Computational Social Science |
| P13  | 3-10    | NLP, Education                      |      |         |                              |

Table 1. Details about the industry researchers who participated in our interview and co-design study. Overall, the researchers work at 12 companies: 10 large technology companies with 25,000 or more employees and two smaller technology companies with 5,000-24,000 employees.

19 researchers are from the U.S., 4 from Europe, and 1 from Australia.

## 4.2 Interview and Co-Design Sessions with Industry Researchers

After creating the initial prototype of the template, we began conducting sessions with 25 industry researchers across organizations and research areas. Each 60-minute session was broken into two components: a semi-structured interview and a co-design activity. Ideally, we would have conducted all interviews first and then invited back the same group of participants for co-design sessions. However, we decided to conduct interviews and co-design sessions together due to the challenges of participants declining to return for a separate co-design session (as documented by prior work [41, 77, 111]) and limited time for conducting the study. During the early stages of the project, we placed more emphasis on the semi-structured interviews. Later, we shifted more of the session time to the critique and co-design of the template itself. Between sessions, we updated the impact assessment template based on feedback from participants, as shown in Figure 3 and detailed in the appendix. This iterative co-design process drew parallels with methodologies deployed in prior work [e.g. 39, 46, 64, 84, 127].

The co-design sessions with researchers were intended to better understand the opportunities for supporting researchers in conducting impact assessment. Within the session, participants were asked to bring a publicly available (e.g., previously published or available on a public platform like arXiv) research paper they had authored as a case study to keep in mind when walking through the template. We first gave participants a high-level overview of the impact assessment template. We then went through the template, asking participants to consider how they would answer the questions with respect to their projects and provide item-level feedback, for instance, identifying

The semi-structured interview was designed to explore industry researchers' perceptions, current practices, and challenges around assessing potential negative societal impact of their work. Weasked participants about their research, their research team, and their relationships with product teams. We then asked them to define the societal impact of industry computing research. Adopting a directed storytelling interview approach similar to prior work [38, 48], we guided participants to reflect on their current practices around whether, when, and how they (try to) assess and mitigate the potential negative societal impact of their work. Finally, we probed deeper into the practical challenges they had encountered.

## Iteratively co-design with both industry computing researchers and impact assessment experts for the Societal Impact Assessment (SIA) Template

Fig. 3. Overview of the iterative co-design process. Version 7 is the SIA Template described in Section 7 and used in our user study. Version 1 and Version 7 are included in the supplementary materials.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01057v3-figure-44df2a72-b7b3-44ec-9bbc-adcc1b17a482.jpg)

sections of the template that might be hard for their team to use. For each component of the template, we also encouraged participants to share what they thought might be helpful for them to better complete the task. We asked participants to share at which research stages would they want to use the template in their own research, as well as how they envision the outcomes of the impact assessment being presented to other researchers, the product teams in their organizations, and the public. Note that we made a conscious decision to focus on designing the content of the impact assessment, deliberately choosing not to concern ourselves with how this would be implemented in an interactive tool. This decision was primarily based on the understanding that the format of interaction would likely need to be tailored specifically to the organization and its context of use. We address this topic as future work in Section 9.2.

To recruit industry researcher participants, we adopted a purposive sampling approach [29]. Our aim was to recruit computing researchers who (1) are currently employed in industry and (2) publish and present their research findings and innovations in peer-reviewed venues such as academic conferences or journals. We allowed two exceptions to the first criterion, allowing two participants (P21 and P25) who currently work in academia but spent extensive periods of time in industry research labs before becoming professors. We believe their experiences could contribute insight into the challenges specific to industry settings. Participants were recruited through direct contacts at large technology companies, email lists within industry research labs, and snowball sampling. The 25 researchers who participated spanned 12 technology companies, work in diverse areas of computing research, and possess various levels of experience. Table 1 provides an overview of the participants' research area and relevant experience.

## 4.3 Co-Design Sessions with Impact Assessment Experts

To complement the insights from industry computing researchers - the target users of impact assessment templates - we also conducted a series of 30-minute co-design sessions with impact assessment experts : researchers and practitioners with prior experience in developing and reviewing impact assessments or research or policy-making experience around responsible computing and technology more broadly (See Table 2). The goal of these sessions was to ensure that the impact assessment template fulfilled its purpose in uncovering potential negative societal impacts. We began conducting these sessions after making several iterations on the template and interleaved them with the co-design sessions with industry researchers, as illustrated in Figure 3. In preparation for each expert co-design session, we shared a current version of the template with the participant and gave

Table 2. Details about the impact assessment experts who participated in our co-design study. All have experience developing and reviewing impact assessments in their own organization (denoted 'Internal impact assessments'), working with impact assessments in other contexts ('Other impact assessments'), or in research or policy-making on responsible computing more broadly ('Other responsible computing').

| ID   | Organization   | Research/Practice Domain            | Relevant Experience         |
|------|----------------|-------------------------------------|-----------------------------|
| E01  | Industry       | Responsible AI (RAI); Healthcare    | Internal impact assessments |
| E02  | Civil society  | Public policy; RAI                  | Internal impact assessments |
| E03  | Academy        | Science and Technology Studies; RAI | Other impact assessments    |
| E04  | Civil society  | AI safety and governance            | Internal impact assessments |
| E05  | Academy        | Philosophy of science               | Other responsible computing |
| E06  | Industry       | Public policy; RAI                  | Internal impact assessments |
| E07  | Industry       | RAI; Healthcare                     | Internal impact assessments |
| E08  | Civil society  | Privacy; RAI                        | Other responsible computing |
| E09  | Civil society  | Public Policy; RAI                  | Other impact assessments    |
| E10  | Academy        | Philosophy                          | Other responsible computing |
| E11  | Industry       | Public policy; RAI                  | Internal impact assessments |
| E12  | Industry       | Public policy; Philosophy           | Internal impact assessments |
| E13  | Academy        | Bioethics; Political science        | Internal impact assessments |
| E14  | Academy        | Philosophy; RAI                     | Other responsible computing |
| E15  | Industry       | AI safety and governance            | Internal impact assessments |
| E16  | Civil society  | AI safety and governance            | Other impact assessments    |

them the opportunity to optionally review and annotate it with any preliminary feedback. During the session, we first solicited feedback on the template's structure and goals. We then prompted the participant to critique each section and provide fine-grained feedback. Finally, we asked the participant to envision how the template might be operationalized in their current organization. We incorporated participants' feedback when producing the next iteration of the template.

To recruit participants, we again adopted a purposive sampling approach [29]. In particular, the authors leveraged their connections with impact assessment experts across academia, industry, and civil society. Among the 21 impact assessment experts we emailed, 16 expressed interest. Of these, 14 participated in a full 30-minute session, while two opted to review the template and annotate it with their feedback without joining a live session. Overall, five of the experts were from civil society organizations, five from academia, and six were employed in industry. Table 2 offers more background information on these experts.

## 4.4 User Study of the SIA template

By the time we produced V7 of the template, we had reached data saturation with the co-design activities with researchers and impact assessment experts. Similar to previous co-design studies [39, 64, 84], we stopped iterating on the template at this point and turned to a user study . While we do not view V7 as a 'perfect' or 'final' version of the template, the user study was intended to give us more insight into the template's usability and usefulness in practice.

Participants in the user study were given one week to complete V7 of the template with respect to an ongoing research project. After completing the template, they were given an exit survey and asked to sign up for an exit interview. The exit survey first asked questions about their usage of the template, such as the amount of time they spent filling it out and how many times they revisited it during the one-week study period. It then asked about the template's overall usability, for example,

asking participants to provide Likert scale ratings for statements like 'the template's organization is logical and intuitive to navigate.' Next, participants were asked whether the template helped them identify potential stakeholders, positive societal impacts, and negative societal impacts. Finally, they were asked for Likert scale ratings of the template's overall usefulness. In the exit interview, which lasted 15-40 min, participants were asked to share any additional thoughts on their experience filling out the template and probed more deeply about their survey responses.

To recruit the participants, we shared a recruitment message in email lists and message boards within the technology company. Initially, 17 research interns signed up for the study, of which 15 filled out the template, completed the exit survey, and participated in an exit interview. Table 3 provides more information about the participants in the user study.

For the user study, we recruited current research interns (primarily PhD students) from a large U.S.-based technology company. Ideally we would have recruited full-time industry researchers across organizations satisfying the same criteria as the participants in the interview and co-design sessions. However, some researchers were reluctant to join because of the potential risk of getting scooped by sharing their ongoing, unpublished research projects. Additionally, full-time researchers were hesitant to make the required time commitment over the course of a week. We believe that recruiting research interns still allowed us to gain valuable insights on the usability and usefulness of the template. We discuss these limitations further in Section 9.3.

## 4.5 Data Analysis

In total, the interviews, co-design sessions, and user study exit interviews yielded approximately 29 hours of recorded audio, which was automatically transcribed, as well as detailed notes taken by the interviewer for the 11 sessions in which the participants opted out of recording. We additionally had the two annotated templates from the impact assessment experts who chose to provide asynchronous feedback and responses to the 15 exit surveys from the user study. To analyze the transcripts from our interviews and co-design sessions, we adopted the reflexive thematic analysis approach described by Braun [25], similar to prior work examining data from interview and co-design activities [46, 64, 84]. All of the authors convened frequently throughout the interviews and co-design sessions to conduct interpretation sessions and discuss the insights. The first author conducted open coding of the transcripts after each interpretation session. Some open codes that emerged from the interviews and co-design sessions with industry computing researchers include: "Participants deflect the responsibility of assessing and addressing the impact of their research to product teams" and "Participants tend to oscillate between identifying stakeholders and assessing the potential impact when co-designing the template." Following the guidelines provided by Braun and Clarke [24], during this coding process, authors continuously discussed discrepancies in interpretation, and the first author iteratively refined the codes based on these discussions [25, 86]. For the user study, we analyzed the exit survey data and computed descriptive statistics. We also performed open coding of the open-text responses to the exit survey and the transcripts of the exit interviews, aiming to synthesize qualitative insights that complement the quantitative evaluation provided by participants. Notice that we chose not to perform and report a content-wise analysis of the template like prior work [cf. 81, 94] because doing so would reveal details about participants' ongoing, unpublished research projects.

## 5 CURRENT PERCEPTIONS, PRACTICES, AND PRACTICAL CHALLENGES

In this section, we identify and explore four factors that influence how industry researchers grapple with (or choose not to grapple with) the potential negative societal impact of their research, summarized in the left part of Figure 1. Some of our results validate findings from prior empirical work [3, 18, 44, 81, 94]; we call out these connections to prior work when they arise. contributes the necessary empirical evidence to be confident of the generality of these observations. Our findings

also extend the existing literature by highlighting structures and dynamics that are unique to computing research in industry.

## 5.1 Relationships between Industry Researchers and Product Teams

Overall, we found that the complex relationships between industry researchers and product teams heavily shape participants' perceptions and current practices around assessing, articulating, and addressing the negative societal impact of their work. In particular, many participants believe that their research has impact through (and only through) its incorporation into products and services. This belief motivates them to attempt to grapple with its potential negative societal impact. However, participants often encountered practical challenges in doing so due to a lack of both transparency and agency around the specifics of when and how their research might be used by product teams. These practical challenges eventually led to participants deflecting the responsibility of grappling with the potential negative societal impact of their own research to product teams and other decision makers within their organizations. Below, we describe these results in more detail.

To this end, we found that participants are particularly motivated to reflect on and assess the potential negative societal impact of their research when they know it will be incorporated into products and services . For example, P17 shared with us that their team was 'just doing [their] own thing' when their work 'was largely theoretical,' but 'started to look into what could go wrong if our research is being widely adopted' when the ML systems they built were incorporated into products. Similarly, P11 suggested that in their organization, conversations about the negative impact of their research often 'happened when a research project was being integrated into a product feature...otherwise we somehow feel like this conversation can wait.' They further acknowledged that prioritizing only research that is being incorporated into products is not a best practice, but their team needed to 'sort out priorities' given limited time and resources.

Many participants expressed a belief that societal impact arises when (and only when) industry research is incorporated into products and services . P04, P07, P11, P15, P18, and P23 all explicitly mentioned that their own research would have only limited impact if it were not integrated into their companies' products and services. P11 told us that their 'research impact is defined by the product impact,' and that industry research can be more impactful than work done in academia due to the 'potential to improve the quality of life for many people through [the] company's products.' Similarly, P13, P21, and P25 - who all had spent multiple years as tenure-track professors in academia in addition to researchers in industry labs - emphasized impact through products and services as a differentiating factor between academic and industry research. As P21 put it: 'In school, you mostly hear people talk about impact as coming up with creative new ideas, getting accepted to conferences, getting grants, [and] having lots of citations. But in a company, my colleagues and I mostly get really excited when your ideas can actually be in the hands of users through products...that's how the impact is being defined there.'

In contrast, several participants with limited contact with product teams rarely assessed the societal impact of their work and believed it was unnecessary . For example, P24, who works on improving the computational efficiency of graphics rendering algorithms and 3D modeling, did not see a need to think about the impact of their work on society 'because the research outcomes are tools for other researchers...not used directly by our customers. ' In addition, when the incorporation of research into products and services effectively motivated participants to consider potential negative societal impact, we found that participants frequently limited their focus to existing customers and intended stakeholders , neglecting indirect stakeholders or other segments of society who might be impacted by their research.

Interestingly, many participants shared that industry researchers do not necessarily know specifically when, where, and how their research will be incorporated into the products,

leading to practical challenges for assessing and addressing negative societal impact . For example, P04, who works closely with product teams on improving the accessibility of mobile apps, told us there were often 'non-disclosure agreements among teams' that essentially left them out on knowing 'whether or not [their] research will eventually be adopted by product teams.' Because of this, P04 suggested that it wouldn't be feasible for them to assess the potential societal impact of their research. Even those researchers who were certain their research would be used by a product team often reported a lack of agency regarding the way in which it would be incorporated. For example, P21 said: 'Once [the product teams] take [the research results], they can really do whatever...and in marketing, they can spin it however they want...I mean you would hope that they would use it responsibly, but really you basically lose control.' Similarly, P23, who works closely with a product team implementing a recommendation system, told us that they were 'pretty much out of touch with [their] own work' once the product team started to incorporate it. This participant suggested that they wish they could 'sit in on some early meetings discussing how our research is being used' so that it would be feasible for them to 'flag anything that might cause downstream harms.'

To this end, we found that many participants tend to deflect the responsibility of assessing and addressing the potential negative societal impact of their own research to product teams and other decision makers within their organizations . For example, P23 suggested that 'it would be unfair to expect us to mitigate harms when the product people are the ones who misuse our research.' Similarly, P11, who works on machine learning models, believed that product teams should be accountable for assessing any negative impact of their research when it is incorporated into products since they are the ones 'deciding what to do with the models we provide.' P04, P05, P16, P18, and P21 all explicitly stated their belief that the primary responsibility for assessing and addressing the potential negative impacts of industry research should rest with product teams, again citing researchers' lack of transparency and agency.

## 5.2 Organizational Dynamics and Cultures that Prioritize Innovation and Speed

A rich line of prior work in the CSCW and broader HCI community has underscored how organizational dynamics and culture influence industry product teams' responsible computing practices [3941, 64, 82, 84, 106, 111, 150]. In line with this, our study reveals how industry researchers are shaped by organizational dynamics and culture - particularly the prioritization of innovation and speed when assessing, articulating, and addressing the potential negative societal impacts of their work.

Among all participants, those in junior roles (P01, P15, P23) expressed particularly strong reluctance to assess and discuss the potential negative societal impact of their work . For instance, P15 told us that they felt pressure to 'establish myself and show that my research is valuable for the company by improving the product.' Although they considered the potential misuse of a face-swapping computer vision algorithm they were working on, especially by users under 18, this participant ultimately did not voice these concerns, stating 'I don't want to be a buzz killer when my manager is very excited about something.' Similarly, P23 shared with us that 'I'm still trying

To start with, many participants shared that they were reluctant or even unwilling to assess and discuss the potential negative societal impact of their work because of an organizational culture of emphasizing the positive side of innovation . For example, P02 reflected that the 'predominant narrative focuses on how many great things the company will contribute to the world.' Because of this, they believed that industry researchers often inadvertently viewed their work through 'rose-tinted glasses' and felt 'peer pressure to always signal how [their] own research agenda is somewhat aligned with [the] company's mission.' In some cases, researchers feared retribution for speaking out about negative impact. For instance, P16 commented on how previous layoffs of industry researchers working on AI ethics had a chilling affect on them and their colleagues, dissuading them from sharing potential negative societal impacts with their managers.

to figure out my role in the team. At least at this stage, what I talk most about to people is how my research can help them instead of the harms my work can do.' Although this participant recognized the importance of assessing potential negative societal impacts, they felt it might conflict with their 'career development as the most recent employee of the team.'

As a result of this culture, participants reported that their efforts to assess and address negative societal impact were often ignored or pushed back on due to mismatched incentives . Specifically, many participants, including P04, P07, P08, P10-P13, P19 and P21, shared stories about how their colleagues from other teams (both in research and product development) often overlooked advice from researchers on assessing and mitigating negative societal impacts. P10, for instance, recounted a time when a team integrated facial recognition algorithms into tools developed for internal workers, disregarding the internal guidelines suggested by their research group. The participant described that the development team is 'encouraged to move forward without any ethical guidance...they're not incentivized in any way, shape, or form to be ethical. They are just trying to create a prototype as fast as possible, and because they report to a general manager who is not in research, they can basically ignore us.' Similarly, P08 shared that by the time they published a paper and highlighted the limitations of their AI models, which were designed for an alt-text generation pipeline, the product team working with them had already incorporated their research into new features to achieve the team's quarterly goal. Reflecting on this, P8 stated ' we spend time writing a paper and wait for peer review, but [product teams] won't wait for our publications...they generally don't read our paper or guidelines. We need ways to affect their day-to-day practice, and the way that you affect their day-to-day practice is through these various compliance programs, or even their KPIs.'

Furthermore, the 'move fast and ship products' culture [82] in technology companies also introduces practical challenges for researchers to appropriately assess and address potential negative impact . For instance, many participants noted that some computing researchers in their companies tend to 'move fast and do groundbreaking innovation' (P16), while some product teams aim to 'launch new product features as fast as possible' (P20). P13 recounted an instance where their research team developed a computer vision algorithm that 'took a very binary gender view of the world' and they 'didn't feel comfortable releasing it until [they] could do something about that.' However, they and their colleagues felt 'internal pressure from the product manager to get things out quickly.' This participant concluded that 'the pressure to get things out quickly is often at odds with carefully assessing the unintended consequences of it.'

## 5.3 Disagreements and Misconceptions around Societal Impact

A number of our study participants believed that assessing the negative societal impact of their work was unnecessary due to their specific areas of research . Echoing prior work [3, 44, 81], we heard this from participants whose research contributions were in theoretical areas of computer science (e.g., machine learning theory), as well as those whose primary research goals were aimed at doing good, for example, mitigating the potential harms of other computing research. Other participants felt that assessing the potential negative societal impacts of their research was unnecessary due to the highly regulated nature of their research application areas , such as medicine, finance, and healthcare. P09, for instance, noted 'the only [potential negative societal impact] I could think about is people making bio-weapons...the problem is that building bio-weapons is hard and requires a lot of resources. It's a bit unnecessary for me and people to worry about the malicious usage because the government will definitely notice if someone is making my research into bio-weapons.' P14, who works on VR/AR research in a Fintech company, expressed that there is no need for researchers to complete impact assessments since the stringent regulations in finance would enforce that care would be taken downstream in deployment. In particular, this participant mentioned that since their company has 'security engineering teams to prevent the malicious actors

and law professionals to handle any legal disputes or litigation,' their responsibility as a research scientist is simply to innovate. We note that both P09 and P14 called out only examples of malicious misuse when contemplating potential negative societal impacts. However, during the co-design process, both P09 and P14 identified potential accidental misuses of their research and agreed that there was still value in assessing and articulating these potential misuses of their work.

All but two participants in our study conflated ensuring research integrity and attending to the welfare of human subjects with addressing the downstream societal impact of research . While prior work has surfaced similar misconceptions [8, 44], we were particularly surprised by how prevalent such misconceptions were among our participants, with participants of varying seniority levels, working in areas including sociology, human-computer interaction, and responsible AI, all hitting this pitfall. For example, when asked to describe an instance when their team attempted to assess the potential negative societal impact of their research, P10 recounted how their team had protected the privacy of participants outside the U.S. while conducting research. After we clarified the question and the participant realized their misunderstanding, they began to reflect on how their team had also been conflating these concepts during their research meetings: 'I guess we have been kind of mixing these two concepts together in our project meetings...but now that I think about it carefully, protecting participants' privacy is actually different from preventing downstream harms.' Consequently, P10 believed that it would be beneficial to provide researchers with learning material and training opportunities around disentangling these concepts and appropriately addressing ethical issues throughout the research lifecycle.

## 5.4 Lack of Sufficient Infrastructure

Prior research has already highlighted the severe absence of efficient and effective infrastructure to support computing researchers in assessing and addressing potential negative impacts [3, 10, 18, 23, 44, 81, 94]. Many industry researchers in our study noted that, while there are processes within their organizations for brainstorming the positive impact of their research, there are often no such processes for assessing and addressing the potential negative societal impact . For example, P01 told us that their team usually schedules multiple 'research brainstorming sessions, ' often resulting in formal documentation around their projects' potential positive impacts, managed by the team lead. In stark contrast, when assessing their research's potential negative impact, this participant's team relies on 'spontaneous conversations about what might go wrong in one meeting...maybe some follow-up if someone happens to bring this up again in a later meeting.' P21 said that while discussions about negative impact occur, there is no formal process, so discussions are often being relegated to 'side chatter. '

In contrast, some participants mentioned that in their companies, formal processes are beginning to emerge to involve industry researchers in assessing and addressing the potential negative societal impact of their research. These processes often come up in the context of discussions around legal and compliance considerations that occur when the company is releasing research artifacts through open source or incorporating research into products. Many participants pointed out a lack of support around cross-functional communication and collaboration between researchers and legal or compliance teams . For example, P22 shared the communication gaps they experienced when discussing the release of an image captioning model with their in-house legal teams: 'I found it useful when legal folks came to talk to us, but the discussion between us was not efficient...I was very confused about all the terminologies they threw at us...[it] felt like we needed a translator despite both speaking English.' Similarly, P14 described feeling 'unprepared and confused' in discussions about the potential misuse of augmented reality services with the company's compliance team, adding 'we all agreed that understanding the platform policies was important to consider the different downstream use cases of our work...but none of us on the research team was aware of those policies

before that chat...I wish we talked to them earlier on.' In light of this, this participant emphasized the need for processes that foster better integrated communication and collaboration between research and compliance teams, particularly for assessing and addressing potential negative impacts.

Participants also mentioned the lack of support and effective infrastructure from publication venues like conferences that require a discussion of societal impact or an explicit broader impact statement. In line with the hypothesis from prior work that authors often write impact statements without sufficient reflexivity [81, 94], we repeatedly heard from many participants (P01, P02, P05P09, P12-P16, P20, P21, P23, P25) that they only start considering the broader impact statement close to the paper deadline . Participants like P13 and P21 expressed a desire for conferences or their own organizations to provide more concrete guidance on assessing and reporting the societal impact of their work. Corroborating findings from prior work [44], many participants in our study mentioned that, without sufficient scaffolding, the conference requirements do not motivate them to conduct a careful impact assessment and can feel burdensome . For example, P03 told us that a rigid broader impact statement requirement did not fundamentally motivate them to assess the impact of their research, as 'anything that dictates how to structure your paper or mandates content...feels like it's taking away some of your agency.' P17 suggested that these conference requirements are treated as mere checklists by industry researchers without 'a norm and culture among the community to treat evaluating the downstream impact of our work as the first class problem. ' In addition, P19 noted that adding a conference requirement without providing sufficient training and support inadvertently adds burden to researchers.

## 6 DESIGN CONSIDERATIONS FOR SOCIETAL IMPACT ASSESSMENT

In this section, we synthesize a list of design considerations (DCs) for the design and implementation of societal impact assessments for computing researchers, as overviewed in Figure 1. We support each DC with empirical evidence from the co-design process with 25 industry computing researchers (whom we refer to as 'researchers' for brevity throughout this section) and 16 impact assessment experts (whom we refer to simply as 'experts'). Additionally, we briefly discuss how each DC might be realized in practice, as suggested by both researchers and experts. We present these DCs in two subsections: those that pertain to template content and structure (Section 6.1), and those that pertain to use within an organization (Section 6.2). In Section 7, we demonstrate how to operationalize these DCs within an actual impact assessment template.

## 6.1 Design Considerations for Template Content and Structure

DC-1. Provide scaffolding for researchers to consider different dimensions of societal impact. This scaffolding could take the form of brainstorming examples, checklists, and guided questions for each dimension.

This sentiment was echoed by all experts. For instance, E07 shared that they observed that researchers often 'stay in their comfort zone' and 'keep bringing up commonplaces' whenconsidering potential downstream harms, rather than 'critically examin[ing] how research could unfold in different ways in the real world.' E03 believed that in addition to considering stakeholders and types of societal impact, the template should also encourage researchers to 'specifically reflect on the longterm and indirect effects of their research on society.' Multiple experts also emphasized the need to

In response to the lack of infrastructure for assessing negative societal impact (Section 5.4), researchers unanimously expressed a desire for processes to help them grapple with the diversity and complexity of potential impacts. These include the various types of stakeholders possibly impacted, the ways in which the research is used, and the different types of societal impact. P13 mentioned constantly being 'afraid of missing some important elements that need consideration.' P19 suggested including scaffolds such as checklists to ensure that researchers address all important considerations.

include a list of concrete examples under each dimension, as they are 'really the best way to get someone to think about a risk or aspect that they hadn't previously considered' (E16).

DC-2. Support heterogeneous, non-linear thinking processes. Specifically, allow researchers to complete the sections of the impact assessment template non-sequentially, in the order they prefer.

While most researchers found the identification of potential stakeholders to be a natural starting point for considering potential impacts, others expressed different preferences. P09 suggested they would prefer to 'start with thinking about the type of impact,' while P23 found it 'more natural to first figure out how people in general might misuse my work, then dive into more details on specific groups of stakeholders.' E07 suggested including 'a questionnaire at the beginning of the template to solicit [a] preference, then adapt the [template] structure based on the response.'

While many existing impact assessments in other domains instruct practitioners to proceed in a linear manner [33, 37, 56], we observed that researchers frequently jumped between different sections of the template. For instance, P17 oscillated between identifying stakeholders and thinking through how these stakeholders might misuse or be impacted by their research, commenting: 'I wonder if there is a way to design the template so that it's easier to simultaneously consider the stakeholders and the negative impact.' P20 went back and forth between considering the limitations of the research and brainstorming the potential negative impact. E06 emphasized that researchers need to consider the 'interactions and trade-offs between the categories of impact,' suggesting that researchers should 'be reminded to reconsider stakeholders while responding to questions in subsequent sections, to prevent overlooking marginalized communities that were not recognized earlier.'

## DC-3. Support conducting the impact assessment iteratively over time. Design the template to be revisited and updated throughout the research process, accommodating changes and new insights as the project evolves.

Many researchers brought up that impact assessment 'shouldn't be a one-shot process' (P16), and that researchers should instead 'fill out this template when starting the project and revisit it from time to time' (P21). Similarly, experts advocated for treating the template as a 'dynamic document, revisited as research progresses' (E09). E05 highlighted the necessity of continuous impact assessment, noting the pitfalls they observed: 'researchers sometimes try to get all questions down in one pass...which shouldn't be the case since they are supposed to iterate on their answers throughout the process...new problems will always emerge as the research evolves.' Both researchers and experts suggested that the template could include version control or progress-tracking features so that researchers could 'have iterated documentation available to reflect on how their thinking [about the societal impact] evolve[d] throughout the research process' (E01).

DC-4. Highlight important considerations for specific research sub-fields and application domains. Specifically, augment the template with domain-specific examples, terminology, and considerations so that the template is more relevant and effective for researchers within particular fields.

Throughout the co-design study, all researchers repeatedly told us that they wished the template would highlight important considerations for their specific research sub-fields and application domains. For example, many researchers working in AI suggested that the template should support AI researchers to 'consider different stages of [the] AI lifecycle, such as problem formulation' (P06). P16 believed it was important to include 'physical health risks and psychological effects unique to our VR research.' Several experts echoed these concerns. For instance, both E01 and E05 used computing research in healthcare as an example, suggesting the template should remind researchers working in a healthcare context to consider 'what type of hospitals are you working with to get your data? Is it public or private?' (E01) and 'differences between private health insurance companies and public health

systems' (E05). Other experts suggested that the environmental impact of model training should be highlighted in the societal impact template for researchers working on large language models.

In practice, experts suggested that template designers could start with a general version of the template, then adapt it to different research domains. One way to achieve this is by including additional modules pertinent to specific research areas. Echoing the recommendations made in prior work examining other forms of impact assessments [18, 93], experts in our study highlighted the need to 'incorporate diverse perspectives from different domains' (E03) and 'assemble a group of researchers in a specific area to tailor the impact assessment template' (E06) to be suitable for a specific domain.

## DC-5. Highlight the value of completing the template. This could include surfacing more positive impacts of the research, improving the rigor of the research, and surfacing future work.

Experts concurred that the template should emphasize its role in enhancing research quality to mitigate the perception that completing an impact assessment is a punitive and burdensome process. E09 recounted that senior leadership in their organization clarified that 'the purpose of impact assessment tools is not punitive but to help improve research quality...and maximize positive impact while minimizing negative ones.' Both E02 and E13 emphasized that aligning the template with researchers' interests is crucial for gaining researcher buy-in.

The challenge of incentivizing researchers to complete impact assessments is well-documented [3, 10, 18, 109]; see Sections 3.1 and 5.4. During the co-design process, almost all researchers indicated they would be more likely to complete an impact assessment if it had a clear benefit for their research. As a comparison, P25 shared that they are 'a big fan of pre-registration in fields like experimental psychology and medicine...since I know [completing pre-registration] would make my work better.' Many researchers found that they appreciated the activities that helped them identify the positive societal impact and limitations of their work, as they are 'something that can directly benefit paper writing' (P22). Some, including P02, P04, P09, P13, and P19, also recognized the benefit of identifying negative impacts early to improve their problem formulation and research design; P19 suggested the template designers should emphasize this benefit.

## DC-6. Suggest appropriate and actionable strategies to address potential negative impact. The template should also assist researchers with the prioritization of impacts.

Resonating with findings from prior work [44], both researchers and experts emphasized that the template should extend beyond simply identifying negative impacts to provide actionable steps for developing a harms mitigation plan. For example, P02 told us that they would 'not feel comfortable to finish an impact assessment without knowing concrete next steps for mitigating harms.' E04 told us that when reviewing impact assessments in their organization, they observed that researchers often acknowledge harms without proposing solutions. To this end, E04 believed that it is crucial for the template to 'prompt people in that general direction to think about appropriate mitigation, instead of just pointing out harms.' Meanwhile, many experts acknowledged the practical difficulty of fully anticipating the likelihood and magnitude of harm. Nonetheless, they believe the template should at least help researchers begin assessing these factors to 'identify high priority risks and prioritize mitigation' (E08). In addition, the template should aid in allocating responsibility for addressing potential negative impact. For example, when critiquing the template (V5), E05 noted that the template should 'acknowledge that some mitigation might come from the researchers, but some might be from the company or society as a whole.' Hence, E05 recommended that the template should 'provide space for researchers to write down: who is responsible for the mitigation? If [researchers] are not responsible, do [they] know whose job it is? [Researchers] can offload responsibilities to someone else, but it is [their] job to clearly indicate who is responsible in the assessment. '

DC-7. Be up front about the limitations and scope of impact assessment. Limitations include the impossibility of anticipating every potential negative impact, the impossibility of foreseeing long-term impacts prior to applying the research in practice, difficulties in quantifying societal effects, and the subjective nature of interpreting societal impact.

Experts noted that a single research team 'can't exhaust all the potential use cases [of the research] or predict how people from a different culture or region might be impacted by their work' (E04), and as such, recommended that template designers frame impact assessment as one component of a broader, continuous dialogue. Many experts stressed that to be effective, self-reflection through the template should lead to further action. E06 suggested that the 'template should be a starting point for researchers to consider who else they should consult.' Similarly, E15 highlighted that, even after completing the template, 'researchers should collaborate with product and compliance teams to monitor the usage of the research, as some harms may only become evident later.' Echoing prior research [3, 88], experts advised that template designers should include disclaimers outlining these and other limitations of impact assessment.

## 6.2 Design Considerations for Use Within an Organization

The following design considerations apply when the impact assessment template will be used within an organization, for example, as part of internal compliance processes.

DC-8. Clarify the role of the impact assessment template within the organization. Specifically, clearly articulate how the completed template will be used and shared within the organization, including any influence it may have on the release or use of research findings. Specify procedures for reviewing completed templates and how the results impact project trajectories.

Experts suggested several key questions to answer when using the template within an organization: At what stage will researchers fill out the template? Who is responsible for completing and reviewing it? Is the process voluntary or mandatory? What incentives or mechanisms are in place to encourage template completion? How does the template integrate with existing organizational procedures? Responses to these questions would vary across organizations. Organizations could potentially develop companion guidelines that cover these questions, similar to Microsoft's Responsible AI Impact Assessment Guide for product teams [89].

Experts emphasized the importance of clarity on how 'socially the template will be situated within an organization' (E09). For example, the template should 'clearly indicate the audience for the questions and ensure appropriate language is used' (E05), since researchers may be accustomed to academic styles of writing which would not be suitable for communicating with stakeholders like product teams. Understanding the intended audience can also incentivize higher quality responses. E12 shared that in their organization, 'once we told the researchers that their manager will read [the assessment], some C-suite executives will also see it...now we see a drastic improvement in the quality of the assessments.'

DC-9. Enable knowledge sharing around impact assessments across teams within an organization. This includes sharing outcomes, examples, experiences, and insights.

Similar to what Do &amp; Pang et al. found in their study with academic computer science researchers [44], many industry researchers in our study also expressed an interest in learning from the experiences of others, especially those in related research areas or within the same organization. For instance, P07 was curious about 'how other VR researchers plan to monitor abusive behaviors' and P15 wanted to know 'what other researchers in my company typically include for this question about the possible stakeholders.' Experts also underscored the importance of enabling knowledge sharing among teams in research, product, and compliance. E02, for example, suggested having

researchers document 'challenges faced while filling out the template for some nice self-reflection...and also [to] provide some valuable insights for other researchers.'

Some researchers mentioned seeing value in their companies hosting a 'workshop to reflect on effective ways to complete the template' (P16). Resonating with recommendations from prior work [18, 44], many experts proposed collecting case studies and news articles on unintended consequences of computing research, alongside 'exemplar assessments to guide those unfamiliar with the impact assessment process' (E12). Some experts, however, cautioned against relying on repositories of completed templates. E02 and E07 both expressed concerns that this might inadvertently encourage researchers to copy answers without deeply reflecting on their own research.

## DC-10. Ensure consistency with other existing tools and guidelines within an organization. For example, reference and integrate other existing documents and resources within the template.

Experts emphasized that template designers need to be cognizant of existing resources and processes within the organization, ensuring the template complements rather than conflicts with or duplicates them. For example, some experts suggested that the template should 'leverage resources from existing departments or programs in the company' (E14) or even 'directly plug into an existing process' (E12). Related to DC-5, experts recommended the template should highlight the additional value it could provide to researchers while ensuring it is coherent with other organizational infrastructure.

In line with the findings of past work on responsible computing in industry [40, 62, 77, 84, 156], many experts highlighted the need to tailor the template to align with institutional infrastructure. E07 noted while reviewing the template, 'We have known limitations and [...] misuse in the &lt;internal reviewing tool&gt;. I am wondering if there is a way to connect both [the internal tool and this impact assessment].' When envisioning using the template within their organization, E11 suggested that 'for misuse, I would also suggest adding &lt;common internal term&gt;, because that is the language we use.'

## 7 THE SOCIETAL IMPACT ASSESSMENT TEMPLATE

In addition to the previous 10 design considerations, the iterative co-design activity also yielded seven versions of the Societal Impact Assessment template (Figure 3). In this section, we provide a high-level description of Version 7 (referred to simply as the SIA template for the rest of this paper). The SIA template includes a list of scaffolding elements and 16 questions divided into 4 main sections (Figure 4). We include the full SIA template in the supplementary materials and at https://perma.cc/B396-DMTV.

As described in Section 4.1, the scaffolding elements and questions in Version 1 of the SIA template drew on a rich line of prior tools and research. They were then iteratively refined by incorporating feedback from both industry computing researchers and impact assessment experts throughout the co-design process. The resulting SIA template, along with the previous 10 high-level design considerations, serve as two complementary resources for future researchers and practitioners to build on, perhaps when developing impact assessment tools and processes for their own organizations. In this section, we demonstrate how (most of) the design considerations in Section 6 were fulfilled in the SIA template. Note that DC-4 is not sufficiently addressed by the current SIA template, nor are DC-8 through DC-10, since these pertain to the template's use within a specific organizational context. We discuss these limitations and future improvements in Sections 8 and 9.2.

Onboarding Section. Because researchers are not necessarily familiar with the process of conducting impact assessments, may not be well-motivated to do so, or might even misunderstand what constitutes societal impact, we provide an onboarding video for researchers who plan to use the SIA template. In the onboarding video, we first highlight the reasons for using the SIA template to motivate researchers to complete it throughout the research life cycle ( DC-5 ). We clarify that

the template is not for addressing risks to human subjects in research, a common misconception (Section 5.3), nor is it a substitute for existing institutional review processes ( DC-10 ). While previewing the template's structure (Figure 4), we highlight that researchers may skip back and forth between the three subsections on potential impact to support non-linear thinking ( DC-2 ). Finally, researchers are instructed to document any challenges faced in answering questions and potentially revisit them, instead of skipping them or providing inadequate responses ( DC-3 and DC-7 ).

Fig. 4. Overview of the SIA template's structure. Please refer to the SIA template included in the supplementary materials for more details on the questions we included in each section and how we guide researchers in thinking through these questions.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01057v3-figure-72976f0c-bc71-4122-94eb-8200f8ff5cba.jpg)

Section 1: Research Project Background. To ensure an effective impact assessment, researchers should describe their project in accessible language for the template's intended readers ( DC-8 ). Therefore, this section first requires a research project description with sufficient details using sufficiently accessible language for readers such as product and compliance teams to understand. We then provide a list of potential assets that researchers might release (e.g., paper, data, code, demos) and prompt researchers to reflect on whether these assets will be shared publicly or only internally within their own institution, as the release of different assets and different release plans can raise different risks ( DC-6 ).

Section 2: Intended Uses and Applications. Because the likelihood of industry research being integrated into products and services might affect the risks raised by the research or the urgency to address them ( DC-6 ), this section guides researchers to share the envisioned applications of the research, with a focus on its relevance to products or services within the organization. This information also serves as a basis for researchers to refer to when considering dimensions of potential impact in the remaining sections.

Section 3: Potential Impact. Here we provide a substantial amount of support for researchers to consider various dimensions of societal impact ( DC-1 ). In the template, we highlight that this section should only serve as a starting point for impact assessment, and researchers should revisit their responses as many times as needed throughout their research ( DC-3 &amp; DC-7 ). Finally, we repeatedly remind researchers that they may skip back and forth between the three subsections, as thinking through impacts may require non-linear thinking (e.g., first thinking about possible harms before considering specific stakeholders) ( DC-2 ). As shown in Figure 4, we decided not to number the three subsections to encourage researchers to complete them in any order ( DC-2 ).

- - Intended Positive Societal Impact. Before diving into negative societal impact, this subsection prompts researchers to think through the intended positive societal impact of their research, by asking them to describe the best-case scenario for all the impacted stakeholders they envisioned in the Stakeholders subsection. To scaffold brainstorming, the SIA template includes a brief description
- - Stakeholders. This subsection scaffolds researchers to identify the possible stakeholders who might use or be impacted by their research, including its future applications. We list individuals (directly and indirectly impacted), marginalized groups, companies, industries, governments, and civil society as potential stakeholders to encourage researchers to think broadly about those who might take up or be impacted by their work. Brief descriptions are provided for each stakeholder category to assist those less experienced in considering these groups.

of each of the following potential types of impact for researchers to consider: economic impact, policy impact, environmental impact, cultural impact, and knowledge impact.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.01057v3-figure-1cc8ac90-94f2-4ed6-81a7-b29bfd5f0144.jpg)

- - Potential Negative Societal Impact. This subsection starts by guiding researchers in considering and documenting the limitations of their research, on the belief that it is helpful to consider how undisclosed or unrecognized limitations can lead to unintentional misuse of research findings and corresponding negative societal impacts. Researchers are then prompted to contemplate potential harms stemming from misuse (accidental unintended use), abuse (intentional malicious use), and intended use (Figure 5). We provide concrete examples for each of these three scenarios to support brainstorming.

Section 4: Planning for Mitigation. This final section guides researchers step-by-step through developing actionable strate-

Fig. 5. Illustration included in the SIA template to guide researchers thinking through scenarios leading to potential negative societal impact.

gies to address potential negative societal impacts surfaced in the previous section ( DC-6 ). The section first helps researchers assess the seriousness of the identified risks by considering the likelihood, magnitude, concentration, and time horizon of the potential negative impacts, and asks researchers to document which risks they view as high and low priority in light of these differences. We then provide a list of methods to help researchers plan possible mitigations throughout different stages of the research life cycle. Finally, we ask researchers how the risks identified as high-priority can be mitigated and what actions they can take to ensure that these mitigations are put in place. To more realistically reflect the scope of what researchers themselves might be able to do to address identified risks ( DC-7 ), we also inquire about steps they can take to encourage others to implement these mitigations if they cannot do so themselves.

## 8 USER STUDY OF THE SOCIETAL IMPACT ASSESSMENT (SIA) TEMPLATE

In this section, we present the findings from the one-week user study, described in Section 4.4 . We report the participants' ratings from the exit survey and their comments from their exit interviews, conducted after they had a week to complete the template. For each rating, we report the mean value ( 𝜇 ) and standard deviation ( 𝜎 ) on a scale of 1 to 5, where 1 represents 'strongly disagree' and 5 represents 'strongly agree.'

As shown in Table 3, participants were in different stages of their research projects: four in early stages, eight in the middle, and three late stages, writing up results. On average, they spent 81.26 minutes (SD=37.58) filling out the template, with reported times ranging from 25 to 150 minutes. Out of the 15 participants, 11 reported revisiting the template during the one-week user study period: four revisited it once, four revisited it twice, one revisited it three times, one revisited it four times, and one revisited it six times.

## 8.1 Perceived Strengths of the SIA Template

In general, participants perceived the template's structure and content as clear and easy to navigate . 87% of participants (13/15) agreed (i.e., selected either 'agree' or 'strongly agree' in the exit survey) that the template is 'easy to understand' ( 𝜇 =4.27, 𝜎 =0.71) and the 'instructions provided with the template are clear' ( 𝜇 =4.33, 𝜎 =0.72). A significant 93% of participants (14/15) agreed that the 'template's organization makes sense and is intuitive to navigate' ( 𝜇 =4.40, 𝜎 =0.83). T08 noted in their exit survey that the SIA template presented ' a thoughtfully laid out and thought-provoking assessment. Very relevant to the current state of technological advances in [their] research field.'

The template also effectively facilitated the identification of unconsidered aspects of societal impact, promoting researchers' critical reflection . Participants rated it highly for the template's utility in identifying potential stakeholders who could benefit from or be impacted by the research ( 𝜇 =4.27, 𝜎 =0.59 and 𝜇 =4.40, 𝜎 =0.74, respectively), as well as potential positive ( 𝜇 =4.00, 𝜎 =0.93) and negative ( 𝜇 =4.07, 𝜎 =1.03) societal impacts. T04 reported that the template prompted them to consider 'minority job candidates who might be affected' and to rethink how their work should be framed and presented. T13 highlighted in the exit survey that 'it was very helpful to think about stakeholders in an increasingly broader context, starting with individuals and ending at governmental level. ' T07 mentioned in their exit survey that 'I originally thought my research had no negative impact because it was merely assistive tech for blind people...but when I really thought about it when filling it out, I found that there were unintended consequences I never thought of.' They added during the exit interview that they would update the system design to potentially mitigate these unintended harms.

Finally, completing the template improved participants' personal motivation to grapple with the potential negative societal impact of their work. Anotable 80% of participants (12/15) agreed that 'the template convinced (or further convinced) me that I should consider and grapple with the potential negative societal impact of my research' ( 𝜇 =4.20, 𝜎 =0.86). All participants agreed that they would 'recommend this template to others conducting computing research' ( 𝜇 =4.27, 𝜎 =0.45). Participants such as T02, T06, T13, and T14 specifically mentioned in both their exit surveys and exit interviews that they would continue using the template themselves for future research projects to assess, articulate, and address potential negative societal impacts.

Additionally, participants found that completing the template was valuable for their research . In particular, a majority of participants (14/15) felt the template could help them 'draft a societal impact statement to include in my research paper' ( 𝜇 =4.40, 𝜎 =0.73). T12 felt 'more prepared to write a blog post about the work after using this template.' In addition, 73% of participants (11/15) found 'the template adds value to the research process' ( 𝜇 =4.07, SD = 0.79). During the exit interview, many participants, including T01, T03-07, T11, and T13, indicated that completing the template helped them enhance the rigor of their research and identify new research directions.

## 8.2 Perceived Limitations and Potential Future Iterations of the SIA Template

Despite overall positive feedback, a number of participants (8 out of 15) felt neutral or disagreed (i.e., selected either 'neutral,' 'disagree,' or 'strongly disagree' in the exit survey) with the statement that 'I would use this template again in the future' ( 𝜇 =3.53, 𝜎 =4.27). Through the exit interview, we identified two main reasons why participants were hesitant about using the template. First, the current SIA template is viewed as too long . Around half of the participants (7/15) disagreed that 'the template length is suitable to help with self-reflection,' with three selecting 'neutral' and four selecting 'disagree' ( 𝜇 =3.40, 𝜎 =1.06). T04's comment in the exist survey captured many participants' concerns: 'It was long - I would do this if properly incentivized (if we were going to talk about it in a meeting for a project, for example), but it's maybe too long for me to fill out voluntarily on my other projects when I already have other tasks to do. On the flip side, it was really useful for thinking about new research directions (that help with the societal impact goal), so I might voluntarily use it.'

Second, the current SIA template is not fully suitable for some research areas. Around half of the participants (7/15) felt neutral or disagreed with the statement that 'I believe the current template is appropriate for my particular area of research' ( 𝜇 =3.73, 𝜎 =1.10). T02, who works on reinforcement learning theory, shared during the interview that 'most questions in section 3 are not applicable for this project' because their project set-up is very specific and only a small group of reinforcement learning theorists can make use of their current results. Similarly, T10 shared in their exit survey that: 'I think this template is really well suited to people doing systems design and development work. For those working in more critical/communications/sociology field the template

| ID   | Research Area          | Project Stage          | Time Spent   |   Revisit # | Length   |
|------|------------------------|------------------------|--------------|-------------|----------|
| T01  | NLP                    | Executing the research | 90 (Minutes) |           1 | 1,062    |
| T02  | Reinforcement Learning | Executing the research | 25           |           0 | 147      |
| T03  | Mathematics            | Early brainstorming    | 60           |           1 | 702      |
| T04  | Machine Learning       | Executing the research | 75           |           2 | 1,079    |
| T05  | AI, Healthcare         | Early brainstorming    | 75           |           0 | 1,312    |
| T06  | Robotics, AI           | Executing the research | 30           |           0 | 246      |
| T07  | Accessibility          | Executing the research | 90           |           3 | 1,189    |
| T08  | HCI, Design            | Early brainstorming    | 150          |           2 | 702      |
| T09  | NLP, HCI               | Executing the research | 60           |           2 | 979      |
| T10  | Sociology              | Executing the research | 75           |           1 | 672      |
| T11  | VR, AR                 | Preparing publication  | 60           |           0 | 1,143    |
| T12  | NLP, HCI               | Executing the research | 90           |           1 | 418      |
| T13  | NLP                    | Preparing publication  | 150          |           6 | 1,136    |
| T14  | NLP                    | Early brainstorming    | 90           |           1 | 461      |
| T15  | NLP, Accessibility     | Preparing publication  | 120          |           4 | 838      |

Table 3. Research interns who participated in our one-week user study. Research areas, project stage, the time spent completing the SIA template, and the number of revisions were all self reported. Length is the word count of their template responses.

isn't as applicable. ' However, during the exit interview, T10 mentioned that they could see that with a modification, ideally by researchers and domain experts in their own areas, they could 'make the template more useful to qualitative research based on the current structure and content.'

In line with our observations from the co-design study (DC-2 in Section 6.1), around half of the participants (8/15) indicated they navigated back and forth between sections when completing the 'Potential Impact' portion of the template. However, participants desired more support for non-linear thinking when contemplating the many possible dimensions of societal impact . For example, T05 shared in the survey that they 'had difficulty going back and forth between sections (stakeholders, harms, etc)' and suggested alternative presentation, such as a table that 'detail[s] each consideration (columns) for each stakeholder (rows).' During the exit interview, many participants discussed the idea of making the template more interactive and dynamic to allow researchers to move between sections more easily.

Validating experts' suggestions that the template should help researchers prioritize risks and develop appropriate responses (DC-6 in Section 6.1), many participants, such as T01, T03, T07-T11, and T13-T15, shared in their exit interviews that they found the scaffold for determining the seriousness of possible harms and for planing mitigation useful. However, some participants shared that they desired even greater support to effectively evaluate the likelihood and magnitude of harms and to formulate appropriate mitigations . For example, T08 believed that they needed to engage with users and discuss design alternatives to identify suitable mitigations. This need for additional support explains why participants rated 'Helping me prepare mitigations for the potential negative societal impact of my research' relatively low ( 𝜇 =3.67, 𝜎 =1.13). In the exit interviews, some participants, such as T01, T04, and T13, suggested that we make the template more interactive, perhaps using generative AI to aid in brainstorming.

Interestingly, 7 out of 15 participants disagreed that 'the template helped me overcome some of the practical challenges that have kept me from considering and grappling with the potential negative societal impact of my research' ( 𝜇 =3.47, 𝜎 =1.19). Through the exit interview, we learned that this

was because they 'didn't perceive any practical challenges preventing [them] from assessing the negative societal impact in the first place' (T09). This feedback may be attributed to the recruitment of industry research interns rather than full-time industry researchers for the one-week user study, a limitation we discuss in the following section.

## 9 DISCUSSION

## 9.1 Unique Challenges for Industry Computing Researchers

Our work extends prior empirical research by highlighting the unique challenges faced by industry computing researchers when grappling with the potential negative societal impact of their work (Section 5). In particular, they encounter challenges shaped by team structures and organizational hierarchies (Section 5.1), organizational dynamics and culture (Section 5.2), and infrastructure provided - or not provided - by their companies (Section 5.4).

Our findings also indicate that even when industry computing researchers are encouraged to consider the potential negative impacts of their work, they often face communication and collaboration barriers with product or compliance teams. These barriers arise from differences in knowledge and vocabulary (Section 5.4), differing timelines and priorities stemming from different job responsibilities (Section 5.2), as well as a lack of transparency into and control over how researchers' work will be incorporated into products and services (Section 5.1). One promising direction is to create 'boundary objects' that can sit among researchers, product teams, and legal and compliance teams to facilitate collaboration [79, 104, 129, 135]. The SIA template itself could potentially serve as a prototype for developing future boundary objects to help researchers communicate their findings in a format and language accessible to other team members within their organizations. Future research should also explore how the SIA template can be integrated with other existing responsible computing tools that have been adopted in industry [20, 55, 66, 67, 91, 114].

Like their peers in the academy, industry computing researchers aim to publish and present at peer-reviewed conferences. As a result, they share similar concerns as researchers in the academy who worry that highlighting the potential negative societal impacts of their work might decrease the likelihood of their work being accepted at these conferences - and thus affect their careers [3, 18, 44]. But industry computing researchers differ from their peers in the academy because their attempts to grapple with these impacts may also be at odds with their companies' interests, placing their jobs at more immediate risk [c.f. 149].

In line with the extensive empirical research on responsible computing and AI development under the organizational cultures and dynamics within industry settings [39-41, 64, 82, 84, 105, 106, 111, 145, 150, 152, 156], we recognize that the challenges faced by industry researchers in addressing negative societal impacts often stem from business-oriented incentives that prioritize existing customers, larger consumer groups, or wealthier markets segments. Addressing these challenges requires political changes that fundamentally alter the prevailing culture of innovation [82, 111, 152] and power dynamics between technology workers and leadership [41, 106, 148, 150]. While we offer practical approaches that could benefit industry computing researchers and technology companies in the short term, broader changes to the technology industry might be required.

## 9.2 Beyond a Societal Impact Assessment Template

Guiding industry computing researchers to engage in self reflection and complete the impact assessment template is merely an initial step towards addressing the potential negative impacts from industry research. To be effective, impact assessments conducted within companies must be complemented by external mechanisms and policies to drive meaningful changes in technology design and deployment [3, 88, 99].

Social science and legal scholars have cautioned against 'performative accountability,' where companies may appear committed to accountability (e.g., by conducting an impact assessment) but engage in practices that undermine public values in reality [45, 54, 141, 142]. Therefore, future researchers, practitioners, and policymakers should draw lessons from the experiences of other fields with impact assessment processes such as environmental impact assessments [31, 107], human right impact assessments [137], and data protection impact assessments [37, 57], among others, in seeking to develop an effective impact assessment for research [5, 88, 93, 113]. Future work should also draw from works in these domains to explore possible ways to try to get people to actually use the impact assessment, ranging from large-scale changes to the norms of the research community to formal policy requirements within particular institutions [116].

Future work should also consider changes to the format of the SIA template to try to overcome some of limitations we identified in Section 8.2. Our co-design mainly focused on the content of a static template; future research could explore new forms of impact assessment beyond a template, such as interactive tools or guidelines [c.f. 6, 65, 95, 101, 102, 120, 126, 154, 155]. An increasing line of recent work has also explored the use of emerging generative AI technologies to assist in brainstorming and creating incident cases [28, 73, 101-103, 112]. However, these studies have also highlighted the caveats and limitations of using generative AI in tasks like impact assessment [28, 112]. More work needs to be done to provide more engaging and effective scaffolding.

In their recent work, Metcalf et al. have also highlighted the risk of conducting impact assessments without the meaningful engagement from diverse stakeholders, as the assessment outcomes might be 'inappropriately distant from the harms experienced by people' [88]. Impact assessments of industry research additionally need to be (1) evaluated by researchers and experts with diverse backgrounds and (2) contested by the communities impacted by the research. Recent work done by Kieslich et al. also demonstrated the importance of engaging end users in anticipating AI impacts [73]. Drawing from prior HCI research [c.f. 104, 120-122, 126, 135, 154], future research could explore tools and processes to facilitate meaningful conversations and deliberations between researchers and diverse community members in discussing how certain research might impact their lives. Researchers should also work closely with policymakers to explore the possibility of incorporating these tools into formal regulatory processes (e.g., in the decision-making of government funding bodies) [87, 110, 113].

## 9.3 Limitations

As mentioned in Section 8.2, one key limitation of our work is the validity of using industry research interns from a single company for the one-week user study . Although testing with research interns still provided valuable insights into the usability and usefulness of our template, their experience and challenges might differ from those of full-time researchers due to variations in their roles, responsibilities, and experiences. In addition, although a one-week study period allowed us to start to understand how researchers would use the template, ideally, we would observe its usage throughout the entire lifecycle of a research project - from formulating research questions to executing the research, publishing papers, and integrating findings into actual products. To this end, future work should explore ecologically valid methods for evaluating the template within an organizational context. For instance, researchers could conduct a longitudinal study to examine how full-time industry researchers use a version of the SIA template, tailored to their specific research domain and organizational dynamics, at different stages of a research project [c.f. 106]. Future work could also explore how different research roles in industry (e.g., research interns, fulltime researchers, and research managers) might have different incentives when using the impact assessment for their research. Researchers could also collaborate with conference organizers to study how researchers fulfill conference requirements using the SIA template [c.f. 133], or perform

a between-subjects study comparing the quality of impact statements between researchers who use the template and those who do not [c.f. 132].

Finally, ideally, we would have left more time between the interviews and co-design sessions to allow more reflections on the interview data to update the impact assessment template.

Another limitation is that our purposive and snowball sampling approach limited the diversity of practitioners involved . Despite efforts to recruit participants from diverse regions, the majority of industry researchers and experts in our study were from the U.S. and U.K. Therefore, future research is necessary to determine whether our empirical findings and design considerations are applicable in non-Western organizational contexts [70, 134]. Moreover, our positionality as U.S.-based researchers with backgrounds in AI and HCI influenced our data interpretation and the integration of participant feedback into the template's design and development [63]. Therefore, we call for future work on this topic conducted by individuals with different backgrounds, including those outside academia and industry.

## 9.4 Potential Negative Societal Impact and a Plan for Mitigation

Olteanu et al. [98] recently argued that research on ethical issues in computing can also create their own risks of harm. Our work is no exception. To assess these risks, we conducted an impact assessment of our own work using the SIA template. We identified a number of dangers. First, researchers and companies who use our template might be left with a false sense of confidence after completing their impact assessments, mistakenly believing that they have exhausted all potential harms that might arise due to the research in question. Second, they might also conclude that it is enough to just conduct the assessment without taken meaningful action towards addressing any of the identified harms. To address these risks, we explicitly highlight them within the template itself, in line with 6.1, and we offer concrete future steps for other researchers and organizations to avoid this in Section 9.2. When open-sourcing the SIA template through a project web page, we plan to further underscore the appropriate use of impact assessments and the importance of continuous monitoring.

Finally, conducting an impact assessment might create an unreasonable burden on researchers , in line with feedback from the user study. Like IRBs, impact assessments also run the risk of stifling research freedom if the procedures are designed poorly [58, 125]. To guard against these risks, organizations should formally recognize the 'invisible labor' that goes into responsible computing practices [cf. 4, 41, 130, 152], appropriately accounting for these efforts in employees' job descriptions and performance reviews. Organizations should also establish and support teams that can train researchers, provide advice, and help reason through difficult questions [cf. 18, 99].

For-profit companies might exploit the SIA template as a tool for ethics washing [4, 39, 111, 148], as others have observed in prior research studying responsible AI [39, 84, 111], privacy and data protection [14, 59, 136, 142], and environmental justice [27, 42, 107, 141]. Prior work on algorithmic auditing [15, 18, 43, 44, 87, 109] offers helpful suggestions for how to ensure the integrity of the impact assessment process and bring about meaningful change, often via external review.

## 10 CONCLUSION

Through interviews with 25 industry computing researchers, co-design sessions with both these researchers and 16 impact assessment experts, and a user study with 15 industry research interns, this paper (1) sheds light on current perceptions, practices, and challenges among computing researchers seeking to confront the potential negative societal impacts of their work (Section 5), (2) provides guidance for researchers, organizations, and policymakers to improve these practices and overcome identified challenges (Section 6), and (3) offers a version of the SIA template with tangible scaffolding materials (Section 7) along with its user study (Section 8) to help computing as a field of research do more to grapple with the consequences of its work. By connecting and contrasting

these contributions with literature from CSCW, design, and the broader fields of HCI and social science, we illuminate paths for future researchers, industry practitioners and organizations, and policy makers to better support responsible research practices in industry settings and beyond.

## ACKNOWLEDGMENTS

We thank all participating industry researchers, impact assessment experts, and industry research interns for making this work possible. We are grateful to Snehal Prabhudesai, Yuda Song, and Samantha Dalal for joining the pilot interview study. We thank Ken Holstein, Motahhare Eslami, Kevin Feng, and Ezra Awumey for their feedback on the draft. Finally, we thank Alicia Edelman Pelton, Forough Poursabzi-Sangdeh, Sunnie Kim, Rock Pang, Michael Madaio, Nari Johnson, Hoda Heidari, Alex London, Jason Hong, Jeff Bigham, Lauren Wilcox, Eric Horvitz, the CMU FEAT reading group, the Microsoft Research FATE group, and Microsoft's Aether RADD focus group for many useful discussions.

## REFERENCES

- [1] AAAI ICWSM 2024 Organizing Committee. 2024. AAAI ICWSM Paper Checklist. (2024). https://drive.google.com/ file/d/1RMsfCq\_dVFwVIpzBUT8wUTdym-Qwi-sV/view?usp=sharing
- [3] Ada Lovelace Institute. 2022. Looking before we leap: Expanding ethical review processes for AI and data science research. (December 2022). https://www.adalovelaceinstitute.org/wp-content/uploads/2022/12/Ada-LovelaceInstitute-Looking-before-we-leap-Dec-2022.pdf
- [2] ACL, Ethic Policy. 2023. ACL 2023 Responsible NLP Research and Ethics Policy. (March 2023). https://2023.aclweb. org/calls/main\_conference/#ethics-policy
- [4] Sanna J Ali, Angèle Christin, Andrew Smart, and Riitta Katila. 2023. Walking the Walk of AI Ethics: Organizational Challenges and the Individualization of Risk among Ethics Entrepreneurs. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 217-226.
- [6] Artefact. 2022. The Tarot Cards of Tech: Discover the power of predicting impact. (Nov 2022). https://tarotcardsoftech. artefactgroup.com/.
- [5] Norberto Nuno Gomes de Andrade and Verena Kontschieder. 2021. AI impact assessment: a policy prototyping experiment. Available at SSRN 3772500 (2021).
- [7] Carolyn Ashurst, Markus Anderljung, Carina Prunkl, Jan Leike, Yarin Gal, Toby Shevlane, and Allan Dafoe. 2020. A Guide to Writing the NeurIPS Impact Statement. (May 2020). https://medium.com/@GovAI/a-guide-to-writing-theneurips-impact-statement-4293b723f832
- [9] Carolyn Ashurst, Solon Barocas, Rosie Campbell, Deborah Raji, and Stuart Russell. 2020. Navigating the Broader Impacts of AI Research. NeurIPS workshop. https://ai-broader-impacts-workshop.github.io/
- [8] Carolyn Ashurst, Solon Barocas, Rosie Campbell, and Deborah Raji. 2022. Disentangling the components of ethical research in machine learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 2057-2068.
- [10] Carolyn Ashurst, Emmie Hine, Paul Sedille, and Alexis Carlier. 2022. AI ethics statements: analysis and lessons learnt from NeurIPS broader impact statements. In 2022 ACM Conference on Fairness, Accountability, and Transparency . 2047-2056.
- [12] Agathe Balayn, Mireia Yurrita, Jie Yang, and Ujwal Gadiraju. 2023. 'Fairness Toolkits, A Checkbox Culture?' On the Factors that Fragment Developer Practices in Handling Algorithmic Harms. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society . 482-495.
- [11] James Auger. 2013. Speculative design: crafting the speculation. Digital Creativity 24, 1 (2013), 11-35.
- [13] Stephanie Ballard, Karen M Chappell, and Kristen Kennedy. 2019. Judgment call the game: Using value sensitive design and design fiction to surface ethical concerns related to technology. In Proceedings of the 2019 on Designing Interactive Systems Conference . 421-433.
- [15] Jack Bandy and Nicholas Diakopoulos. 2020. Auditing news curation systems: A case study examining algorithmic and editorial logic in Apple News. In Proceedings of the International AAAI Conference on Web and Social Media , Vol. 14. 36-47.
- [14] Kenneth A Bamberger and Deirdre K Mulligan. 2011. Privacy on the Books and on the Ground. Stanford Law Review (2011), 247-315.
- [16] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness,

|      | Accountability, and Transparency (FAccT) . 610-623. https://doi.org/10.1145/3442188.3445922                                                                                                                                                                                                                                                                                     |
|------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [17] | Luciana Benotti and Patrick Blackburn. 2022. Ethics consideration sections in natural language processing papers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . 4509-4516.                                                                                                                                                        |
| [18] | Michael S Bernstein, Margaret Levi, David Magnus, Betsy A Rajala, Debra Satz, and Quinn Waeiss. 2021. Ethics and society review: Ethics reflection as a precondition to research funding. Proceedings of the National Academy of Sciences 118, 52 (2021), e2117261118.                                                                                                          |
| [19] | Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan. 2021. Introducing the NeurIPS 2021 paper checklist. NeurIPS blog. https://blog.neurips.cc/2021/03/26/introducing-the-neurips-2021-paper-checklist/                                                                                                                                               |
| [20] | Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI . Technical Report MSR-TR-2020-32. Microsoft. https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for- assessing-and-improving-fairness-in-ai/ |
| [21] | Mark Blythe. 2014. Research through design fiction: narrative in real and imaginary abstracts. In Proceedings of the SIGCHI conference on human factors in computing systems . 703-712.                                                                                                                                                                                         |
| [22] | Edyta Bogucka, Marios Constantinides, Sanja Šćepanović, and Daniele Quercia. 2024. Co-designing an AI impact assessment report template with AI practitioners and AI compliance experts. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , Vol. 7. 168-180.                                                                                                |
| [23] | Margarita Boyarskaya, Alexandra Olteanu, and Kate Crawford. 2020. Overcoming Failures of Imagination in AI Infused System Development and Deployment. arXiv preprint arXiv:2011.13416 (2020).                                                                                                                                                                                   |
| [24] | Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77-101.                                                                                                                                                                                                                                        |
| [25] | Virginia Braun and Victoria Clarke. 2019. Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health 11, 4 (2019), 589-597.                                                                                                                                                                                                                  |
| [26] | Philip AE Brey. 2012. Anticipatory ethics for emerging technologies. NanoEthics 6, 1 (2012), 1-13.                                                                                                                                                                                                                                                                              |
| [27] | Howard Brown and Timothy Larson. 1998. Making business integration work: A survival strategy for EHS managers. Environmental Quality Management 7, 3 (1998), 1-8.                                                                                                                                                                                                               |
| [28] | Zana Buçinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, and Saleema Amershi. 2023. AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms. arXiv preprint arXiv:2306.03280 (2023).                                                                                                                                              |
| [29] | Steve Campbell, Melanie Greenwood, Sarah Prior, Toniele Shearer, Kerrie Walkem, Sarah Young, Danielle Bywaters, and Kim Walker. 2020. Purposive sampling: complex or simple? Research case examples. Journal of research in Nursing 25, 8 (2020), 652-661.                                                                                                                      |
| [30] | Canadian Institute For Advanced Research. 2022. A Culture of Ethical AI. (July 2022). https://cifar.ca/wp-content/ uploads/2022/08/CIFAR-AI-Insights-EN-FINAL-AODA.pdf                                                                                                                                                                                                          |
| [31] | Matthew Cashmore, Richard Gwilliam, Richard Morgan, Dick Cobb, and Alan Bond. 2004. The interminable issue of effectiveness: substantive purposes, outcomes and research challenges in the advancement of environmental impact assessment theory. Impact Assessment and Project Appraisal 22, 4 (2004), 295-310.                                                                |
| [32] | Center for Advanced Study in the Behavioral Sciences. 2020. Ethics & Society Review · Stanford University. (2020). https://casbs.stanford.edu/ethics-society-review-stanford-university                                                                                                                                                                                         |
| [33] | Chief Information Officers Council, U.S. 2021. Algorithmic Impact Assessment. (May 2021). https://www.cio.gov/aia- eia-js/#/                                                                                                                                                                                                                                                    |
| [34] | Adele E Clarke et al. 2016. Anticipation work: Abduction, simplification, hope. Boundary objects and beyond: Working with Leigh Star 85 (2016).                                                                                                                                                                                                                                 |
| [35] | CVPR, Ethics Guidelines. 2023. CVPR 2024 Ethics Guidelines for Authors. (November 2023). https://cvpr2023.thecvf. com/Conferences/2023/EthicsGuidelines                                                                                                                                                                                                                         |
| [36] | Lachlan D. Urquhart and Peter J. Craigon. 2021. The Moral-IT Deck: a tool for ethics by design. Journal of Responsible Innovation 8, 1 (2021), 94-126.                                                                                                                                                                                                                          |
| [37] | Data Protection Impact Assessment, General Data Protection Regulation. 2023. GDPR, Privacy Impact Assess- ment. (Nov 2023). https://gdpr-info.eu/issues/privacy-impact-assessment/#:~:text=Basically%2C%20a%20data% 20protection%20impact,of%20the%20GDPR%20is%20relevant.                                                                                                      |
| [38] | Scott Davidoff, Min Kyung Lee, Charles Yiu, John Zimmerman, and Anind K Dey. 2006. Principles of smart home control. In UbiComp 2006: Ubiquitous Computing: 8th International Conference, UbiComp 2006 Orange County, CA, USA, September 17-21, 2006 Proceedings 8 . Springer, 19-34.                                                                                           |
| [39] | Wesley Hanwen Deng, Boyuan Guo, Alicia Devrio, Hong Shen, Motahhare Eslami, and Kenneth Holstein. 2023.                                                                                                                                                                                                                                                                         |

- [39] Wesley Hanwen Deng, Boyuan Guo, Alicia Devrio, Hong Shen, Motahhare Eslami, and Kenneth Holstein. 2023. Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-18.

- [40] Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, and Haiyi Zhu. 2022. Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits. In 2022 ACM Conference on Fairness, Accountability, and Transparency . ACM, Seoul Republic of Korea, 473-484. https://doi.org/10.1145/3531146.3533113
- [42] Andrea di Miceli, Birgit Hagen, Maria Pia Riccardi, Francesco Sotti, and Davide Settembre-Blundo. 2021. Thriving, Not Just Surviving in Changing Times: How Sustainability, Agility and Digitalization Intertwine with Organizational Resilience. Sustainability (2021).
- [41] Wesley Hanwen Deng, Nur Yildirim, Monica Chang, Motahhare Eslami, Kenneth Holstein, and Michael Madaio. 2023. Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 705-716.
- [43] Nicholas Diakopoulos. 2015. Algorithmic accountability: Journalistic investigation of computational power structures. Digital journalism 3, 3 (2015), 398-415.
- [45] Lauren B Edelman and Shauhin A Talesh. 2011. To comply or not to comply-That isn't the question: How organizations construct the meaning of compliance. Explaining compliance: Business responses to regulation (2011), 103-122.
- [44] Kimberly Do, Rock Yuren Pang, Jiachen Jiang, and Katharina Reinecke. 2023. 'That's important, but...': How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-16.
- [46] Salma Elsayed-Ali, Sara E Berger, Vagner Figueredo De Santana, and Juana Catalina Becerra Sandoval. 2023. Responsible &amp; Inclusive Cards: An online card tool to promote critical reflection in technology industry work practices. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-14.
- [48] Shelley Evenson. 2006. Directed storytelling: Interpreting experience for design. Design Studies: Theory and research in graphic design (2006), 231-240.
- [47] Chris Elsden, David Chatting, Abigail C Durrant, Andrew Garbett, Bettina Nissen, John Vines, and David S Kirk. 2017. On speculative enactments. In Proceedings of the 2017 CHI conference on human factors in computing systems . 5386-5399.
- [49] United States. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 1978. The Belmont report: ethical principles and guidelines for the protection of human subjects of research . Vol. 1. Department of Health, Education, and Welfare, National Commission for the ....
- [51] Batya Friedman and David G Hendry. 2019. Value sensitive design: Shaping technology with moral imagination . Mit Press.
- [50] Sarah Fox, Amanda Menking, Stephanie Steinhardt, Anna Lauren Hoffmann, and Shaowen Bardzell. 2017. Imagining intersectional futures: Feminist approaches in CSCW. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing . 387-393.
- [52] Batya Friedman, Peter H Kahn, Alan Borning, and Alina Huldtgren. 2013. Value sensitive design and information systems. Early engagement and new technologies: Opening up the laboratory (2013), 55-95.
- [54] Ben Gansky and Sean McDonald. 2022. CounterFAccTual: How FAccT undermines its organizing principles. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency . 1982-1992.
- [53] Batya Friedman and Lisa P Nathan. 2010. Multi-lifespan information system design: A research initiative for the HCI community. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . 2243-2246.
- [55] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. 2021. Datasheets for datasets. Commun. ACM 64, 12 (2021), 86-92.
- [57] Government of Canada. 2023. Privacy Impact Assessment, Office of teh Privacy Commissioner of Canada. (Nov 2023). https://www.priv.gc.ca/en/privacy-topics/privacy-impact-assessments/
- [56] Government of Canada. 2020. Algorithmic Impact Assessment, Government of Canada. (May 2020). https: //open.canada.ca/data/en/dataset/5423054a-093c-4239-85be-fa0b36ae0b2e
- [58] Christine Grady. 2015. Institutional review boards: Purpose and challenges. Chest 148, 5 (2015), 1148-1155.
- [60] Brent Hecht. 2020. Suggestions for Writing NeurIPS 2020 Broader Impacts Statements. (2020). https://brenthecht. medium.com/suggestions-for-writing-neurips-2020-broader-impacts-statements-121da1b765bf
- [59] Marco Gutfleisch, Jan H Klemmer, Niklas Busch, Yasemin Acar, M Angela Sasse, and Sascha Fahl. 2022. How does usable security (not) end up in software products? results from a qualitative interview study. In 43rd IEEE Symposium on Security and Privacy, IEEE S&amp;P . 22-26.
- [61] Brent Hecht, Lauren Wilcox, Jeffrey P Bigham, Johannes Schöning, Ehsan Hoque, Jason Ernst, Yonatan Bisk, Luigi De Russis, Lana Yarosh, Bushra Anjum, et al. 2021. It's time to do something: Mitigating the negative impacts of computing through a change to the peer review process. arXiv preprint arXiv:2112.09544 (2021).
- [62] AmyK.Heger, Liz B. Marquis, Mihaela Vorvoreanu, Hanna Wallach, and Jennifer Wortman Vaughan. 2022. Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata. Proceedings of the ACM on Human-Compututer Interaction 6, CSCW2, Article 340 (2022), 29 pages. https://doi.org/10.1145/3555760

- [63] Andrew Gary Darwin Holmes. 2020. Researcher Positionality-A Consideration of Its Influence and Place in Qualitative Research-A New Researcher Guide. Shanlax International Journal of Education 8, 4 (2020), 1-10.
- [65] Matthew K Hong, Adam Fourney, Derek DeBellis, and Saleema Amershi. 2021. Planning for natural language failures with the ai playbook. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-11.
- [64] Kenneth Holstein, Bruce M McLaren, and Vincent Aleven. 2019. Designing for complementarity: Teacher and student needs for orchestration support in AI-enhanced classrooms. (2019), 157-171.
- [66] IBM Resaerch Trusted AI. 2021. AIF360 API. (2021). https://aif360.res.ibm.com/
- [68] ICML, Publication Ethics. 2023. International Conference on Machine Learning, Publication Ethics. (November 2023). https://icml.cc/Conferences/2023/PublicationEthics
- [67] IBM Resaerch Trusted AI. 2021. AIX360 API. (2021). https://aix360.mybluemix.net/
- [69] Marina Jirotka, Barbara Grimpe, Bernd Stahl, Grace Eden, and Mark Hartswood. 2017. Responsible research and innovation in the digital age. Commun. ACM 60, 5 (2017), 62-68.
- [71] Nari Johnson and Hoda Heidari. 2023. Assessing AI Impact Assessments: A Classroom Study. arXiv preprint arXiv:2311.11193 (2023).
- [70] Anna Jobin, Marcello Ienca, and Effy Vayena. 2019. The global landscape of AI ethics guidelines. Nature Machine Intelligence 1, 9 (2019), 389-399.
- [72] Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting interpretability: understanding data scientists' use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems . 1-14.
- [74] Zenia Kotval. 2006. Fiscal Impact Analysis: Methods, Cases, and Intellectual Debate Zenia Kotval and John Mullin With research assistance from Maureen Lempke. (2006).
- [73] Kimon Kieslich, Nicholas Diakopoulos, and Natali Helberger. 2023. Anticipating Impacts: Using Large-Scale Scenario Writing to Explore Diverse Implications of Generative AI in the News Environment. arXiv preprint arXiv:2310.06361 (2023).
- [75] Sandjar Kozubaev, Chris Elsden, Noura Howell, Marie Louise Juul Søndergaard, Nick Merrill, Britta Schulte, and Richmond Y Wong. 2020. Expanding modes of reflection in design futuring. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1-15.
- [77] Michelle Seng Ah Lee and Jat Singh. 2021. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems . 1-13.
- [76] Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. 2023. Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey. In Proceedings of the The 17th Conference of the European Chapter of the Association for Computational Linguistics .
- [78] Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, and James Zou. 2024. What's documented in AI? Systematic Analysis of 32K AI Model Cards. arXiv preprint arXiv:2402.05160 (2024).
- [80] Hsuan-Tien Lin, Maria-Florina Balcan, Raia Hadsell, and Marc'Aurelio Ranzato. 2020. Getting Started with NeurIPS 2020. NeurIPS blog. https://neuripsconf.medium.com/getting-started-with-neurips-2020-e350f9b39c28
- [79] Q Vera Liao, Hariharan Subramonyam, Jennifer Wang, and Jennifer Wortman Vaughan. 2023. Designerly understanding: Information needs for model transparency to support design ideation for AI-powered user experience. In Proceedings of the 2023 CHI conference on human factors in computing systems . 1-21.
- [81] David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R Hullman, and Tina Eliassi-Rad. 2022. Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society . 424-435.
- [83] Michael Madaio, Shivani Kapania, Rida Qadri, Ding Wang, Andrew Zaldivar, Remi Denton, and Lauren Wilcox. 2024. Learning about Responsible AI On-The-Job: Learning Pathways, Orientations, and Aspirations. In The 2024 ACM Conference on Fairness, Accountability, and Transparency . 1544-1558.
- [82] Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. 2022. Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 26 pages.
- [84] Michael A Madaio, Luke Stark, Jennifer Wortman Vaughan, and Hanna Wallach. 2020. Co-designing checklists to understand organizational challenges and opportunities around fairness in ai. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1-14.
- [86] Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability and inter-rater reliability in qualitative research: Norms and guidelines for CSCW and HCI practice. Proceedings of the ACM on human-computer interaction 3, CSCW (2019), 1-23.
- [85] Thomas Markussen and Eva Knutz. 2013. The poetics of design fiction. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces . 231-240.

| [87]   | Danaë Metaxa, Joon Sung Park, Ronald E Robertson, Karrie Karahalios, Christo Wilson, Jeff Hancock, Christian Sandvig, et al. 2021. Auditing algorithms: Understanding algorithmic systems from the outside in. Foundations and Trends® in Human-Computer Interaction 14, 4 (2021), 272-344.       |
|--------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [88]   | Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. 2021. Algorithmic impact assessments and accountability: The co-construction of impacts. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency . 735-746.         |
| [89]   | Microsoft. 2022. Microsoft Responsible AI Impact Assessment Guide. (June 2022). https://blogs.microsoft.com/wp- content/uploads/prod/sites/5/2022/06/Microsoft-RAI-Impact-Assessment-Guide.pdf                                                                                                    |
| [90]   | Microsoft. 2022. Microsoft Responsible AI Impact Assessment Template. (June 2022). https://blogs.microsoft.com/wp- content/uploads/prod/sites/5/2022/06/Microsoft-RAI-Impact-Assessment-Template.pdf                                                                                              |
| [91]   | Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency . 220-229.            |
| [92]   | Richard K Morgan. 2012. Environmental impact assessment: the state of the art. Impact assessment and project appraisal 30, 1 (2012), 5-14.                                                                                                                                                        |
| [93]   | Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf. 2021. Assembling accountability: algorithmic impact assessment for the public interest. Available at SSRN 3877437 (2021).                                                                           |
| [94]   | Priyanka Nanayakkara, Jessica Hullman, and Nicholas Diakopoulos. 2021. Unpacking the expressed consequences of AI research in broader impact statements. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society 795-806.                                                       |
| [95]   | Lisa P Nathan, Batya Friedman, Predrag Klasnja, Shaun K Kane, and Jessica K Miller. 2008. Envisioning systemic effects on persons and society throughout interactive system design. In Proceedings of the 7th ACM conference on Designing interactive systems . 1-10.                             |
| [96]   | National Academies of Sciences, Engineering, and Medicine et al. 2022. Fostering Responsible Computing Research: Foundations and Practices. (2022).                                                                                                                                               |
| [97]   | National Artificial Intelligence Research Resource Task Force. 2023. Strengthening and Democratizing the U.S. Artificial Intelligence Innovation Ecosystem. (Janurary 2023). https://www.ai.gov/wp-content/uploads/2023/01/NAIRR-TF- Final-Report-2023.pdf                                        |
| [98]   | Alexandra Olteanu, Michael Ekstrand, Carlos Castillo, and Jina Suh. 2023. Responsible AI Research Needs Impact Statements Too. arXiv preprint arXiv:2311.11776 (2023).                                                                                                                            |
| [99]   | Partnership on AI. 2021. Managing the Risks of AI Research: Six Recommendations for Responsible Publication. (2021). https://partnershiponai.org/paper/responsible-publication-recommendations/                                                                                                   |
| [100]  | OpenAI. 2022. OpenAI: Our approach to alignment research. https://openai.com/blog/our-approach-to-alignment- research/                                                                                                                                                                            |
| [101]  | Rock Yuren Pang and Katharina Reinecke. 2023. Anticipating Unintended Consequences of Technology Using Insights from Creativity Support Tools. arXiv preprint arXiv:2304.05687 (2023).                                                                                                            |
| [102]  | Rock Yuren Pang, Sebastin Santy, René Just, and Katharina Reinecke. 2024. BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1-18.                                              |
| [103]  | Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology . 1-18. |
| [104]  | Soya Park, April Yi Wang, Ban Kawas, Q Vera Liao, David Piorkowski, and Marina Danilevsky. 2021. Facilitating knowledge sharing from domain experts to data scientists for building nlp models. In 26th International Conference on Intelligent User Interfaces . 585-596.                        |
| [105]  | Samir Passi and Solon Barocas. 2019. Problem formulation and fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency . 39-48.                                                                                                                                    |
| [106]  | Samir Passi and Steven J Jackson. 2018. Trust in data science: Collaboration, translation, and accountability in corporate data science projects. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1-28.                                                                      |
| [107]  | Judith Petts. 2009. Handbook of Environmental Impact Assessment, Volume 2: Impact and Limitations . Vol. 2. John Wiley & Sons.                                                                                                                                                                    |
| [108]  | David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, and Felix Portnoy. 2021. How ai developers overcome communication challenges in a multidisciplinary team: A case study. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-25.                       |
| [109]  | Carina EA Prunkl, Carolyn Ashurst, Markus Anderljung, Helena Webb, Jan Leike, and Allan Dafoe. 2021. Institution-                                                                                                                                                                                 |

- [109] Carina EA Prunkl, Carolyn Ashurst, Markus Anderljung, Helena Webb, Jan Leike, and Allan Dafoe. 2021. Institutionalizing ethics in AI through broader impact requirements. Nature Machine Intelligence 3, 2 (2021), 104-110.

## CSCW178:34

| [110]   | Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 conference on fairness, accountability, and transparency . 33-44.   |
|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [111]   | Bogdana Rakova, Jingying Yang, Henriette Cramer, and Rumman Chowdhury. 2021. Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-23.                                                                                               |
| [112]   | Charvi Rastogi, Marco Tulio Ribeiro, Nicholas King, Harsha Nori, and Saleema Amershi. 2023. Supporting human-ai collaboration in auditing llms with llms. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society 913-926.                                                                                                                            |
| [113]   | Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. 2018. Algorithmic Impact Assessments: A Practical Framework for Public Agency. AI Now (2018).                                                                                                                                                                                                     |
| [114]   | People + AI Research. 2021. People AI Guidebook. (2021). https://pair.withgoogle.com/guidebook/                                                                                                                                                                                                                                                                         |
| [115]   | Brianna Richardson, Jean Garcia-Gathright, Samuel F Way, Jennifer Thom, and Henriette Cramer. 2021. Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-13.                                                                               |
| [116]   | Gloire Rubambiza, Phoebe Sengers, Hakim Weatherspoon, and Jen Liu. 2024. Seam Work and Simulacra of Societal Impact in Networking Research: A Critical Technical Practice Approach. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1-19.                                                                                                  |
| [117]   | Malak Sadek, Marios Constantinides, Daniele Quercia, and Céline Mougenot. 2024. Guidelines for Integrating Value Sensitive Design in Responsible AI Toolkits. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1-20.                                                                                                                        |
| [118]   | Andrew D Selbst. 2021. AN INSTITUTIONAL VIEW OF ALGORITHMIC IMPACT. Harvard Journal of Law & Technology 35, 1 (2021).                                                                                                                                                                                                                                                   |
| [119]   | Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N'Mah Yilla- Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, et al. 2023. Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society . 723-741.                     |
| [120]   | Hong Shen, Wesley H Deng, Aditi Chattopadhyay, Zhiwei Steven Wu, Xu Wang, and Haiyi Zhu. 2021. Value cards: An educational toolkit for teaching social impacts of machine learning through deliberation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency . 850-861.                                                             |
| [121]   | Hong Shen, Haojian Jin, Ángel Alexander Cabrera, Adam Perer, Haiyi Zhu, and Jason I Hong. 2020. Designing alter- native representations of confusion matrices to support non-expert public understanding of algorithm performance. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1-22.                                                          |
| [122]   | Hong Shen, Leijie Wang, Wesley H Deng, Ciell Brusse, Ronald Velgersdijk, and Haiyi Zhu. 2022. The Model Card Authoring Toolkit: Toward Community-centered, Deliberation-driven AI Design. In 2022 ACM Conference on Fairness, Accountability, and Transparency . 440-451.                                                                                               |
| [123]   | Eli Sherman and Ian Eisenberg. 2024. AI Risk Profiles: A Standards Proposal for Pre-deployment AI Risk Disclosures. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 23047-23052.                                                                                                                                                            |
| [124]   | Ben Shneiderman. 2020. Bridging the gap between ethics and practice: guidelines for reliable, safe, and trustworthy human-centered AI systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 4 (2020), 1-31.                                                                                                                                           |
| [125]   | George Silberman and Katherine L Kahn. 2011. Burdens on research imposed by institutional review boards: the state of the evidence and its implications for regulatory reform. The Milbank Quarterly 89, 4 (2011), 599-627.                                                                                                                                             |
| [126]   | CEstelle Smith, BowenYu, Anjali Srivastava, Aaron Halfaker, Loren Terveen, and Haiyi Zhu. 2020. Keeping community in the loop: Understanding wikipedia stakeholder values for machine learning-based systems. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1-14.                                                                   |
| [127]   | Jessie J Smith, Saleema Amershi, Solon Barocas, Hanna Wallach, and Jennifer Wortman Vaughan. 2022. REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In 2022 ACM Conference on Fairness, Accountability, and Transparency . 587-597.                                                                                          |
| [128]   | Data & Society. 2023. Data & Society Announces the Launch of its Algorithmic Impact Methods Lab. https: //datasociety.net/algorithmic-impact-methods-lab/                                                                                                                                                                                                               |
| [129]   | Susan Leigh Star and James R Griesemer. 1989. Institutional ecology,translations' and boundary objects: Amateurs and professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. Social studies of science 19, 3 (1989), 387-420.                                                                                                                                |
| [130]   | Susan Leigh Star and Anselm Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work 8, 1-2 (1999), 9-30.                                                                                                                                                                                      |
| [131]   | Stephanie B Steinhardt and Steven J Jackson. 2015. Anticipation work: Cultivating vision in collective practice. In                                                                                                                                                                                                                                                     |

- [131] Stephanie B Steinhardt and Steven J Jackson. 2015. Anticipation work: Cultivating vision in collective practice. In Proceedings of the 18th ACM conference on computer supported cooperative work &amp; social computing . 443-453.

| [132]   | Ivan Stelmakh, Charvi Rastogi, Nihar B Shah, Aarti Singh, and Hal Daumé III. 2023. A large scale randomized controlled trial on herding in peer-review discussions. Plos one 18, 7 (2023), e0287443.                                                                                                                                                                                                                                                                                                                                                                                      |
|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [133]   | Ivan Stelmakh, Nihar B Shah, Aarti Singh, and Hal Daumé III. 2021. Prior and prejudice: The novice reviewers' bias against resubmissions in conference peer review. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-17.                                                                                                                                                                                                                                                                                                                                           |
| [134]   | Christian Sturm, Alice Oh, Sebastian Linxen, Jose Abdelnour Nocera, Susan Dray, and Katharina Reinecke. 2015. How WEIRD is HCI? Extending HCI principles to other countries and cultures. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems . 2425-2428.                                                                                                                                                                                                                                                                          |
| [135]   | Hariharan Subramonyam, Jane Im, Colleen Seifert, and Eytan Adar. 2022. Solving separation-of-concerns problems in collaborative design of human-AI systems through leaky abstractions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems . 1-21.                                                                                                                                                                                                                                                                                                            |
| [136]   | Mohammad Tahaei, Alisa Frik, and Kami Vaniea. 2021. Privacy champions in software teams: understanding their motivations, strategies, and challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-15.                                                                                                                                                                                                                                                                                                                                             |
| [137]   | Jacqueline Rita Tedaldi, Nora Götzmann, Elin Wrzoncki, Roya Høvsgaard, Tulika Bansal, and Cathrine Bloch Veiberg. 2016. Human Rights Impact Assessment Guidance and Toolbox. (2016).                                                                                                                                                                                                                                                                                                                                                                                                      |
| [138]   | The Ada Lovelace Insitute. 2022. Algorithmic impact assessment: a case study in healthcare. (Feb 2022). https: //www.adalovelaceinstitute.org/report/algorithmic-impact-assessment-case-study-healthcare/                                                                                                                                                                                                                                                                                                                                                                                 |
| [139]   | The Ada Lovelace Insitute. 2022. Pioneering framework for assessing the impact of medical AI set to be trialled by NHS in world-first pilot. (Feb 2022). https://www.adalovelaceinstitute.org/press-release/algorithmic-impact- assessment-nhs/                                                                                                                                                                                                                                                                                                                                           |
| [140]   | The White House. 2023. National Artificial Intelligence Research Resource Task Force Releases Final Report. (Janu- rary 2023). https://www.whitehouse.gov/ostp/news-updates/2023/01/24/national-artificial-intelligence-research- resource-task-force-releases-final-report/                                                                                                                                                                                                                                                                                                              |
| [141]   | Laurence Vigneau, Michael Humphreys, and Jeremy Moon. 2015. How do firms comply with international sustain- ability standards? Processes and consequences of adopting the global reporting initiative. Journal of business ethics 131 (2015), 469-486.                                                                                                                                                                                                                                                                                                                                    |
| [142]   | Ari Ezra Waldman. 2021. Industry unbound: The inside story of privacy, data, and corporate power . Cambridge University Press.                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| [143]   | Kent Walker and Marian Croak. 2021. An update on our progress in responsible AI innovation. (2021). https: //blog.google/technology/ai/update-our-progress-responsible-ai-innovation/                                                                                                                                                                                                                                                                                                                                                                                                     |
| [144]   | Hanna Wallach. 2021. Navigating the Broader Impacts of Machine Learning Research. https: //hannawallach.medium.com/navigating-the-broader-impacts-of-machine-learning-research-f2d72a37a5b? source=friends_link&sk=3b9e2ab57cb142e00cf0b55fe5bc1df3                                                                                                                                                                                                                                                                                                                                       |
| [145]   | Qiaosi Wang, Michael Adam Madaio, Shivani Kapania, Shaun Kane, Michael Terry, Lauren Wilcox, et al. 2023. Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. (2023).                                                                                                                                                                                                                                                                                                                                                                                 |
| [146]   | Zijie J. Wang, Chinmay Kulkarni, Lauren Wilcox, Michael Terry, and Michael Madaio. 2024. Farsight: Fostering Responsible AI Awareness During AI Application Prototyping. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1-40.                                                                                                                                                                                                                                                                                                                               |
| [147]   | Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John F. J. Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sande Minnich Brown, Zachary Kenton, William T. Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William S. Isaac, Julia Haas, Sean Legassick, Geoffrey Irving, and Iason Gabriel. 2022. Taxonomy of Risks posed by Language Models. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022). https://api.semanticscholar.org/CorpusID:249872629 |
| [148]   | David Gray Widder and Dawn Nafus. 2023. Dislocated accountabilities in the 'AI supply chain': Modularity and developers' notions of responsibility. Big Data & Society 10, 1 (2023), 20539517231177620.                                                                                                                                                                                                                                                                                                                                                                                   |
| [149]   | David Gray Widder, Sarah West, and Meredith Whittaker. 2023. Open (for Business): big tech, concentrated power, and the political economy of open AI. Concentrated Power, and the Political Economy of Open AI (August 17, 2023) (2023).                                                                                                                                                                                                                                                                                                                                                  |
| [150]   | David Gray Widder, Derrick Zhen, Laura Dabbish, and James Herbsleb. 2023. It's about power: What ethical concerns do software engineers have, and what do they (feel they can) do about them?. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency . 467-479.                                                                                                                                                                                                                                                                                         |
| [151]   | Erin D Williams and Michele Schoonmaker. 2005. Federal protection for human research subjects: an analysis of the common rule and its interactions with FDA regulations and the HIPAA Privacy Rule. Congressional Information Service, Library of Congress.                                                                                                                                                                                                                                                                                                                               |
| [152]   | Richmond Y Wong. 2021. Tactics of Soft Resistance in User Experience Professionals' Values Work. Proceedings of the                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |

- [152] Richmond Y Wong. 2021. Tactics of Soft Resistance in User Experience Professionals' Values Work. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1-28.

[153] Richmond Y. Wong, Michael A. Madaio, and Nick Merrill. 2022. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. Proceedings of the ACM on Human-Computer Interaction 7 (2022), 1 - 27. https://api.semanticscholar. org/CorpusID:246904325

- [155] Qian Yang, Justin Cranshaw, Saleema Amershi, Shamsi T Iqbal, and Jaime Teevan. 2019. Sketching nlp: A case study of exploring the right things to design with language intelligence. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems . 1-12.

[154] Richmond Y Wong and Tonya Nguyen. 2021. Timelines: A world-building activity for values advocacy. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-15.

- [156] Nur Yildirim, Mahima Pushkarna, Nitesh Goyal, Martin Wattenberg, and Fernanda Viégas. 2023. Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People+ AI Guidebook. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1-13.
- [158] Haiyi Zhu, Bowen Yu, Aaron Halfaker, and Loren Terveen. 2018. Value-sensitive algorithm design: Method, case study, and lessons. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1-23.
- [157] Amy X Zhang, Michael Muller, and Dakuo Wang. 2020. How do data science workers collaborate? roles, workflows, and tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1-23.

## A APPENDIX

## A.1 Evolution of the SIA Template

In the supplementary material, we include the initial prototype of the SIA template ( V1, used with participants P01-P05 ) and the version used in the user study (V7). Here we briefly describe the major changes that were made in each version of the template between V1 and V7. In every iteration, we additionally made wording and formatting changes throughout the template based on participants' feedback.

## V2 (used with participants P06-P11) :

- · Added more examples in the scaffolding provided in the 'Potential Negative Societal Impact' section to cover a more diverse range of computing research topics.
- · Rephrased the questions in the first two sections ('Research Project Background' and 'Intended Users and Applications') to make them more directly applicable to research projects instead of product and service development.
- · Moved the 'Misuse (unintentional, accidental misuse)' section right after the 'Known Limitation' section within the 'Potential Negative Societal Impact' section since participants P01-P05 often naturally thought about misuse after considering limitations.

## V3 (used with participants P12-P15) :

- · Emphasized the non-linear nature of brainstorming stakeholders by reminding researchers to revisit the stakeholders section if they think of more stakeholders later.
- · Added a sentence to highlight the differences between assessing societal impact and preventing risks to human subjects, as participants often confused these concepts.
- · Added examples to the scaffolding in the 'Intended Positive Societal Impact' section, as participants often needed help in this area.

## V4 (used with participants P16-P18 and E01-E03) :

- · Further emphasized the heterogeneous, non-linear thinking processes of impact assessment by adding notes suggesting researchers complete the sections of the template non-sequentially, in their preferred order.
- · Added a 3-minute introduction video as onboarding material, as described in Section 7.
- · Added a section titled "Why would you want to use this template" at the top as motivation.

## V5 (used with participants P19-P22 and E04-E10) :

- · Reorganized the categories within the stakeholders section to distinguish industry, civil society, and governments, with a concise description of each to scaffold brainstorming, as suggested by impact assessment experts.
- · Added more categories of limitations in the 'Known Limitations' section to cover potential limitations of design and qualitative industry computing research.
- · Added illustrations (e.g., Figures 4 and 5) throughout the template to help researchers better understand the organization of the template and key concepts.
- · Provided a keyword summary for all scaffolding examples throughout the template for more efficient access.

## V6 (used with participants P23-P25 and E11-E16) :

- · Added instructions to remind researchers to document different versions of their project, their thinking process, and the challenges they encountered. Encouraged transparency about struggles throughout the template.
- · Highlighted the limitations of the impact assessment template, noting that it is not meant to replace other existing mechanisms or compliance processes in the company, within the onboarding video and instructions.
- · Incorporated the sentence 'While thinking about the scenarios, consider which group(s) of stakeholders you identified in Section 3 might be harmed. In what ways?' into all questions in the 'Potential Impact' section to better facilitate thinking about negative impact and stakeholders together.
- · Reorganized the strategies for addressing potential negative societal impact based on feedback from impact assessment experts.

## V7 (used with participants T01-T15) :

- · Based on feedback from impact assessment experts, expanded the 'Planning for Mitigations' section to guide researchers in determining the seriousness of the risks posed by potential negative societal impacts and how to prioritize them. Added scaffolding questions to prompt researchers to consider the likelihood, magnitude, concentration, and time horizon of the potential negative societal impact.
- · Streamlined the template to ensure the format and structure were ready for the user study.
- · Further improved the clarity of the questions and the transitions between all sections.
Wesley Hanwen Deng Solon Barocas Jennifer Wortman Vaughan
Supporting Industry Computing Researchers in Assessing, Articulating, and Addressing the Potential Negative Societal Impact of Their Work
Recent years have witnessed increasing calls for computing researchers to
grapple with the societal impacts of their work. Tools such as impact
assessments have gained prominence as a method to uncover potential impacts,
and a number of publication venues now encourage authors to include an impact
statement in their submissions. Despite this push, little is known about the
way researchers assess, articulate, and address the potential negative societal
impact of their work -- especially in industry settings, where research
outcomes are often quickly integrated into products. In addition, while there
are nascent efforts to support researchers in this task, there remains a dearth
of empirically-informed tools and processes. Through interviews with 25
industry computing researchers across different companies and research areas,
we first identify four key factors that influence how they grapple with (or
choose not to grapple with) the societal impact of their research. To develop
an effective impact assessment template tailored to industry computing
researchers' needs, we conduct an iterative co-design process with these 25
industry researchers and an additional 16 researchers and practitioners with
prior experience and expertise in reviewing and developing impact assessments
or broad responsible computing practices. Through the co-design process, we
develop 10 design considerations to facilitate the effective design,
development, and adaptation of an impact assessment template for use in
industry research settings and beyond, as well as our own ``Societal Impact
Assessment'' template with concrete scaffolds. We explore the effectiveness of
this template through a user study with 15 industry research interns, revealing
both its strengths and limitations. Finally, we discuss the implications for
future researchers and organizations seeking to foster more responsible
research practices.