## /volume\_up Everything We Hear: Towards Tackling Misinformation in Podcasts

## Sachin Pathiyan Cherumanal

## Ujwal Gadiraju

## Damiano Spina

RMIT University Melbourne, Australia s3874326@student.rmit.edu.au

Delft University of Technology Delft, The Netherlands u.k.gadiraju@tudelft.nl

## ABSTRACT

Advances in generative AI, the proliferation of large multimodal models (LMMs), and democratized open access to these technologies have direct implications for the production and diffusion of misinformation. In this prequel, we address tackling misinformation in the unique and increasingly popular context of podcasts. The rise of podcasts as a popular medium for disseminating information across diverse topics necessitates a proactive strategy to combat the spread of misinformation. Inspired by the proven effectiveness of auditory alerts in contexts like collision alerts for drivers and error pings in mobile phones, our work envisions the application of auditory alerts as an effective tool to tackle misinformation in podcasts. We propose the integration of suitable auditory alerts to notify listeners of potential misinformation within the podcasts they are listening to, in real-time and without hampering listening experiences. We identify several opportunities and challenges in this path and aim to provoke novel conversations around instruments, methods, and measures to tackle misinformation in podcasts.

RMIT University Melbourne, Australia damiano.spina@rmit.edu.au may sometimes unintentionally spread misinformation. One such example is when Joe Rogan's influential podcast hosted infamous physician Dr. Robert Malone promoting COVID-19 conspiracy theories (Refer [7]). As a result, such podcasts may end up amplifying inaccurate or biased information to their listeners, which can negatively impact society. The democratization of podcasts and the lack of editorial oversight, fact-checking, or accountability mechanisms in the podcasting ecosystem may contribute to the spread of misinformation. Furthermore, podcast users are loyal listeners and this only exacerbates the issue of misinformation unless users are made aware [12]. This makes the problem of tackling misinformation still an open challenge in audio-based information access systems such as spoken conversational search (where communication between a user and system occurs verbally through audio) and podcasts.

## CCS CONCEPTS

## · Human-centered computing → Auditory feedback .

## KEYWORDS

Auditory Interventions; Podcasts; Misinformation

## ACMReference Format:

Sachin Pathiyan Cherumanal, Ujwal Gadiraju, and Damiano Spina. 2024. /volume\_up Everything We Hear: Towards Tackling Misinformation in Podcasts. In International Conference On Multimodal Interaction (ICMI '24), November 4-8, 2024, San José, Costa Rica. ACM, New York, NY, USA, 6 pages. https: //doi.org/10.1145/3678957.3678959

## 1 INTRODUCTION

Podcasts have shown to have a growing listenership over the recent years [31, 67]. As of 2024, there are 464.7 million global podcast listeners. This number is predicted to reach 504.9 million by 2024 [61]. In today's world, podcasts have become a major source of information on a variety of topics such as politics, current affairs, culture, health, and controversial issues. Surveys have shown that people listen to podcasts while completing mundane tasks such as commuting, and household chores [68]. It is important to note that podcasts

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00292v1-figure-9435fa8e-61a9-4b8f-a583-c31144ed8085.jpg)

This work is licensed under a Creative Commons AttributionNoDerivs International 4.0 License.

ICMI '24, November 4-8, 2024, San José, Costa Rica © 2024 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-0462-8/24/11 https://doi.org/10.1145/3678957.3678959

Fact-checking is a long-standing practice in journalism that publishes an evidence-based analysis (i.e., reports) of the accuracy of a political claim, news report, or other public text [23]. Previous studies [55] show it's effective against online misinformation. However, fact-check reports can be complex and require detailed reading on a screen. Recent research has explored different presentation strategies [28, 35]. Our focus is on exploring techniques to alert users to the existence of fact-checking reports verifying specific claims mentioned in podcasts. In a visual presentation, this can be accomplished with a hyperlink, citation, or even warning labels [40, 53]. However, in audio-only scenarios like podcasts, where interruptions are not viable (to sustain the train of thought of listeners), finding ways to inform users about potential misinformation and the availability of associated fact-checking reports is crucial. In our work, the goal of these alerts/interventions is to signal potential misinformation and promote critical thinking.

Auditory interventions have been shown to act as a vital corollary to visual warnings [15] and cues can be useful especially when the user is engaged in other tasks as is the case with most podcast users [68]. Henceforth, in our paper, we discuss the use of auditory intervention techniques to alert podcast listeners about any misinformation contained in certain parts of the podcast. This alert can also be seen as a warning. Such auditory interventions are not a new concept and have been a part of our everyday lives for years. For example, collision warnings in vehicles [24], mobile notifications, or auditory cues (i.e., interventions) on websites [4, 48]. However, utilizing auditory interventions is not straightforward.

We propose a novel approach to tackle misinformation in podcasts by harnessing auditory interventions to alert users of potential inaccuracies within snippets of the podcast. We identify the research and empirical gap, discuss open challenges and opportunities, and shed light on potential empirical studies, instruments, and data required to address the current gaps. We hope that this prequel

can inspire researchers and practitioners to tackle this important societal problem of misinformation in podcasts.

## 2 BACKGROUND

## 2.1 Misinformation, Fact-Checking, and Podcasts

With the rapid digitization of society, users frequently encounter misinformation, causing harm to life, injury, income, business, emotions, trust, reputation, safety, privacy, decision-making, and more [37, 69]. Misinformation has been shown to be multi-modal [6] and exists on various platforms like YouTube [29], X (Twitter) [21], search engines [43, 56, 64], and podcasts [31]. Numerous techniques (manual and automated) and tools [51] have been proposed to measure, detect, and mitigate misinformation. However, most work focuses on screen-based interfaces, while addressing misinformation in audio-based formats like podcasts and spoken conversational searches is unexplored [30, 33]. Fact-checking reports (i.e., manual detection) validated by professionals have been popular method in tackling misinformation, however, such interventions need to be disseminated on the internet wider and faster than fake news to have an impact [37]. It is important to note that our work focuses on presenting interventions for misinformation rather than automatically detecting them. In a text-based interface, a fact-checking report can be presented with a hyperlink or citation. However, in audio-only scenarios like podcasts and spoken conversational search, where speech-based interruptions are not viable (to sustain the train of thought of listeners and create good listening experiences), informing users about potential misinformation and availability of associated fact-check reports is crucial but challenging. In this paper, we focus on podcasts, where the interaction is one-way i.e., unlike spoken conversational systems which are interactive. However, the format of a podcast can be quite complex and vary in presentation styles, involving multiple speakers to just a single speaker, i.e., a monologue [8]. Advancing our empirical understanding would require considering monologue-style podcasts distinctly from multi-speaker settings [65]. It is stated that training and warning the users are the two main strategies to improve deception detection, as they lead to improved deception detection accuracy [11, 25, 58]. According to Wogalter [72], a warning is characterized as a communication medium or a tool that informs about hazards. We can relate this to our context and, misinformation/deceptive content can be thought of as analogous to hazards. There has been empirical evidence indicating that a warning improves individuals' detection skills [5, 73]. Furthermore, research has also shown that up-front warnings can reduce later reliance on misinformation [62]. However, these studies focused on visual warnings in websites. While one can argue that visual warnings [32] help in raising a user's suspicion towards deception, they fail to maintain the user's attention long enough during a podcast. As previous studies have shown attention maintenance is necessary for the receiver to extract the necessary information from the content as well as the warning [9]. Auditory warnings/interventions can be beneficial (especially in an audio channel like a podcast) because they leverage people's 'always-on' hearing ability. In this work, we highlight the potential for using auditory interventions (e.g., an error ping/a chime on your mobile phone) in podcasts. This raises the following questions: (i) Can auditory warnings included in podcasts assist users in recognizing misinformation? (ii) What kind of auditory warnings are most salient and consistently perceived by different users? (iii) What is the general impact of various auditory signals on users, and how does it impact the user's comprehension of the podcast content, interpretation of the signals, and general listening experience?

Sections 2.2 and 2.3 review literature on auditory interventions and experimental methodologies for their impact in podcasts, respectively.

## 2.2 Auditory Interventions

From the starting-up chime of computers to the error pings and mail notifications, auditory interventions have been an integral part of human-computer interaction [10, 57]. Such interventions are also popularly used in mission-critical technologies in the automotive industry to warn the driver of a collision [13, 24], in our mobile phones to notify the users of an incoming email [19], or as cues to indicate the click of a button [46]. These auditory warnings can be broadly categorized into speech-based and non-speech-based. It is worth noting that the most recent research on audio interventions (also known as nudges) [22] explores six different intervention techniques that can be categorized into three groups: (i) linguistic, (ii) natural voice emphasis, and (iii) artificial voice emphasis. It is important to mention that these interventions are primarily speech-based. Although speech is the most semantically rich acoustic medium, it also has a few shortcomings, one of which is privacy concerns, making it the least preferred option for some types of public notifications or reminders [3, 19, 52]. On the contrary, non-speech-based warnings have shown more confidentiality, speech independence, and wider applicability in different countries, languages, and dialects [36], making it the most suited for an audio-only context such as podcasts. Non-speech warnings have shown better user task performance [44] and were shown to be preferred over speechbased ones for longer audio content, as the latter can interfere with concurrent speech communication. Recent work by Nees and Liebman [49] compares such non-speech-based warnings and are classified into three categories[1] (i) auditory icons - sounds that can be easily attributed to objects or events generating sounds in everyday situations and (ii) earcons [66] - abstract sounds with no ecological relationship to their referent (target object or event); and (iii) spearcons (Speech-Based Earcons) [71]. Although there have been previous studies comparing auditory icons, earcons, and spearcons, there has been no research on using auditory icons to alert users to misinformation. We propose focusing efforts on auditory icons because previous research suggests they are easier to learn compared to earcons [14], and spearcons may interfere with podcast audio content. These auditory icons are further classified into three types based on their relationship with the referent [20]. 1 (i) Symbolic: the relation between the sound(s) of the auditory icon and the referent is essentially arbitrary; (ii) Iconic or Nomic: the sound of the auditory icon is related to the physical source of the referent; and (iii) Metaphorical: where the relation between sounds in the auditory icon and the referent is not completely arbitrary, yet also not fully dependent on physical causation. The arbitrariness is based on some similarities between the sound and the referent.

1 Here, by referent we mean the target phrase that involves misinformation.

Figure 1: Illustrating the different positioning of auditory icons. The red horizontal line represents a section of the podcast that contains misinformation, and the red flag represents the position of the auditory icon. (i) Start of the snippet (S), (ii) End of the snippet (E), (iii) the Start and End of the snippet (S&amp;E) (iv) Concurrent (C), where the auditory icon would be played concurrently with the snippet that has misinformation and finally (v) No Warning, where no auditory icons will be placed. Note that the red horizontal line and the red flag are for demonstration purposes only, these are not to be displayed to the user. The small blue ellipse denotes the current position on the seek bar.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00292v1-figure-d7d0c21d-2fa7-4d24-ae41-a41a977f9009.jpg)

Gaver [20] argued that with regard to his classification, in general, iconic/nomic mappings are more powerful than symbolic and metaphorical mappings, because iconic/nomic mappings show a direct relation between the auditory icon and the referent's physical source. This raises the following questions: (i) Which type of auditory icon would be the most effective design choice to implement as a misinformation alert in audio-only content such as podcasts (i.e., iconic/nomic or metaphorical)? 2 and (ii) How do we place the auditory icons in a podcast w.r.t. the snippet that contains misinformation? Figure 1 shows some of the alternatives.

## 2.3 Human Factors in Auditory Warnings and Misinformation

Warnings can be transmitted primarily along two dimensions visual (e.g., text, pictures, or both) and audio ( e.g., chimes, pings, voice). A user's response/perception to a warning can vary based on the channel it is transmitted. In this section, we review previous work on human factors in auditory warnings, relevant to our context (i.e., Podcasts). While one of the main advantages of auditory warnings is that it is generally guaranteed that the message will impinge on the receptors in the ear [9] making sure the users hear it. However, recent studies have also shown auditory feedback may affect a users comprehension [ ? ]. This warrants a listening comprehension task in future studies that look at auditory warnings in podcasts. This task will involve participants answering questions about the content of the podcast they listened to, as suggested by Pennycook et al. [54]. While we focus on the content of the podcast, it is also imperative we understand, the effect auditory warnings have on users. This raises further questions, such as the following: Can users conceptually map the auditory warnings to misinformation? or do users need to be trained to map these warnings to misinformation? Moreover, a recent work shows auditory warnings

2 We exempt the symbolic auditory icon from the study as these can be thought of as reserved non-speech sounds that may have already been conceptually mapped to a specific function of the society (e.g., a police siren).

may affect working memory [36] , warranting study of their longterm psychological effects.Recent work has also shown that the consumption and interpretation of misinformation can be primarily attributed to cognitive biases like Confirmation Bias 3 , Believability [17] and topic familiarity[2].We are uncertain how pre-existing beliefs affect the effectiveness of auditory warnings. This leads us to our next question: how do user's pre-existing beliefs/cognitive biases impact the effectiveness and perception of such auditory warnings?

In summary, we discussed the use of auditory interventions to address misinformation in podcasts. We proposed focusing on monologue-style podcasts and examining how auditory warnings impact user comprehension and the ability to detect misinformation. We then discussed the different dimensions (i.e., types and placement) of auditory warnings, including their effects on user comprehension, and the potential influence of pre-existing beliefs or cognitive biases. In light of this background knowledge, we put forth the following open challenges:

- (1) Can auditory warnings included in podcasts assist users in recognizing misinformation?
- (2) What kinds of auditory warnings are more effective in helping users be aware of misinformation? Do they impact the user's comprehension of the information? Which ones are most salient and consistently perceived by different users? What is their impact on user engagement?
- (3) How does the position of auditory warnings in a podcast w.r.t the snippet impact the user's comprehensibility and misinformation detection?
- (4) Without any prior training, how do users perceive these auditory warnings? I.e., Can users conceptually map the auditory warnings to the presence of misinformation?
- (5) Does users' pre-existing beliefs/cognitive biases impact the effectiveness and perception of such auditory warnings?

## 3 PROPOSED APPROACH

We aim to foster discussions among researchers and practitioners by outlining open research challenges and emphasizing future steps in exploring auditory interventions against misinformation in podcasts. However, conducting experiments to study their effectiveness requires controlling multiple variables (see Table 1). Moreover, such studies require controlling for potential effects from podcast content characteristics, such as topic complexity, political or societal stance of speakers, language comprehensibility, and podcast length. In the next section, we propose controlling these variables by simulating a podcast experience using synthesized content and detailing the approach researchers and practitioners could adopt.

## 3.1 Data and Instruments

This section discusses the topics used in the experiments, the process used to generate podcast content, and the auditory icons used. Podcast Topics and Content. We pick two topics with different levels of topical difficulty and potential knowledge gain [18] Altitude Sickness ( 𝑇 1) and Carpenter Bees ( 𝑇 2). To control for interaction effects between characteristics of the podcast speaker and users'

3 Confirmation bias is a cognitive bias that causes a consumer of new information to believe it is true if it aligns with their current beliefs or ideology[34]

Table 1: Experiment variables to be considered while studying the effectiveness of auditory warnings along with the rationale.

| Variable                              | Rationale                                                                                                         |
|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|
| 1. Auditory Icon Type                 | Section 3.1                                                                                                       |
| 2. Position of Auditory Icon          | Section 2.2                                                                                                       |
| 3. Comprehensibility                  | Auditory feedback affects website usability [50], but their impact on podcast listeners is unclear.               |
| 4. Misinformation Recall              | The user can be asked to identify the snippet they felt contained misinformation.                                 |
| 5. Podcast Topic                      | Gadiraju et al. [18]; Gwizdka and Spence [27]; Li and Belkin [38]                                                 |
| 6. User Engagement                    | Noh et al. [50]                                                                                                   |
| 7. Icon recall rate                   | Testing the user's recall of the auditory icons.                                                                  |
| 8. Task Load                          | ul Fazal et al. [70];Sanderson et al. [60]                                                                        |
| 9. PESQ score                         | The score is an Automatic measure of sound quality Loizou [39];Gohsen et al. [22]                                 |
| 10. Gender and Age                    | Martínez-Costa et al. [41]                                                                                        |
| 11. Language Proficiency              | Muda et al. [47]                                                                                                  |
| 12. Education Level                   | A user's education level has been shown to have a positive impact on the recognition of fake news [74].           |
| 13. Preferred Information Consumption | A user's preferred mode of information consumption has shown to impact user engagement [59].                      |
| 14. Declared impairments              | Considering individual variations in those with brain injuries or neurological conditions [63].                   |
| 15. Speculated Meaning                | Determining users' perception of auditory icons can reveal if they can map auditory warnings to misinfor- mation. |

Table 2: Sample of a podcast content as generated by ChatGPT - GPT-3.5 on 21 November 2023 - along with SSML tags to simulate a monologue podcast.

| Topic                     | Content                                                                                                                                                                            |
|---------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Altitude Sickness ( 𝑇 1 ) | <speak> <p> Imagine you're flying in an airplane [. . . ] Sometimes, your body gets a bit sick. </p> <p> [. . . ] you can maybe take some medicine. </p> </speak>                  |
| Carpenter Bees ( 𝑇 2 )    | <speak> <p> There are special bees called carpenter bees. [. . . ] dead wood or bamboo. </p> <p> It's a bit tricky to tell [. . . ] tummies can tell the difference. </p> </speak> |

perception of misinformation, we synthesize the content of the podcast using ChatGPT for both topics using the prompt: 'Tell me about [INFORMATION NEED] using words a five-year-old would understand in a short paragraph of fewer than 200 words' (see Table 2). 4

Next, we propose manually altering the snippet content to simulate misinformation, followed by generating an audio snippet for each topic using text-to-speech tools. 5 Previous studies show that content length and comprehensibility can affect a user's information consumption behavior. Therefore, we propose controlling these factors in future experiments. For instance, by controlling the readability scores - Flesch-Kincaid ( 𝐹 ) [16] and Gunning Fog ( 𝐺 ) [26] , word count ( 𝑊𝐶 ), audio length in seconds ( 𝐿 ) across topics used in the study 𝑇 1 ( 𝐹 = 7 18 . , 𝐺 = 9 34 . , 𝑊𝐶 = 156 , 𝐿 = 47 ) and 𝑇 2 ( 𝐹 = 6 97 . , 𝐺 = 9 30 . , 𝑊𝐶 = 155 , 𝐿 = 50 . Note that there is no ) major difference between the scores of 𝑇 1 and 𝑇 2. Using synthesized podcasts also ensures that variables like sound effects (e.g.,

4 Werefer to the information needs for Altitude Sickness and Carpenter Bees as discussed by Gadiraju et al. [18].

5 To test feasibility, we used the Amazon Polly Neural Engine with English (US) language and Joanne (Female) voice.

intro/outro music, advertisements) in real-world podcasts don't interfere with the experimental variables. Furthermore, we propose the experiments (where the participant would be facing the screen throughout the experiment) to exclude warning labels or visual indicators in the podcast interface to simulate a real-world scenario where users may not be visually interacting with the podcast [68]. Auditory Icons. To instantiate an experiment, we propose to start by focusing on two sub-categories of auditory icons (i) iconic/nomic and (ii) metaphorical). Given the nature of iconic/nomic (i.e., the icon is closely related to the target) we propose to create one icon for each topic. Consequently, 𝑇 1 would have an icon that sounds like ' Whispers in the wind '. 𝑇 2 would have an icon that sounds like ' Swarm of buzzing bees with distorted voices '. Meanwhile, for metaphorical type, we propose to use only one type of icon for both the topics ' Alarm bells '.When creating these auditory icons, we emphasize keeping the loudness under 60 dB 6 to prevent hearing impairments and avoid the unpleasantness and urgency associated with higher decibel levels [42, 45].

## 4 SUMMARY

We propose a novel approach to tackle misinformation in podcasts by harnessing auditory signals to alert listeners about potential inaccuracies within snippets of the podcast. By integrating such auditory signals we aim to empower listeners with real-time awareness. We identify and synthesize several research questions entailing this context. We discuss several research gaps that warrant empirical investigation into the use of auditory signals for this purpose, and possibly in the future, the potential to personalize the experience of identifying misinformation with custom auditory icons. We believe this can inform the next steps to address the problem of tackling misinformation in audio-based content such as podcasts.

## ACKNOWLEDGMENTS

This research is partially supported by the Australian Research Council (CE200100005, DE200100064) and the TU Delft AI Initiative.

6 Audio loudness is measured in decibels (dB). Typically, levels from 0 dB to 60 dB do not cause hearing damage with repeated exposure.

- /volume\_up Everything We Hear: Towards Tackling Misinformation in Podcasts

## REFERENCES

- [1] Rafa Absar and Catherine Guastavino. 2008. Usability of Non-Speech Sounds in User Interfaces. In Proceedings of the 14th International Conference on Auditory Display (ICAD '08) . International Community for Auditory Display. http://hdl. handle.net/1853/49965
- [2] Leif Azzopardi. 2021. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (Canberra ACT, Australia) (CHIIR '21) . Association for Computing Machinery, New York, NY, USA, 27-37. https: //doi.org/10.1145/3406522.3446023
- [3] Chris Baber and Jan Noyes. 2014. Interactive Speech Technology: Human factors issues in the application of speech input/output to computers . CRC Press. https: //doi.org/10.1201/9781482272512
- [4] Jared M. Batterman, Jonathan H. Schuett, and Bruce N. Walker. 2013. Auditory Displays for Accessible Fantasy Sports. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (Bellevue, Washington) (ASSETS '13) . Association for Computing Machinery, New York, NY, USA, Article 38, 2 pages. https://doi.org/10.1145/2513383.2513397
- [5] David P. Biros, Joey F. George, and Robert W. Zmud. 2002. Inducing Sensitivity to Deception in Order to Improve Decision Making Performance: A Field Study. MIS Quarterly 26, 2 (06 2002), 119-144. https://www.proquest.com/scholarly-journals/inducing-sensitivitydeception-order-improve/docview/218132860/se-2
- [6] Danielle Caled and Mário J Silva. 2022. Digital Media and Misinformation: An Outlook on Multidisciplinary Strategies against manipulation. Journal of Computational Social Science 5, 1 (2022), 123-159. https://doi.org/10.1007/s42001021-00118-8
- [7] Kevin Matthe Caramancion. 2022. An Exploration of Mis/Disinformation in Audio Format Disseminated in Podcasts: Case Study of Spotify. In 2022 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) . 1-6. https://doi.org/10.1109/IEMTRONICS55184.2022.9795760
- [8] Ben Carterette, Rosie Jones, Gareth F. Jones, Maria Eskevich, Sravana Reddy, Ann Clifton, Yongze Yu, Jussi Karlgren, and Ian Soboroff. 2021. Podcast Metadata and Content: Episode Relevance and Attractiveness in Ad Hoc Search. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR '21) . Association for Computing Machinery, New York, NY, USA, 2247-2251. https://doi.org/10.1145/ 3404835.3463101
- [9] Vincent C. Conzola and Michael S. Wogalter. 2001. A Communication--Human Information Processing (C-HIP) Approach to Warning Effectiveness in the Workplace. Journal of Risk Research 4, 4 (2001), 309-322. https://doi.org/10.1080/ 13669870110062712
- [10] Ádám Csapó and György Wersényi. 2013. Overview of Auditory Representations in Human-Machine Interfaces. ACM Comput. Surv. 46, 2, Article 19 (dec 2013), 23 pages. https://doi.org/10.1145/2543581.2543586
- [11] Bella M. DePaulo, James J. Lindsay, Brian E. Malone, Muhlenbruck Laura, Charlton Kelly, and Cooper Harris. 2003. Cues to deception. Psychological Bulletin 129, 1 (2003), 74-118. https://doi.org/10.1037/0033-2909.129.1.74
- [12] Ute Döring, Barbara Müller, Svenja Rohr, John Ruhrmann, and Melissa Schäfer. 2022. Listen and Read: The Battle for Attention: A New Report About Key Audience Behaviour in the Age of EBooks, Audiobooks and Podcasts. Publishing Research Quarterly 38, 1 (2022), 40-52. https://doi.org/10.1007/s12109-021-098532
- [13] Ke Duan, Xuedong Yan, Xiaomeng Li, and Junyu Hang. 2023. Improving Drivers' Merging Performance in Work Zone Using an In-Vehicle Audio Warning. Transportation Research Part F: Traffic Psychology and Behaviour 95 (2023), 297-321. https://doi.org/10.1016/j.trf.2023.04.004
- [14] Judy Edworthy. 2012. Medical Audible Alarms: A Review. Journal of the American Medical Informatics Association 20, 3 (10 2012), 584-589. https://doi.org/10.1136/ amiajnl-2012-001061
- [15] Judy Edworthy and Rachael Hards. 1999. Learning Auditory Warnings: The Effects of Sound Type, Verbal Labelling and Imagery on the Identification of Alarm Sounds. International Journal of Industrial Ergonomics 24, 6 (1999), 603-618. https://doi.org/10.1016/S0169-8141(98)00066-3
- [16] Rudolph Flesch. 1948. A New Readability Yardstick. Journal of Applied Psychology 32, 3 (06 1948), 221-233. https://doi.org/10.1037/h0057532
- [17] Aaron M French, Veda C Storey, and Linda Wallace. 2023. The Impact of Cognitive Biases on the Believability of Fake News. European Journal of Information Systems (2023), 1-22. https://doi.org/10.1080/0960085X.2023.2272608
- [18] Ujwal Gadiraju, Ran Yu, Stefan Dietze, and Peter Holtz. 2018. Analyzing Knowledge Gain of Users in Informational Search Sessions on the Web. In Proceedings of the 2018 Conference on Human Information Interaction &amp; Retrieval (New Brunswick, NJ, USA) (CHIIR '18) . Association for Computing Machinery, New York, NY, USA, 2-11. https://doi.org/10.1145/3176349.3176381
- [19] Stavros Garzonis, Simon Jones, Tim Jay, and Eamonn O'Neill. 2009. Auditory Icon and Earcon Mobile Service Notifications: Intuitiveness, Learnability, Memorability and Preference. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI '09) . Association for Computing
- Machinery, New York, NY, USA, 1513-1522. https://doi.org/10.1145/1518701. 1518932
- [20] William W. Gaver. 1987. Auditory Icons: Using Sound in Computer Interfaces. SIGCHI Bull. 19, 1 (jul 1987), 74. https://doi.org/10.1145/28189.1044809
- [21] Amira Ghenai. 2017. Health Misinformation in Search and Social Media. In Proceedings of the 2017 International Conference on Digital Health (London, United Kingdom) (DH '17) . Association for Computing Machinery, New York, NY, USA, 235-236. https://doi.org/10.1145/3079452.3079483
- [22] Marcel Gohsen, Johannes Kiesel, Mariam Korashi, Jan Ehlers, and Benno Stein. 2023. Guiding Oral Conversations: How to Nudge Users Towards Asking Questions?. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval (Austin, TX, USA) (CHIIR '23) . Association for Computing Machinery, New York, NY, USA, 34-42. https://doi.org/10.1145/3576840.3578291
- [23] Lucas Graves and Michelle A Amazeen. 2019. Fact-Checking as Idea and Practice in Journalism. In Oxford research encyclopedia of communication . https://doi.org/ 10.1093/acrefore/9780190228613.013.808
- [24] Rob Gray. 2011. Looming Auditory Collision Warnings for Driving. Human Factors 53, 1 (2011), 63-74. https://doi.org/10.1177/0018720810397833
- [25] Stefano Grazioli. 2004. Where Did They Go Wrong? An Analysis of the Failure of Knowledgeable Internet Consumers to Detect Deception Over the Internet. Group Decision and Negotiation 13, 2 (03 2004), 149-172. https://doi.org/10.1023/B: GRUP.0000021839.04093.5d
- [26] Robert Gunning. 2004. Plain Language at Work Newsletter. (2004).
- [27] Jacek Gwizdka and Ian Spence. 2006. What Can Searching Behavior Tell Us About the Difficulty of Information Tasks? A Study of Web Navigation. Proceedings of the American Society for Information Science and Technology 43, 1 (2006), 1-22. https://doi.org/10.1002/meet.14504301167
- [28] Danula Hettiachchi, Kaixin Ji, Jenny Kennedy, Anthony McCosker, Flora D. Salim, Mark Sanderson, Falk Scholer, and Damiano Spina. 2023. Designing and Evaluating Presentation Strategies for Fact-Checked Content. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birmingham, United Kingdom) (CIKM '23) . Association for Computing Machinery, New York, NY, USA, 751-761. https://doi.org/10.1145/3583780.3614841
- [29] Eslam Hussein, Prerna Juneja, and Tanushree Mitra. 2020. Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1, Article 48 (may 2020), 27 pages. https://doi.org/10.1145/3392854
- [30] Kaixin Ji, Sachin Pathiyan Cherumanal, Johanne R. Trippas, Danula Hettiachchi, Flora D. Salim, Falk Scholer, and Damiano Spina. 2024. Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search. arXiv:2405.12480 https://arxiv.org/abs/2405.12480
- [31] Rosie Jones, Hamed Zamani, Markus Schedl, Ching-Wei Chen, Sravana Reddy, Ann Clifton, Jussi Karlgren, Helia Hashemi, Aasish Pappu, Zahra Nazari, Longqi Yang, Oguz Semerci, Hugues Bouchard, and Ben Carterette. 2021. Current Challenges and Future Directions in Podcast Information Access. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR '21) . Association for Computing Machinery, New York, NY, USA, 1554-1565. https://doi.org/10.1145/3404835. 3462805
- [32] Aaron S Kesselheim, John Connolly, James Rogers, and Jerry Avorn. 2015. Mandatory Disclaimers on Dietary Supplements Do Not Reliably Communicate the Intended Issues. Health Affairs 34, 3 (2015), 438-446. https://doi.org/10.1377/ hlthaff.2014.0515
- [33] Johannes Kiesel, Damiano Spina, Henning Wachsmuth, and Benno Stein. 2021. The Meant, the Said, and the Understood: Conversational Argument Search and Cognitive Biases. In Proceedings of the 3rd Conference on Conversational User Interfaces (Bilbao (online), Spain) (CUI '21) . Association for Computing Machinery, New York, NY, USA, Article 20, 5 pages. https://doi.org/10.1145/3469595.3469615
- [34] Antino Kim, Patricia L Moravec, and Alan R Dennis. 2019. Combating fake news on social media with source ratings: The effects of user and expert reputation ratings. Journal of Management Information Systems 36, 3 (2019), 931-968. https: //doi.org/10.1080/07421222.2019.1628921
- [35] Loukas Konstantinou, Ana Caraban, and Evangelos Karapanos. 2019. Combating misinformation through nudging. In Human-Computer Interaction-INTERACT 2019: 17th IFIP TC 13 International Conference, Paphos, Cyprus, September 2-6, 2019, Proceedings, Part IV 17 . Springer, 630-634. https://doi.org/10.1007/978-3030-29390-1\_51
- [36] Zhaoli Lei, Shu Ma, Hongting Li, and Zhen Yang. 2022. The Impact of Different Types of Auditory Warnings on Working Memory. Frontiers in Psychology 13 (2022), 780657. https://doi.org/10.3389/fpsyg.2022.780657
- [37] Jiexun Li and Xiaohui Chang. 2023. Combating Misinformation By Sharing The Truth: A Study on the Spread Of Fact-Checks on Social Media. Information systems frontiers 25, 4 (2023), 1479-1493. https://doi.org/10.1007/s10796-022-10296-z
- [38] Yuelin Li and Nicholas J. Belkin. 2008. A Faceted Approach to Conceptualizing Tasks in Information Seeking. Information Processing &amp; Management 44, 6 (2008), 1822-1837. https://doi.org/10.1016/j.ipm.2008.07.005
- [39] Philipos C. Loizou. 2011. Speech Quality Assessment . Springer Berlin Heidelberg, Berlin, Heidelberg, 623-654. https://doi.org/10.1007/978-3-642-19551-8\_23

- [40] Cameron Martel and David G. Rand. 2023. Misinformation Warning Labels Are Widely Effective: A Review of Warning Effects and Their Moderating Features. Current Opinion in Psychology 54 (Dec. 2023), 101710. https://doi.org/10.1016/j. copsyc.2023.101710
- [41] María-Pilar Martínez-Costa, Fernando López-Pan, Nataly Buslón, and Ramón Salaverría. 2023. Nobody-fools-me perception: Influence of Age and Education on Overconfidence About Spotting Disinformation. Journalism Practice 17, 10 (2023), 2084-2102. https://doi.org/10.1080/17512786.2022.2135128
- [42] Denis McKeown and Sarah Isherwood. 2007. Mapping Candidate Within-Vehicle Auditory Displays to Their Referents. Human Factors 49, 3 (2007), 417-428. https://doi.org/10.1518/001872007X200067
- [43] Danaë Metaxa and Nicolás Torres-Echeverry. 2017. Google's Role in Spreading Fake News and Misinformation. Available at SSRN 3062984 (2017). https://doi. org/10.2139/ssrn.3062984
- [44] Lai Jian Ming, Faieza Abdul Aziz, and Barkawi Sahari. 2008. A Study on RealTime Auditory Feedback Technique in Manipulation Task. In 2008 International Symposium on Information Technology , Vol. 1. 1-6. https://doi.org/10.1109/ITSIM. 2008.4631608
- [45] Kathryn L Momtahan. 1992. Mapping of Psychoacoustic Parameters to the Perceived Urgency of Auditory Warning Signals. (1992).
- [46] WMorrissey and M Zajicek. 2000. Can Sound Output Enhance Graphical Computer Interfaces? Contemporary Ergonomics (2000), 172-176.
- [47] Rafał Muda, Gordon Pennycook, Damian Hamerski, and Michał Białek. 2023. People Are Worse At Detecting Fake News in Their Foreign Language. Journal of Experimental Psychology: Applied (2023). https://doi.org/10.1037/xap0000475
- [48] EmmaMurphy,EndaBates, and Dónal Fitzpatrick. 2010. Designing Auditory Cues to Enhance Spoken Mathematics for Visually Impaired Users. In Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (Orlando, Florida, USA) (ASSETS '10) . Association for Computing Machinery, New York, NY, USA, 75-82. https://doi.org/10.1145/1878803.1878819
- [49] Michael A Nees and Eliana Liebman. 2023. Auditory Icons, Earcons, Spearcons, and Speech: A Systematic Review and Meta-Analysis of Brief Audio Alerts in Human-Machine Interfaces. Auditory Perception &amp; Cognition (2023), 1-30. https: //doi.org/10.1080/25742442.2023.2219201
- [50] Nur Fadhilah Mohd Noh, Suziah Sulaiman, Azelin Mohamed Noor, and Janson Luke Ong Wai Kit. 2018. A Preliminary Study on the Effect of Audio Feedback to Support Comprehension of Web Content Among Non-visual Internet Users. In Recent Trends in Information and Communication Technology . Springer International Publishing, 335-342. https://doi.org/10.1007/978-3-319-59427-9\_36
- [51] Femi Olan, Uchitha Jayawickrama, Emmanuel Ogiemwonyi Arakpogun, Jana Suklan, and Shaofeng Liu. 2022. Fake news on social media: the Impact on Society. Information Systems Frontiers (2022), 1-16. https://doi.org/10.1007/s10796-02210242-z
- [52] Roy D Patterson. 1990. Auditory Warning Sounds in the Work Environment. Philosophical Transactions of the Royal Society of London. B, Biological Sciences 327, 1241 (1990), 485-492. https://doi.org/10.1098/rstb.1990.0091
- [53] Gordon Pennycook, Adam Bear, Evan T. Collins, and David G. Rand. 2020. The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings. Management Science 66, 11 (Nov. 2020), 4944-4957. https://doi.org/10.1287/mnsc.2019.3478
- [54] Gordon Pennycook, Jonathon McPhetres, Yunhao Zhang, Jackson G Lu, and David G Rand. 2020. Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention. Psychological science 31, 7 (2020), 770-780. https://doi.org/10.1177/0956797620939054
- [55] Ethan Porter and Thomas J. Wood. 2021. The Global Effectiveness of FactChecking: Evidence from Simultaneous Experiments in Argentina, Nigeria, South Africa, and the United Kingdom. Proceedings of the National Academy of Sciences 118, 37 (Sept. 2021), e2104235118. https://doi.org/10.1073/pnas.2104235118
- [56] Ronak Pradeep, Xueguang Ma, Rodrigo Nogueira, and Jimmy Lin. 2021. Vera: Prediction Techniques for Reducing Harmful Misinformation in Consumer Health Search. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Canada) (SIGIR '21) . Association for Computing Machinery, New York, NY, USA, 2066-2070. https: //doi.org/10.1145/3404835.3463120
- [57] Agnieszka Roginska. 2013. Auditory Icons, Earcons, and Displays: Information and Expression Through Sound. The psychology of music in multimedia 339 (2013).
- [58] Bjorn Ross, Anna Jung, Jennifer Heisel, and Stefan Stieglitz. 2018. Fake News on Social Media: The (In)Effectiveness of Warning Messages. In Proceedings of the 39th International Conference on Information Systems (ICIS 2018) . Association for Information Systems, United States. https://icis2018.aisconferences.org/
- [59] Christine Rzepka, Benedikt Berger, and Thomas Hess. 2022. Voice assistant vs. Chatbot-Examining The Fit Between Conversational Agents' Interaction Modalities and Information Search Tasks. Information Systems Frontiers 24, 3 (2022), 839-856. https://doi.org/10.1007/s10796-021-10226-5
- [60] Jasmyne A Sanderson, Vanessa Bowden, Briony Swire-Thompson, Stephan Lewandowsky, and Ullrich KH Ecker. 2022. Listening to Misinformation While Driving: Cognitive Load and the Effectiveness of (Repeated) Corrections. Journal of Applied Research in Memory and Cognition (2022). https://doi.org/10.1037/ mac0000057
- [61] Rohit Shewale. 2024. Podcast Statistics For 2024 . Accessed on July 26, 2024.
- [62] Jana Siebert and Johannes Ulrich Siebert. 2023. Effective Mitigation of the Belief Perseverance Bias After the Retraction of Misinformation: Awareness Training and Counter-Speech. Plos one 18, 3 (2023), e0282202. https://doi.org/10.1371/ journal.pone.0282202
- [63] Laurianne Sitbon, Gerd Berget, and Margot Brereton. 2023. Perspectives of Neurodiverse Participants in Interactive Information Retrieval. Foundations and Trends® in Information Retrieval 17, 2 (2023), 124-243. https://doi.org/10.1561/ 1500000086
- [64] Qiurong Song and Jiepu Jiang. 2022. How Misinformation Density Affects Health Information Search. In Proceedings of the ACM Web Conference 2022 (Virtual Event, Lyon, France) (WWW'22) . Association for Computing Machinery, New York, NY, USA, 2668-2677. https://doi.org/10.1145/3485447.3512141
- [65] Damiano Spina, Johanne R. Trippas, Lawrence Cavedon, and Mark Sanderson. 2015. SpeakerLDA: Discovering Topics in Transcribed Multi-Speaker Audio Contents. In Proceedings of the Third Edition Workshop on Speech, Language &amp; Audio in Multimedia (Brisbane, Australia) (SLAM '15) . Association for Computing Machinery, New York, NY, USA, 7-10. https://doi.org/10.1145/2802558.2814649
- [66] Ratan Bahadur Thapa, Mexhid Ferati, and G. Anthony Giannoumis. 2017. Using Non-Speech Sounds to Increase Web Image Accessibility for Screen-Reader Users. In Proceedings of the 35th ACM International Conference on the Design of Communication (Halifax, Nova Scotia, Canada) (SIGDOC '17) . Association for Computing Machinery, New York, NY, USA, Article 19, 9 pages. https: //doi.org/10.1145/3121113.3121231
- [67] Mi Tian, Claudia Hauff, and Praveen Chandar. 2022. On the Challenges of Podcast Search at Spotify. In Proceedings of the 31st ACM International Conference on Information &amp; Knowledge Management (Atlanta, GA, USA) (CIKM '22) . Association for Computing Machinery, New York, NY, USA, 5098-5099. https://doi.org/10.1145/3511808.3557518
- [68] Stephanie J Tobin and Rosanna E Guadagno. 2022. Why People Listen: Motivations and Outcomes of Podcast Listening. Plos one 17, 4 (2022), e0265806. https://doi.org/10.1371/journal.pone.0265806
- [69] Thi Tran, Rohit Valecha, Paul Rad, and H. Raghav Rao. 2020. An Investigation of Misinformation Harms Related to Social Media During Humanitarian Crises. In Secure Knowledge Management In Artificial Intelligence Era , Sanjay K. Sahay, Nihita Goel, Vishwas Patil, and Murtuza Jadliwala (Eds.). Springer Singapore, Singapore, 167-181. https://doi.org/10.1007/978-981-15-3817-9\_10
- [70] Muhammad Abu ul Fazal, Sam Ferguson, and Zafar Saeed. 2021. Investigating Cognitive Workload in Concurrent Speech-Based Information Communication. International Journal of Human-Computer Studies (2021). https: //api.semanticscholar.org/CorpusID:240143761
- [71] Bruce N Walker, Jeffrey Lindsay, Amanda Nance, Yoko Nakano, Dianne K Palladino, Tilman Dingler, and Myounghoon Jeon. 2013. Spearcons (Speech-Based Earcons) Improve Navigation Performance in Advanced Auditory Menus. Human Factors 55, 1 (2013), 157-182. https://doi.org/10.1177/0018720812450587
- [72] Michael S Wogalter. 2006. Purposes and Scope of Warnings. Handbook of warnings 864 (2006).
- [73] Bo Zhang, Mu Wu, Hyunjin Kang, Eun Go, and S. Shyam Sundar. 2014. Effects of Security Warnings and Instant Gratification Cues on Attitudes toward Mobile Websites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14) . Association for Computing Machinery, New York, NY, USA, 111-114. https://doi.org/10.1145/2556288.2557347
- [74] Aljaž Zrnec, Marko Poženel, and Dejan Lavbič. 2022. Users' Ability to Perceive Misinformation: An Information Quality Assessment Approach. Information Processing &amp; Management 59, 1 (2022), 102739. https://doi.org/10.1016/j.ipm. 2021.102739
Sachin Pathiyan Cherumanal Ujwal Gadiraju Damiano Spina
Everything We Hear: Towards Tackling Misinformation in Podcasts
Advances in generative AI, the proliferation of large multimodal models
(LMMs), and democratized open access to these technologies have direct
implications for the production and diffusion of misinformation. In this
prequel, we address tackling misinformation in the unique and increasingly
popular context of podcasts. The rise of podcasts as a popular medium for
disseminating information across diverse topics necessitates a proactive
strategy to combat the spread of misinformation. Inspired by the proven
effectiveness of \textit{auditory alerts} in contexts like collision alerts for
drivers and error pings in mobile phones, our work envisions the application of
auditory alerts as an effective tool to tackle misinformation in podcasts. We
propose the integration of suitable auditory alerts to notify listeners of
potential misinformation within the podcasts they are listening to, in
real-time and without hampering listening experiences. We identify several
opportunities and challenges in this path and aim to provoke novel
conversations around instruments, methods, and measures to tackle
misinformation in podcasts.