title: MEETING DELEGATE: Benchmarking LLMs on Attending Meetings on Our Behalf
summary: In contemporary workplaces, meetings are essential for exchanging ideas and
  ensuring team alignment but often face challenges such as time consumption,
  scheduling conflicts, and inefficient participation. Recent advancements in
  Large Language Models (LLMs) have demonstrated their strong capabilities in
  natural language generation and reasoning, prompting the question: can LLMs
  effectively delegate participants in meetings? To explore this, we develop a
  prototype LLM-powered meeting delegate system and create a comprehensive
  benchmark using real meeting transcripts. Our evaluation reveals that GPT-4/4o
  maintain balanced performance between active and cautious engagement
  strategies. In contrast, Gemini 1.5 Pro tends to be more cautious, while Gemini
  1.5 Flash and Llama3-8B/70B display more active tendencies. Overall, about 60\%
  of responses address at least one key point from the ground-truth. However,
  improvements are needed to reduce irrelevant or repetitive content and enhance
  tolerance for transcription errors commonly found in real-world settings.
  Additionally, we implement the system in practical settings and collect
  real-world feedback from demos. Our findings underscore the potential and
  challenges of utilizing LLMs as meeting delegates, offering valuable insights
  into their practical application for alleviating the burden of meetings.
paper_authors._id: 67a998fe495b23306cdbf51d
paper_authors.name: Lingxiang Hu
paper_authors._id: 67a998fe495b23306cdbf51e
paper_authors.name: Shurun Yuan
paper_authors._id: 67a998fe495b23306cdbf51f
paper_authors.name: Xiaoting Qin
paper_authors._id: 67a998fe495b23306cdbf520
paper_authors.name: Jue Zhang
paper_authors._id: 67a998fe495b23306cdbf521
paper_authors.name: Qingwei Lin
paper_authors._id: 67a998fe495b23306cdbf522
paper_authors.name: Dongmei Zhang
paper_authors._id: 67a998fe495b23306cdbf523
paper_authors.name: Saravan Rajmohan
paper_authors._id: 67a998fe495b23306cdbf524
paper_authors.name: Qi Zhang
paper_title: MEETING DELEGATE: Benchmarking LLMs on Attending Meetings on Our Behalf
paper_summary: In contemporary workplaces, meetings are essential for exchanging ideas and
  ensuring team alignment but often face challenges such as time consumption,
  scheduling conflicts, and inefficient participation. Recent advancements in
  Large Language Models (LLMs) have demonstrated their strong capabilities in
  natural language generation and reasoning, prompting the question: can LLMs
  effectively delegate participants in meetings? To explore this, we develop a
  prototype LLM-powered meeting delegate system and create a comprehensive
  benchmark using real meeting transcripts. Our evaluation reveals that GPT-4/4o
  maintain balanced performance between active and cautious engagement
  strategies. In contrast, Gemini 1.5 Pro tends to be more cautious, while Gemini
  1.5 Flash and Llama3-8B/70B display more active tendencies. Overall, about 60\%
  of responses address at least one key point from the ground-truth. However,
  improvements are needed to reduce irrelevant or repetitive content and enhance
  tolerance for transcription errors commonly found in real-world settings.
  Additionally, we implement the system in practical settings and collect
  real-world feedback from demos. Our findings underscore the potential and
  challenges of utilizing LLMs as meeting delegates, offering valuable insights
  into their practical application for alleviating the burden of meetings.
paper_ai_keywords: Large Language Models (LLMs) natural language generation reasoning meeting delegate system benchmark real meeting transcripts GPT-4/4o Gemini 1.5 Pro Gemini 1.5 Flash Llama3-8B/70B key point ground-truth transcription errors