title: LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews
summary: Peer review is a cornerstone of quality control in scientific publishing.
  With the increasing workload, the unintended use of `quick' heuristics,
  referred to as lazy thinking, has emerged as a recurring issue compromising
  review quality. Automated methods to detect such heuristics can help improve
  the peer-reviewing process. However, there is limited NLP research on this
  issue, and no real-world dataset exists to support the development of detection
  tools. This work introduces LazyReview, a dataset of peer-review sentences
  annotated with fine-grained lazy thinking categories. Our analysis reveals that
  Large Language Models (LLMs) struggle to detect these instances in a zero-shot
  setting. However, instruction-based fine-tuning on our dataset significantly
  boosts performance by 10-20 performance points, highlighting the importance of
  high-quality training data. Furthermore, a controlled experiment demonstrates
  that reviews revised with lazy thinking feedback are more comprehensive and
  actionable than those written without such feedback. We will release our
  dataset and the enhanced guidelines that can be used to train junior reviewers
  in the community. (Code available here:
  https://github.com/UKPLab/arxiv2025-lazy-review)
paper_authors._id: 67ff9068de173bffef0e7eae
paper_authors.name: Sukannya Purkayastha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-16T11:57:51.596Z
paper_authors._id: 6408e9f2bf3e9a4bb2e09400
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678306212334-6408e9f2bf3e9a4bb2e09400.jpeg
paper_authors.fullname: Sukannya Purkayastha
paper_authors.type: user
paper_authors.user: sukannya
paper_authors._id: 67ff9068de173bffef0e7eaf
paper_authors.name: Zhuang Li
paper_authors._id: 67ff9068de173bffef0e7eb0
paper_authors.name: Anne Lauscher
paper_authors._id: 67ff9068de173bffef0e7eb1
paper_authors.name: Lizhen Qu
paper_authors._id: 67ff9068de173bffef0e7eb2
paper_authors.name: Iryna Gurevych
paper_title: LazyReview A Dataset for Uncovering Lazy Thinking in NLP Peer Reviews
paper_summary: Peer review is a cornerstone of quality control in scientific publishing.
  With the increasing workload, the unintended use of `quick' heuristics,
  referred to as lazy thinking, has emerged as a recurring issue compromising
  review quality. Automated methods to detect such heuristics can help improve
  the peer-reviewing process. However, there is limited NLP research on this
  issue, and no real-world dataset exists to support the development of detection
  tools. This work introduces LazyReview, a dataset of peer-review sentences
  annotated with fine-grained lazy thinking categories. Our analysis reveals that
  Large Language Models (LLMs) struggle to detect these instances in a zero-shot
  setting. However, instruction-based fine-tuning on our dataset significantly
  boosts performance by 10-20 performance points, highlighting the importance of
  high-quality training data. Furthermore, a controlled experiment demonstrates
  that reviews revised with lazy thinking feedback are more comprehensive and
  actionable than those written without such feedback. We will release our
  dataset and the enhanced guidelines that can be used to train junior reviewers
  in the community. (Code available here:
  https://github.com/UKPLab/arxiv2025-lazy-review)
paper_ai_summary: A dataset called LazyReview is introduced to improve the detection of lazy thinking in peer reviews, showing that instruction-based fine-tuning of LLMs enhances their ability to identify such heuristics.
paper_ai_keywords: peer-review sentences lazy thinking LazyReview Large Language Models zero-shot setting instruction-based fine-tuning peer reviews