title: Vera: A General-Purpose Plausibility Estimation Model for Commonsense
    Statements
summary: Despite the much discussed capabilities of today's language models, they are
  still prone to silly and unexpected commonsense failures. We consider a
  retrospective verification approach that reflects on the correctness of LM
  outputs, and introduce Vera, a general-purpose model that estimates the
  plausibility of declarative statements based on commonsense knowledge. Trained
  on ~7M commonsense statements created from 19 QA datasets and two large-scale
  knowledge bases, and with a combination of three training objectives, Vera is a
  versatile model that effectively separates correct from incorrect statements
  across diverse commonsense domains. When applied to solving commonsense
  problems in the verification format, Vera substantially outperforms existing
  models that can be repurposed for commonsense verification, and it further
  exhibits generalization capabilities to unseen tasks and provides
  well-calibrated outputs. We find that Vera excels at filtering LM-generated
  commonsense knowledge and is useful in detecting erroneous commonsense
  statements generated by models like ChatGPT in real-world settings.
paper_authors._id: 645847698eef89b5c3c1d268
paper_authors.name: Jiacheng Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-10T10:36:54.632Z
paper_authors._id: 635f46d1928a42bc95cfcf7c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/635f46d1928a42bc95cfcf7c/5KF8aLiDCJdl7B1SdJ-7V.png
paper_authors.fullname: Jiacheng Liu
paper_authors.type: user
paper_authors.user: liujch1998
paper_authors._id: 645847698eef89b5c3c1d269
paper_authors.name: Wenya Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T13:56:07.443Z
paper_authors._id: 630ca0b2b66f088d54761836
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1661771932639-noauth.jpeg
paper_authors.fullname: wang
paper_authors.type: user
paper_authors.user: wenya
paper_authors._id: 645847698eef89b5c3c1d26a
paper_authors.name: Dianzhuo Wang
paper_authors._id: 645847698eef89b5c3c1d26b
paper_authors.name: Noah A. Smith
paper_authors._id: 645847698eef89b5c3c1d26c
paper_authors.name: Yejin Choi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-10T08:38:59.687Z
paper_authors._id: 64d42729f63b01b7f676b176
paper_authors.avatarUrl: /avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
paper_authors.fullname: Yejin Choi
paper_authors.type: user
paper_authors.user: yejinchoinka
paper_authors._id: 645847698eef89b5c3c1d26d
paper_authors.name: Hannaneh Hajishirzi
paper_title: Vera: A General-Purpose Plausibility Estimation Model for Commonsense
    Statements
paper_summary: Despite the much discussed capabilities of today's language models, they are
  still prone to silly and unexpected commonsense failures. We consider a
  retrospective verification approach that reflects on the correctness of LM
  outputs, and introduce Vera, a general-purpose model that estimates the
  plausibility of declarative statements based on commonsense knowledge. Trained
  on ~7M commonsense statements created from 19 QA datasets and two large-scale
  knowledge bases, and with a combination of three training objectives, Vera is a
  versatile model that effectively separates correct from incorrect statements
  across diverse commonsense domains. When applied to solving commonsense
  problems in the verification format, Vera substantially outperforms existing
  models that can be repurposed for commonsense verification, and it further
  exhibits generalization capabilities to unseen tasks and provides
  well-calibrated outputs. We find that Vera excels at filtering LM-generated
  commonsense knowledge and is useful in detecting erroneous commonsense
  statements generated by models like ChatGPT in real-world settings.
paper_ai_summary: Vera, a model trained on commonsense statements, effectively verifies and separates correct from incorrect statements across diverse domains, outperforming existing models and aiding in detecting erroneous statements from language models.
paper_ai_keywords: commonsense verification Vera declarative statements commonsense knowledge training objectives general-purpose model well-calibrated outputs LM-generated commonsense knowledge ChatGPT