title: I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
summary: Large Language Models are known to capture real-world knowledge, allowing
  them to excel in many downstream tasks. Despite recent advances, these models
  are still prone to what are commonly known as hallucinations, causing them to
  emit unwanted and factually incorrect text. In this work, we propose a novel
  calibration method that can be used to combat hallucinations. We add a special
  [IDK] ("I don't know") token to the model's vocabulary and introduce an
  objective function that shifts probability mass to the [IDK] token for
  incorrect predictions. This approach allows the model to express uncertainty in
  its output explicitly. We evaluate our proposed method across multiple model
  architectures and factual downstream tasks. We find that models trained with
  our method are able to express uncertainty in places where they would
  previously make mistakes while suffering only a small loss of encoded
  knowledge. We further perform extensive ablation studies of multiple variations
  of our approach and provide a detailed analysis of the precision-recall
  tradeoff of our method.
paper_authors._id: 6759fcc41f49e61f776bb7ce
paper_authors.name: Roi Cohen
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-12-11T21:01:39.374Z
paper_authors._id: 65b7afaeb4ec059b974e9189
paper_authors.avatarUrl: /avatars/08059754f2d299b76ab4649e8d768253.svg
paper_authors.fullname: Roi Cohen
paper_authors.type: user
paper_authors.user: roicohen9
paper_authors._id: 6759fcc41f49e61f776bb7cf
paper_authors.name: Konstantin Dobler
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-13T07:30:55.400Z
paper_authors._id: 62a33fe73621790e22b26c87
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62a33fe73621790e22b26c87/U5LnpPTlWcdshWFch2adD.jpeg
paper_authors.fullname: Konstantin Dobler
paper_authors.type: user
paper_authors.user: konstantindobler
paper_authors._id: 6759fcc41f49e61f776bb7d0
paper_authors.name: Eden Biran
paper_authors._id: 6759fcc41f49e61f776bb7d1
paper_authors.name: Gerard de Melo
paper_title: I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token
paper_summary: Large Language Models are known to capture real-world knowledge, allowing
  them to excel in many downstream tasks. Despite recent advances, these models
  are still prone to what are commonly known as hallucinations, causing them to
  emit unwanted and factually incorrect text. In this work, we propose a novel
  calibration method that can be used to combat hallucinations. We add a special
  [IDK] ("I don't know") token to the model's vocabulary and introduce an
  objective function that shifts probability mass to the [IDK] token for
  incorrect predictions. This approach allows the model to express uncertainty in
  its output explicitly. We evaluate our proposed method across multiple model
  architectures and factual downstream tasks. We find that models trained with
  our method are able to express uncertainty in places where they would
  previously make mistakes while suffering only a small loss of encoded
  knowledge. We further perform extensive ablation studies of multiple variations
  of our approach and provide a detailed analysis of the precision-recall
  tradeoff of our method.
paper_ai_keywords: Large Language Models hallucinations calibration method: Distill n' Explain (DnX) special [IDK] ("I don't know") token objective function surrogate GNN uncertainty Distill n\' Explain (DnX) special [IDK] ("I don't know") token objective function surrogate GNN uncertainty Distill n' Explain (DnX) surrogate GNN surrogate GNN