title: Reverse Training to Nurse the Reversal Curse
summary: Large language models (LLMs) have a surprising failure: when trained on "A
  has a feature B", they do not generalize to "B is a feature of A", which is
  termed the Reversal Curse. Even when training with trillions of tokens this
  issue still appears due to Zipf's law - hence even if we train on the entire
  internet. This work proposes an alternative training scheme, called reverse
  training, whereby all words are used twice, doubling the amount of available
  tokens. The LLM is trained in both forward and reverse directions by reversing
  the training strings while preserving (i.e., not reversing) chosen substrings,
  such as entities. We show that data-matched reverse-trained models provide
  superior performance to standard models on standard tasks, and compute-matched
  reverse-trained models provide far superior performance on reversal tasks,
  helping resolve the reversal curse issue.
paper_authors._id: 65fb9dc00b78c48c9e373b44
paper_authors.name: Olga Golovneva
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:50:17.843Z
paper_authors._id: 6318b1e06e6bd039440871d6
paper_authors.avatarUrl: /avatars/2d9f96eb4092a9d1734dcc09303b259a.svg
paper_authors.fullname: Olga Golovneva
paper_authors.type: user
paper_authors.user: Golovneva
paper_authors._id: 65fb9dc00b78c48c9e373b45
paper_authors.name: Zeyuan Allen-Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:50:23.906Z
paper_authors._id: 6316d14c0db6c679cf77fe1f
paper_authors.avatarUrl: /avatars/b33dcadc0ff8941da6aa850284be3207.svg
paper_authors.fullname: Zeyuan Allen-Zhu
paper_authors.type: user
paper_authors.user: zhuzeyuan
paper_authors._id: 65fb9dc00b78c48c9e373b46
paper_authors.name: Jason Weston
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-21T11:50:31.277Z
paper_authors._id: 62f023a36a027498eaa2f9cc
paper_authors.avatarUrl: /avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
paper_authors.fullname: Jason Weston
paper_authors.type: user
paper_authors.user: spermwhale
paper_authors._id: 65fb9dc00b78c48c9e373b47
paper_authors.name: Sainbayar Sukhbaatar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-30T16:01:21.239Z
paper_authors._id: 66a8611eb51510d82ed54231
paper_authors.avatarUrl: /avatars/ad559e774fee4914091b82c9831ae2a2.svg
paper_authors.fullname: Sainbayar Sukhbaatar
paper_authors.type: user
paper_authors.user: sainbar
paper_title: Reverse Training to Nurse the Reversal Curse
paper_summary: Large language models (LLMs) have a surprising failure: when trained on "A
  has a feature B", they do not generalize to "B is a feature of A", which is
  termed the Reversal Curse. Even when training with trillions of tokens this
  issue still appears due to Zipf's law - hence even if we train on the entire
  internet. This work proposes an alternative training scheme, called reverse
  training, whereby all words are used twice, doubling the amount of available
  tokens. The LLM is trained in both forward and reverse directions by reversing
  the training strings while preserving (i.e., not reversing) chosen substrings,
  such as entities. We show that data-matched reverse-trained models provide
  superior performance to standard models on standard tasks, and compute-matched
  reverse-trained models provide far superior performance on reversal tasks,
  helping resolve the reversal curse issue.
paper_ai_summary: Reverse training enhances LLM performance on reversal tasks by doubling the training data through forward and reverse direction training while preserving entities.
paper_ai_keywords: Large language models Reversal Curse Zipf's law reverse training training strings entities