title: MixCE: Training Autoregressive Language Models by Mixing Forward and
    Reverse Cross-Entropies
summary: Autoregressive language models are trained by minimizing the cross-entropy of
  the model distribution Q relative to the data distribution P -- that is,
  minimizing the forward cross-entropy, which is equivalent to maximum likelihood
  estimation (MLE). We have observed that models trained in this way may
  "over-generalize", in the sense that they produce non-human-like text.
  Moreover, we believe that reverse cross-entropy, i.e., the cross-entropy of P
  relative to Q, is a better reflection of how a human would evaluate text
  generated by a model. Hence, we propose learning with MixCE, an objective that
  mixes the forward and reverse cross-entropies. We evaluate models trained with
  this objective on synthetic data settings (where P is known) and real data, and
  show that the resulting models yield better generated text without complex
  decoding strategies. Our code and models are publicly available at
  https://github.com/bloomberg/mixce-acl2023
paper_authors._id: 64741d21485a7c8e1bd57722
paper_authors.name: Shiyue Zhang
paper_authors._id: 64741d21485a7c8e1bd57723
paper_authors.name: Shijie Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-08T08:52:04.019Z
paper_authors._id: 6137ad94501f80a6f6e1eac9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1631038777636-noauth.jpeg
paper_authors.fullname: Shijie Wu
paper_authors.type: user
paper_authors.user: shijie-wu
paper_authors._id: 64741d21485a7c8e1bd57724
paper_authors.name: Ozan Irsoy
paper_authors._id: 64741d21485a7c8e1bd57725
paper_authors.name: Steven Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-30T09:57:06.435Z
paper_authors._id: 646e4e8ecbb7bb99651180db
paper_authors.avatarUrl: /avatars/0a53e18f81a6d517487477d8874aaa83.svg
paper_authors.fullname: Steven Lu
paper_authors.type: user
paper_authors.user: slu126
paper_authors._id: 64741d21485a7c8e1bd57726
paper_authors.name: Mohit Bansal
paper_authors._id: 64741d21485a7c8e1bd57727
paper_authors.name: Mark Dredze
paper_authors._id: 64741d21485a7c8e1bd57728
paper_authors.name: David Rosenberg
paper_title: MixCE: Training Autoregressive Language Models by Mixing Forward and
    Reverse Cross-Entropies
paper_summary: Autoregressive language models are trained by minimizing the cross-entropy of
  the model distribution Q relative to the data distribution P -- that is,
  minimizing the forward cross-entropy, which is equivalent to maximum likelihood
  estimation (MLE). We have observed that models trained in this way may
  "over-generalize", in the sense that they produce non-human-like text.
  Moreover, we believe that reverse cross-entropy, i.e., the cross-entropy of P
  relative to Q, is a better reflection of how a human would evaluate text
  generated by a model. Hence, we propose learning with MixCE, an objective that
  mixes the forward and reverse cross-entropies. We evaluate models trained with
  this objective on synthetic data settings (where P is known) and real data, and
  show that the resulting models yield better generated text without complex
  decoding strategies. Our code and models are publicly available at
  https://github.com/bloomberg/mixce-acl2023
paper_ai_keywords: autoregressive language models forward cross-entropy maximum likelihood estimation (MLE) reverse cross-entropy MixCE