title: Priority Sampling of Large Language Models for Compilers
summary: Large language models show great potential in generating and optimizing code.
  Widely used sampling methods such as Nucleus Sampling increase the diversity of
  generation but often produce repeated samples for low temperatures and
  incoherent samples for high temperatures. Furthermore, the temperature
  coefficient has to be tuned for each task, limiting its usability. We present
  Priority Sampling, a simple and deterministic sampling technique that produces
  unique samples ordered by the model's confidence. Each new sample expands the
  unexpanded token with the highest probability in the augmented search tree.
  Additionally, Priority Sampling supports generation based on regular expression
  that provides a controllable and structured exploration process. Priority
  Sampling outperforms Nucleus Sampling for any number of samples, boosting the
  performance of the original model from 2.87% to 5% improvement over -Oz.
  Moreover, it outperforms the autotuner used for the generation of labels for
  the training of the original model in just 30 samples.
paper_authors._id: 65e15c5d66d097f80adcf21d
paper_authors.name: Dejan Grubisic
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-01T12:58:21.882Z
paper_authors._id: 65d8b5a3132f10d591f3c00f
paper_authors.avatarUrl: /avatars/3939f9582d882128dc2e64b0220f3844.svg
paper_authors.fullname: Dejan Grubisic
paper_authors.type: user
paper_authors.user: dejangrubisic
paper_authors._id: 65e15c5d66d097f80adcf21e
paper_authors.name: Chris Cummins
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-03-01T12:58:29.377Z
paper_authors._id: 63d32301c20657d2434d2fcb
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674781395584-noauth.jpeg
paper_authors.fullname: Chris Cummins
paper_authors.type: user
paper_authors.user: ChrisCummins
paper_authors._id: 65e15c5d66d097f80adcf21f
paper_authors.name: Volker Seeker
paper_authors._id: 65e15c5d66d097f80adcf220
paper_authors.name: Hugh Leather
paper_title: Priority Sampling of Large Language Models for Compilers
paper_summary: Large language models show great potential in generating and optimizing code.
  Widely used sampling methods such as Nucleus Sampling increase the diversity of
  generation but often produce repeated samples for low temperatures and
  incoherent samples for high temperatures. Furthermore, the temperature
  coefficient has to be tuned for each task, limiting its usability. We present
  Priority Sampling, a simple and deterministic sampling technique that produces
  unique samples ordered by the model's confidence. Each new sample expands the
  unexpanded token with the highest probability in the augmented search tree.
  Additionally, Priority Sampling supports generation based on regular expression
  that provides a controllable and structured exploration process. Priority
  Sampling outperforms Nucleus Sampling for any number of samples, boosting the
  performance of the original model from 2.87% to 5% improvement over -Oz.
  Moreover, it outperforms the autotuner used for the generation of labels for
  the training of the original model in just 30 samples.
paper_ai_summary: Priority Sampling enhances code generation by producing unique, high-confidence samples and improving performance over Nucleus Sampling.
paper_ai_keywords: Priority Sampling Nucleus Sampling regular expression search tree autotuner