  The conference peer review process involves three constituencies with
different objectives: authors want their papers accepted at prestigious venues
(and quickly), conferences want to present a program with many high-quality and
few low-quality papers, and reviewers want to avoid being overburdened by
reviews. These objectives are far from aligned, primarily because the
evaluation of a submission is inherently noisy. Over the years, conferences
have experimented with numerous policies to navigate the tradeoffs. These
experiments include setting various bars for acceptance, varying the number of
reviews per submission, requiring prior reviews to be included with
resubmissions, and others. In this work, we investigate, both analytically and
empirically, how well various policies work, and more importantly, why they do
or do not work.
  We model the conference-author interactions as a Stackelberg game in which a
prestigious conference commits to an acceptance policy; the authors
best-respond by (re)submitting or not (re)submitting to the conference in each
round of review, the alternative being a "sure accept" (such as a lightly
refereed venue). Our main results include the following observations: 1) the
conference should typically set a higher acceptance threshold than the actual
desired quality; we call this the "resubmission gap". 2) the reviewing load is
heavily driven by resubmissions of borderline papers - therefore, a judicious
choice of acceptance threshold may lead to fewer reviews while incurring an
acceptable loss in conference quality. 3) conference prestige, reviewer
inaccuracy, and author patience increase the resubmission gap, and thus
increase the review load for a fixed level of conference quality. For
robustness, we further consider different models of paper quality and compare
our theoretical results to simulations based on plausible parameters estimated
from real data.

Computer Science and Game Theory