2.4 Data Collection

To collect process supervision data, we present human data-labelers with step-
by-step solutions to MATH problems sampled by the large-scale generator.
Their task is to assign each step in the solution a label of positive, negative,
or neutral, as shown in Figure 1. A positive label indicates that the step is cor-
rect and reasonable. A negative label indicates that the step is either incorrect
or unreasonable. A neutral label indicates ambiguity. In practice, a step may
be labelled neutral if it is subtly misleading, or if it is a poor suggestion that
is technically still valid. We permit neutral labels since this allows us to defer
the decision about how to handle ambiguity: at test time, we can treat neutral
labels as either positive or negative. A more detailed description of the labelling
instructions is provided in Appendix D.
Let’s Verify Step by Step