We now conduct a direct comparison of outcome and process supervision. We
first sample between 1 and 200 solutions per problem from a small-scale genera-
tor. For each dataset, we provide three forms of supervision: process supervision
from PRMlarge, outcome supervision from PRMlarge, and outcome supervision
from final-answer checking. The choice of supervision is the only difference
between these three series of reward models, which are otherwise trained on
identical datasets. See Appendix H for more details about how PRMlarge is
used for outcome and process supervision.
Let’s Verify Step by Step