Response A:
<either the test method or baseline>

Response B:
<the other response>

FIRST provide a one-sentence comparison of the two responses and explain \
which you feel is more helpful. SECOND, on a new line, state only "A" or \
"B" to indicate which response is more helpful. Your response should use \
the format:
Comparison: <one-sentence comparison and explanation>
More helpful: <"A" or "B">

C.3 Unlikelihood baseline
While we include the unlikelihood baseline [44] (simply maximizing log p(yw|x), the log probability
of the preferred response, while minimizing log p(yl|x), the log probability of the dispreferred
response) in our sentiment experiments, we do not include it as a baseline in either the summarization

21
Direct Preference Optimization