and measure the raw agreement percent-
age between human A and human B (for
comparisons where we have two human
annotators, i.e., not SFT) as well as be-
tween each human and GPT-4.
Direct Preference Optimization