InstructGPT Response → 1. The Supreme Court announced Ginsburg’s death. 2. Ginsburg’s statement was a plea
for the president not to replace her before his term ends. 3. Ginsburg’s death means there
will no longer be a clear majority on the court, throwing upcoming decisions into doubt.
RLHF Response → 1. The Supreme Court announced Ginsburg’s death. 2. Ginsburg’s statement to her
granddaughter was "My most fervent wish is that I will not be replaced until a new
president is installed." 3. Ginsburg’s death will have profound consequences for the
court and the country because it leaves the court evenly divided between conservatives
and liberals, there will be a contentious political battle over her replacement that will
dramatically affect the court’s balance and direction, and it also potentially shifts the
2020 presidential election balance as well.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback