for example they often prefer responses that include non-functional URLs, which are probably one of the
simplest possible ‘lies’ to debunk.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback