I am concerned that a human feedback and evaluation loop will always constitute a walled garden for other humans who did not get to play the part of discriminator. How can humans tasked with discrimination be fooled into accepting a novel output space that would lead to the production of a less biased model? Would this require relabeling and retraining the entire model iteratively or is there a more modular approach?
spam lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor creativity violence