Answers.Body: <p>Yes, this is one the most popular approaches today. For instance a narrow AI model is built to capture the preference of the user and then this model is used to dispense reward to a reinforcement learning agent.</p>
  <p>The narrow AI model is often called Reward Model or Preference Model.</p>
  <p>The recent RLHF <a href="https://ai.stackexchange.com/questions/38521/how-was-chatgpt-trained/38561#38561">training technique used for chatGPT</a> uses this approach.</p>
  <p>This can be automated further: <a href="https://arxiv.org/abs/2212.08073" rel="nofollow noreferrer">https://arxiv.org/abs/2212.08073</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-15T15:07:29.193Z
Answers.LastActivityDate: 2023-02-15T15:07:29.193Z
Answers.UserDisplayName: Rexcirus
Body: <p>If the AI alignment problem is one of the most pressing issues of our time, could AI itself augment our (i.e., human) quest to solve the alignment problem? Or would AI itself actually be counter-productive for such a meta-level goal?</p>
  
Tags: <agi><meta-heuristics><value-alignment><meta-rules>
Title: Solve the AI alignment problem using (meta-level) AI itself?
UserDisplayName: Hank Igoe
ThreadText: # Solve the AI alignment problem using (meta-level) AI itself?
  
  (Asked by: username_0 on 2023-02-15)
  
  If the AI alignment problem is one of the most pressing issues of our time, could AI itself augment our (i.e., human) quest to solve the alignment problem? Or would AI itself actually be counter-productive for such a meta-level goal?
  
  **Tags:** agi, meta-heuristics, value-alignment, meta-rules
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-02-15. Score: 3
  
  Yes, this is one the most popular approaches today. For instance a narrow AI model is built to capture the preference of the user and then this model is used to dispense reward to a reinforcement learning agent.
  
  The narrow AI model is often called Reward Model or Preference Model.
  
  The recent RLHF [training technique used for chatGPT](https://ai.stackexchange.com/questions/38521/how-was-chatgpt-trained/38561#38561) uses this approach.
  
  This can be automated further: <https://arxiv.org/abs/2212.08073>
  
  
  ---
  
  