Answers.Body: <p>I think what you are really asking is why don't we talk about alignment from the perspective of general deep learning?</p>
  <p>In fact, we do talk about alignment for general AI system that's being used in production. For example, <a href="https://aclanthology.org/2020.findings-emnlp.301.pdf" rel="nofollow noreferrer">making large language model less toxic</a> is an active
  area of research.</p>
  <p>The reason you see a lot of references to RL in the context alignment is that one method that has been actively researched to make our AI systems aligned with our preference is through <a href="https://openai.com/blog/instruction-following/" rel="nofollow noreferrer">training them via RL</a>. The reward function is, not surprisingly, good alignment with human preferences.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-01T04:47:49.523Z
Answers.LastActivityDate: 2022-08-01T04:47:49.523Z
Answers.UserDisplayName: hongshan.li
Body: <p>I have been reading some articles on AI safety and they almost always speak of AI Safety from the reinforcement learning (RL) perspective, i.e. where we have some artificially intelligent agent acting in an environment so to maximise some reward.</p>
  <p>Is there a reason why the focus is very often from this RL POV?</p>
  <p>Thanks.</p>
  <p>Examples:</p>
  <ul>
  <li><a href="https://www.lesswrong.com/posts/C4tR3BEpuWviT7Sje/2021-ai-alignment-literature-review-and-charity-comparison" rel="nofollow noreferrer">2021 AI Alignment Literature Review and Charity Comparison</a></li>
  <li><a href="https://arxiv.org/abs/2109.13916" rel="nofollow noreferrer">Unsolved Problems in ML Safety</a></li>
  <li><a href="https://www.lesswrong.com/tag/paperclip-maximizer" rel="nofollow noreferrer">Paperclip Maximizer</a></li>
  <li><a href="https://docs.google.com/document/d/1FbTuRvC4TFWzGYerTKpBU7FJlyvjeOvVYF2uYNFSlOc/edit" rel="nofollow noreferrer">AI Alignment Research Overview by Jacob Steinhardt</a></li>
  </ul>
  
Tags: <reinforcement-learning><agi><ai-safety><singularity>
Title: Why are AI Safety discussions almost always from the perspective of reinforcement learning?
UserDisplayName: thesofakillers
ThreadText: # Why are AI Safety discussions almost always from the perspective of reinforcement learning?
  
  (Asked by: username_0 on 2022-07-31)
  
  I have been reading some articles on AI safety and they almost always speak of AI Safety from the reinforcement learning (RL) perspective, i.e. where we have some artificially intelligent agent acting in an environment so to maximise some reward.
  
  Is there a reason why the focus is very often from this RL POV?
  
  Thanks.
  
  Examples:
  
  * [2021 AI Alignment Literature Review and Charity Comparison](https://www.lesswrong.com/posts/C4tR3BEpuWviT7Sje/2021-ai-alignment-literature-review-and-charity-comparison)
  * [Unsolved Problems in ML Safety](https://arxiv.org/abs/2109.13916)
  * [Paperclip Maximizer](https://www.lesswrong.com/tag/paperclip-maximizer)
  * [AI Alignment Research Overview by Jacob Steinhardt](https://docs.google.com/document/d/1FbTuRvC4TFWzGYerTKpBU7FJlyvjeOvVYF2uYNFSlOc/edit)
  
  **Tags:** reinforcement-learning, agi, ai-safety, singularity
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-01. Score: 3 | Accepted answer
  
  I think what you are really asking is why don't we talk about alignment from the perspective of general deep learning?
  
  In fact, we do talk about alignment for general AI system that's being used in production. For example, [making large language model less toxic](https://aclanthology.org/2020.findings-emnlp.301.pdf) is an active
  area of research.
  
  The reason you see a lot of references to RL in the context alignment is that one method that has been actively researched to make our AI systems aligned with our preference is through [training them via RL](https://openai.com/blog/instruction-following/). The reward function is, not surprisingly, good alignment with human preferences.
  
  
  ---
  
  