On Device Learning
Interesting thoughts. I was having a very similar chat with a friend about this recently, including exactly the question of &quot;what should the reward function be&quot;, or at least the most minimal one.<p>Some aspects we thought of:<p>- pain (minimal reward): this should probably be hardwired straight to the artificial brain, though it can&#x27;t be enough as otherwise no activity would take place. The agent would learn that the best course of action is to sit still<p>- so we also came up with curiosity being a necessity. Encountering an unseen or hard to predict state leads to positive reward<p>- although I am not sure whether pain is not sufficient by itself. E.g. in nature, actions are still necessary. Otherwise, other pain signals (hunger and thirst) start showing up.<p>- what is tricky to figure out is how this works for more complicated intelligence such as humans. Let&#x27;s be honest, most babies are fed whenever necessary by their caretakers. What causes them to learn? What causes grown ups to learn?<p>So something important that we&#x27;ll have to figure out is what needs to be hardwired versus what emerges from 2nd order things such as chemistry, hormons, gut bacteria, upbringing, role of parents etc., and whether there&#x27;s a difference between non-conscious or simple intelligence versus complex ones w.r.t. the necessity of these aspects. E.g. you might talk about aspects such as &quot;love&quot; (towards your partner, children, born or not, future generations) but it is much more unclear how necessary this is, and how to quantify it.<p>Perhaps indeed this emerges from the basic reward function but only after a meta-simulation:<p>&gt; I don’t think there’s a way to learn it aside from millennia of multi agent survival competition.