John Carmack: I’m going to work on artificial general intelligence
I&#x27;ve been dabbling in AGI and it seems like the field has a lot of low-hanging fruit. I&#x27;ll bet Carmack can offer some significant contributions.<p>I&#x27;ll take an opportunity to plug a paper I recently published on comparing relative intelligence. The punchline will illuminate the low-hangingness of the fruit in this field.<p>Suppose X and Y are AGIs and you want to know which is more intelligent. For any interactive reward-giving environment E, you could place X into E and see how much reward X gets; likewise for Y. If X gets more reward, you can consider that as evidence of X being more intelligent. But there are many environments, and X might do better in some, Y in others. How can you combine those pieces of evidence into a final judgment?<p>The epiphany I had (obvious in hindsight) is that the above situation is actually an <i>election</i> in disguise. The voters are interactive reward-giving environments, voting (via their rewards) in an intelligence contest between different AGIs. This allows us to import centuries of research on voting and elections! In particular, by using theorems about elections published in the 1970s, I was able to provide an elegant notion of relative intelligence.<p>The notion I provided is elegant enough that some theorems can even be proved with it, for example, formalizations of the idea that &quot;higher-intelligence team-members make higher-intelligence teams&quot;. Which emphasizes the low-fruit-hanginess of the field: as obvious as that idea seems, apparently no-one was able to prove it with previous formal intelligence measures, probably because those previous intelligence measures were too complicated to reason about!<p>Here&#x27;s the paper: <a href="https:&#x2F;&#x2F;philpapers.org&#x2F;archive&#x2F;ALEIVU.pdf" rel="nofollow">https:&#x2F;&#x2F;philpapers.org&#x2F;archive&#x2F;ALEIVU.pdf</a>