Body: <p>An example is the halting problem, which states computing cannot be solved by exhaustion, but which humans avoid trivially by becoming exhausted.</p>
  <p>Humans typically give up what seems like a lost cause after a certain point, whereas a computer will just keep chugging along.</p>
  <ul>
  <li>Do flaws have utility?</li>
  </ul>
  <p>Can inability be a strength and will AGI require such limitations to achieve human-level intelligence?  Humans are simply not capable of infinite loops, except arguably in cases of mental illness. Are there other problems similar to the halting problem where weakness is a benefit?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-05T01:22:31.193Z
Comments.Text: It's not clear to me why you claim that humans can solve the halting problem. If that was true, that would imply that there are functions that Turing machines cannot compute, so there are more powerful computers than Turing machines, which, as far as I know, has not yet been proven (see the Church-Turing thesis).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-05T08:04:10.077Z
Comments.Text: I agree with nbro, I think you need to make it a little clearer that you don't mean "give up" = "solve". Other than that, I think this could be an interesting question about utility and embodiment (at least that would be my take on what leads to an answer)
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-12T17:17:00.140Z
Comments.Text: Computers can also give up after a certain point. This is one way that algorithms are made to have guaranteed halting: by including a counter that ends the algorithm after a certain number of steps, if not halted before. A more physical analogy is running the computer off a battery. Then when it runs out of power it will stop through 'exhaustion' like a human. You said "give up what seem like a lost cause" which to me is very different from exhaustion -- it is more that the human has guessed their procedure does not halt and therefore terminates early. I don't really see that as a flaw.
Comments.UserDisplayName: user7834
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-17T22:44:59.977Z
Comments.Text: @NeilSlater & nbro — good point.  I should have used "solved" and replaced it with "avoid".  Basically, nature seems to produce that which is minimally optimal, and designs are often flawed but just adequate enough.  Workarounds seem to be our holy grail in much of computer development.
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-18T01:02:03.893Z
Comments.Text: @user7834 I think that's a solid answer, because workarounds have high utility in applied computing, even when not optimal solutions. Wondering if there are other "flaws" that would be useful too #FeatureNotABug
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-18T12:20:11.093Z
Comments.Text: @DukeZhou Glad it's helpful.
Comments.UserDisplayName: user7834
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-16T23:06:53.697Z
Comments.Text: I think this post still contains wrong assumptions. 1. The fact that, sometimes, we're able to recognize an unsolvable or difficult problem is not a flaw, but rather the opposite. In fact, if there's a way to do what you want but in a smarter way (e.g. by spending less time), that would have more utility. Right? 2. We're not always able to tell whether a program will terminate. In some cases, that's easy for us, but we can't tell for sure in all cases too. Of course, that will happen if we run out of resources.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-16T23:06:56.143Z
Comments.Text: Anyway, to make this question answerable, you need to define what you mean by "flaw" by providing some concrete examples of what you really have in mind. Regarding the loops, [here](https://ai.stackexchange.com/q/2516/2444) you have another related post. Regarding this ambiguous claim "Humans are simply not capable of infinite loops, except arguably in cases of mental illness": if by 'a person capable of "loops"', you mean someone that continuously does the same  "wrong" thing, of course, in practice, the loop will not be infinite because the person will eventually die.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-16T23:13:43.600Z
Comments.Text: But that's the same thing with computers. In theory (i.e. Turing machines), they can loop forever, but, in practice, this does not happen. If humans are computers, then, in theory, we may also be able to loop forever.
Comments.UserDisplayName: nbro
Tags: <ai-design><philosophy><agi><human-like><halting-problem>
Title: Might AGI need to be flawed?
UserDisplayName: DukeZhou
ThreadText: # Might AGI need to be flawed?
  
  (Asked by: username_0 on 2021-06-04)
  
  An example is the halting problem, which states computing cannot be solved by exhaustion, but which humans avoid trivially by becoming exhausted.
  
  Humans typically give up what seems like a lost cause after a certain point, whereas a computer will just keep chugging along.
  
  * Do flaws have utility?
  
  Can inability be a strength and will AGI require such limitations to achieve human-level intelligence? Humans are simply not capable of infinite loops, except arguably in cases of mental illness. Are there other problems similar to the halting problem where weakness is a benefit?
  
  **Tags:** ai-design, philosophy, agi, human-like, halting-problem
  
  > **Comments:**
  > **username_1**: It's not clear to me why you claim that humans can solve the halting problem. If that was true, that would imply that there are functions that Turing machines cannot compute, so there are more powerful computers than Turing machines, which, as far as I know, has not yet been proven (see the Church-Turing thesis).  
  > **username_2**: I agree with username_1, I think you need to make it a little clearer that you don't mean "give up" = "solve". Other than that, I think this could be an interesting question about utility and embodiment (at least that would be my take on what leads to an answer)  
  > **username_3**: Computers can also give up after a certain point. This is one way that algorithms are made to have guaranteed halting: by including a counter that ends the algorithm after a certain number of steps, if not halted before. A more physical analogy is running the computer off a battery. Then when it runs out of power it will stop through 'exhaustion' like a human. You said "give up what seem like a lost cause" which to me is very different from exhaustion -- it is more that the human has guessed their procedure does not halt and therefore terminates early. I don't really see that as a flaw.  
  > **username_0**: @username_2 & username_1 — good point.  I should have used "solved" and replaced it with "avoid".  Basically, nature seems to produce that which is minimally optimal, and designs are often flawed but just adequate enough.  Workarounds seem to be our holy grail in much of computer development.  
  > **username_0**: @username_3 I think that's a solid answer, because workarounds have high utility in applied computing, even when not optimal solutions. Wondering if there are other "flaws" that would be useful too #FeatureNotABug  
  > **username_3**: @username_0 Glad it's helpful.  
  > **username_1**: I think this post still contains wrong assumptions. 1. The fact that, sometimes, we're able to recognize an unsolvable or difficult problem is not a flaw, but rather the opposite. In fact, if there's a way to do what you want but in a smarter way (e.g. by spending less time), that would have more utility. Right? 2. We're not always able to tell whether a program will terminate. In some cases, that's easy for us, but we can't tell for sure in all cases too. Of course, that will happen if we run out of resources.  
  > **username_1**: Anyway, to make this question answerable, you need to define what you mean by "flaw" by providing some concrete examples of what you really have in mind. Regarding the loops, [here](https://ai.stackexchange.com/q/2516/2444) you have another related post. Regarding this ambiguous claim "Humans are simply not capable of infinite loops, except arguably in cases of mental illness": if by 'a person capable of "loops"', you mean someone that continuously does the same  "wrong" thing, of course, in practice, the loop will not be infinite because the person will eventually die.  
  > **username_1**: But that's the same thing with computers. In theory (i.e. Turing machines), they can loop forever, but, in practice, this does not happen. If humans are computers, then, in theory, we may also be able to loop forever.  
  
  ---
  
  ## 0 answers
  
  