Why Conscious AI Is a Bad, Bad Idea
&gt; We think we’re intelligent, know we’re conscious, and so assume the two go together.<p>Excuse me, I know _I_ am conscious. There is literally nothing you can do to proof that there&#x27;s anything outside of what I am conscious of. The world could be a very stable illusion, a dream, a simulation and it would all be the same.<p>I don&#x27;t understand why people think that it matters whether  AI is conscious or not. We can&#x27;t even prove that other fellow humans are? But we treat them like they are. We, I at least, feel sad when a tree is cut down. But I don&#x27;t feel the same for a bunch of rocks that are being blown up? Actually I do, when a sacred or beautiful rock is cut in half (like the one that was suspended on a small cliff).<p>What I am trying to say is, it doesn&#x27;t matter if the subject is actually conscious. All that matter is we (I), feel&#x2F;think that it is conscious, or deserving of care and respect.<p>I think what you are afraid of here is agency, which is something that might be dangerous to endow a super intelligent being with.