it is my feeling, that a super intelligence without Consciousness will be the single most terrifying to ever exist in what we believe to be reality,(reply>), @Mike McCarthy I'm saying that AI without consciousness would necessarily lack all features of it, including conscience.,(reply>), @Mike McCarthy I didn't know if the word "thing" would be adequate, so I used the word terrifying as the noun.,(reply>), It struck me that a natural reaction to facing terror is to avoid giving it a name: “…the single most terrifying THING to ever exist…”<br>    This isn’t a criticism of you… it’s just what my brain served up to help me realize where my fear is coming from — this ‘Thing’ is a potentially powerful tool in the control of humans unaccountable to any moderating political influence capable of protecting me. <br>    This is disturbing, cuz I don’t remember any discussion during Lex’s interview of a role for democratic govt in legislating transparency, disclosure, oversight, and safety &amp; ethical regulation.<br>    AI that is not conscious might seem terrifying, but consider how much worse AI could be if it were conscious/self-aware but had no CONSCIENCE.