1. Because we have empathy and because we have no compelling economic reason to (economics always overruling silly things like empathy).<p>2. Empathy, which is a part of our emotions, is an incredibly complex thing we can&#x27;t even begin to formalize yet - it&#x27;s extremely unlikely that a random AI we&#x27;d build would happen to have the exact same emotional makeup as we do.<p>3. &quot;Smart&quot; has a particular meaning here when used in discussing AI. It implies raw intelligence, not morals. A smart AI may be incredibly good at evaluating information, inferring new facts and doing complex tasks. None of it means it will necessarily have to ask itself philosophical questions that we do - those actually mostly come from our emotions, from our desire to understand our place in the universe.<p>The core of the AI-threat issue is that for an AI to not be dangerous it would have to value us <i>and</i> the same things we value. We can&#x27;t agree about our exact values even among ourselves, and we&#x27;re nowhere near even trying to formalize it on a level of detail that would be useful for programming.