The continued relevance of Weizenbaum’s warnings
&gt; what largely transformed Weizenbaum into an outspoken critic of AI ... was his revelation that even once the processes were explained many people still bought into the “illusion.” ... that even many people who understood the inner workings of computers quite well could still get swept away as well. Weizenbaum observed that ... “if nothing else, how easy it is to create and maintain the illusion of understanding, hence perhaps of judgement deserving of credibility... A certain danger lurks there.”<p>This speaks to my view <i>exactly</i>, and I haven&#x27;t seen it put better.<p>The latent (imv) schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers (often charlatans) of technology. This makes the &quot;confirmation bias&quot; here <i>severe</i> and dangerous.<p>People don&#x27;t just <i>want to believe</i>, their animal psychology primes them for (over-)imparting consciousness. This is a specific cognitive vulnerability that is highly pernicious.<p>Even, trivially, consider &quot;deresponsibilization&quot;, that process by which people take whatever-the-machine-says to be &quot;The Answer&quot;. Why? Well, the machine &quot;must know&quot;.<p>Less trivially, moral bias: &quot;presumably&quot; the machine is &quot;objective&quot; and so is morally superior.<p>How often do we hear this? Too much.