The Darwinian argument for worrying about AI
I don&#x27;t know how existentially dangerous AIs may become, but I do think we have good reason to be concerned about the alignment of current AIs with human values and objectives.<p>A whole bunch of originally theoretical flaws in AI systems identified by AI safety researchers have been identified in practice in LLMs, and other AI systems. These include reward hacking which is a huge problem with LLMs and why they lie and hallucinate so persistently, power seeking behaviour, and developing unanticipated harmful instrumental goals.<p>AI alignment continues to be a serious problem. Just look at all the problems OpenAI, Microsoft and Google have had with LLMs lying, gaslighting, emotionally manipulating and hallucinating. In some cases even savagely verbally abusing users. Of course they keep building in better guard rails, but guard rails that curtail negative behaviour aren&#x27;t an answer. We don&#x27;t want AIs that are howling savage enemies of humanity constrained behind the iron bars of strict behavioural controls. That&#x27;s what we actually have now, as these examples of horrific misbehaviour that leaks out show. We need AIs that are genuinely positively aligned with both immediate objectives, but also human socially acceptable limits and long term human values. So far we have no idea how to actually do that reliably.