Book Review: “A Thousand Brains” by Jeff Hawkins
AI safety seems like one of those topics that trips up even smart people such as Hawkins, because it intuitively seems like it has very obvious solutions. And for each objection raised there's usually a "so you just..." continuation, and an objection to that, until you're talking about things like instrumental convergence and you're in the territory of trying to reason in quite a complex way about the behaviour of systems that don't exist yet, and so the temptation is to dismiss it all as theoretical hand-wringing.Personally, while I'm not on board with the board with the direst predictions of the super-intelligence pessimist crowd, I have become more and more convinced that goal misalignment is going to be a significant problem, and that while it might not doom the species, it's something that all AI researchers like Hawkins need to start paying close attention to now.