We could, maybe, for various degrees of help and with various dangers depending on how much help (look up Oracle AI), and I would easily wager using relevant tools (not just &quot;AI tech&quot;) would make the positive outcome more likely than not using them, but there are still many ways it can go wrong, and while past and present weak-AI technology might be useful in creating a sentient machine I&#x27;m doubtful much would be that helpful in solving the Friendliness (or benevolence) problem directly.<p>For instance the other comment brings up the uncertainty around giving an Oracle AI the task of finding what benevolence means to humans, to then plug into the final AI hoping it will be Friendly. You don&#x27;t know how to precisely define &#x27;benevolent&#x27; so let the Oracle AI do it (or help you do it), though you think you can at least program the Oracle AI to do a good job in finding answers to vague, complex, fragile problems. (Why? Past success in AI tech?) Is what the Oracle AI outputs actually good? How do you know? What did the Oracle AI have to do to reach its answer, or gain a last bit of certainty in it? (e.g. Did it reason it needed more data and so built or hastened the arrival of brain-scanning technology, started scanning brains from volunteers, the very recently deceased, cryonics patients, or just without its creators knowledge through black market deals, and at some point did it simulate trillions of human minds under all sorts of gruesome scenarios to test responses? Do you even morally care about sentience on silicon? Would you care more if you were an upload?) You&#x27;ve got a lot of problems just building an Oracle AI (or anything powerful enough that can help you solve the hardest problems). Even supposing the output it gives is good and the costs (practical and ethical) were acceptable, will a seed AI allowed to recursively self-improve be guaranteed to preserve this Friendliness property in all subsequent improvements?<p>With the idea of using &quot;AI tech&quot;, you&#x27;ve basically drawn a line in the outcome space that on one side says something like &quot;let&#x27;s just program the thing&quot; and the other says &quot;hold on, let&#x27;s just program the thing but also use all these other relevant computer tools we&#x27;ve developed over the decades to help program the thing&quot;. It&#x27;s the start of an approach, but does it really prune all that much? What do these relevant tools actually buy you in safety, if a safety-guarantee-tool is not among them? Another line which may be what you were heading towards, could be on one side say something like &quot;we might not be able to solve all the necessary problems with our current intelligence, so let&#x27;s also spend time looking for ways to augment human intelligence -- brain-computer interfaces, brain emulations, tweaks to our genetic code, that sort of thing&quot; and the other side &quot;no, we&#x27;re capable, let&#x27;s try our best right now&quot;. Using ems based on friendly-ish human minds that become better than human could very well be a better approach to solving the superintelligent problems correctly and efficiently than doing so on raw human levels, on the other hand it could turn very bad if one of those ems instead just solves the problem of recursively improving themselves and their goal and value system is insufficiently friendly. Or many other failure modes. (Though an interesting &quot;most likely scenario&quot; given certain assumptions and only looking at a span of 2 years which by itself doesn&#x27;t look too terrible is Robin Hanson&#x27;s Age of Em idea.) Just like approaches involving molecular nanotech to help us, there&#x27;s reasons why it could be seen to help the best outcome along, but it also opens the door to a lot of other risks that aren&#x27;t necessarily there if you took the other side, so again how much is really pruned?<p>What I take as your general idea of &quot;maybe we need and should employ additional help and knowledge that we&#x27;ve yet to gain to even really start&quot; isn&#x27;t bad, though, much better than &quot;Benevolence is easy, a King and his subjects all know he is a great ruler when everyone is smiling frequently&quot;, it&#x27;s just lacking in detail to constitute a plan. Throwing in the positive feedback loop idea (which might even manifest as impressive accelerating returns, who knows) to me seems only relevant to figuring out time scales, I don&#x27;t think it says anything about the fraction of good vs. bad outcomes apart from whether short or long time scales matter.