In fact, in Brin's story "Lungfish," some von Neumann machines are keeping a close watch over the Earth right now, waiting for us to reach a certain level of sophistication before they make their move.