The Earth is a rather hostile place when you use a megayear scale.  Paleontologists and geologists have also found a variety of events (e.g, megacaldera such as Yellowstone cutting loose, meteor impacts) that are correlated with mass extinctions.  We shouldn't tarry in establishing self-sustaining habitats off the Earth.