I'd say if we want to keep things looking relatively like modern day a disease would be the best way to go for why the population has dropped off, there aren't many other things I can think of that would leave most stuff intact.