Wars? Revolution within a country? Economic collapse and anarchy? Simply a community that wants to be closed off (aka Waco without the religious nutcases)? We don't need a reduced population I don't think - just a changed/damaged society that is sufficient for this band of 'haves' to wall themselves in and make their own society.