Surveillance is not about protecting us. It's about control.
I think a lot of people are OK with mass surveillance given a basically benign, good government (which many people tend to think our government is, currently).The problem is that once all of this machinery is in place, it can be used by any government that happens to come to power -- it makes it too easy for a non-benign, dictatorial government to say in power, egregiously abusing the ability to spy on arbitrary citizens.