Just for the record, I'm now an atheist, but I spent a number of years actively involved in my church, leading bible studies, and was an active part of the pastoral team (though a civilian; I wasn't a pastor per se).

You'll likely find that many, many atheists either grew up in religious households or came to religion on their own and then found their way out of it, as I did.

There's even an organization created exclusively for leaders in the church community who have concluded that they are atheists: http://clergyproject.org. It's a large and rapidly growing organization.