This is indeed the case. The weakness of our current SSL infrastructure is that any CA can sign any site's cert. So the more trusted CAs you have, the wider the attack vector is--all it takes is one rogue CA (or more likely, one CA with weak security) and you can be effectively man-in-the-middled.