I say no stealth armor personally.  If it adds too much complication to everything, then it's too much.  Perhaps they have low-grade, faulty armor that only reduces visibility by a certain percentage, but no high-grade stuff.