When high-status males leave their wives for a younger model, you can stigmatize them, damaging their social, and even professional, standing.
When stigmatizing a high-status male, there is a high risk of accidentally also stigmatizing the wife.