Something you said at the end reminded me of something my sister told me. A couple things actually. I was going through a really hard time and she told me “You have to stop being a passive bystander in your own life”. She was right!  

The other thing is something her husband’s grandmother told her. This little old lady from Africa gave her this advice: “If you have to be a bitch, be an alpha bitch.”  So be an alpha bitch OP.  Take no crap from that guy. You deserve so much better. And honestly, if you are at all in the position to, I would get a new job away from the other woman.