You doing everything in your power to keep him happy while he just kinda does whatever he wants (even to the degree of outright insulting you) is a super uneven power dynamic, what he said is verbal abuse, and "I didn't mean it like that I was just mad" is cheap gaslighting. It's clear you hold him in higher regard than he does you and he's taking advantage of that.