Instead of trying to do “what’s right” try doing what’s best for you. It sounds like you have a bad relationship with your fiancé. If he’s really making you uncomfortable in some ways do you want to spend the rest of your life with that discomfort? I’m not telling you to dump the guy and jump into the other dude’s arms. But sit down and think what’s best for you.