Sometimes the best way to get the information you want is to be blunt. Being this way doesn't leave any wiggle room for misinterpretation and gets you a straight answer. For example, invite your friend out to coffee and say, “I'm just wondering if you have feelings for my boyfriend. Some of the things you do and ways you behave make me suspicious.” Your friend may get their feelings hurt, but it's better to know. You will have to become more direct if they don't get the hint or they continue their behavior. Your friendship is compromised anyways because of their flirting, so telling them to stop talking to your man won't do additional harm. For example, you can say, “I'm not sure if you're trying to be funny or if you don't realize you're flirting, but it's making me uncomfortable and I want you to stop.” Try to say this to them when you are alone. Making a scene in front of people will only make the situation worse. Stop bringing your friend around your boyfriend or end the relationship with them all together if the flirting doesn't stop. It's clearly not a good friendship in the first place if your friend doesn't respect you or your relationship enough to back off.

Summary:
Ask them if they like your boyfriend. Tell them to back off. Avoid putting yourself in the situation again.