How can I use AirPlay to watch on the television, but audio through my headphones? I have been using Netflix (for example) on my phone with headphones, but then I realized I would have a better experience watching it on the TV, but keep the audio on the phone.

Is there any way to project a video (only) onto the Apple TV, and keep the audio on my iPad/iPhone/Mac?

... I'll contrast this with OS X, where I'm able to airplay and switch between headphones and TV speakers just by plugging in headphones.

In addition there is this Midi control panel where I can configure outputs:
This cannot be done natively from an iOS device as there are no output settings built in to the iOS platform that allow the splitting of the audio from the video while streaming a file. With the use of extra hardware and/or 3rd party software though what you're trying to accomplish can technically be done in a round-about way, just not the exact way you're describing, where your headphones would be connected to your phone. 

This link has some options (some fairly cheap) that you could explore though:
Use Headphones with Apple TV.

I did notice you mentioned owning a Mac though. One option not mentioned in that article (because it's not Apple TV related) would be to just use your Mac connected to your tv, using a long HDMI cable and a 'Mini display port to HDMI adapter' ($10 on Amazon). That way you could sit with your Mac beside you and your headphones plugged into it, although you would have that HDMI cable strung across the room while in use that you would have to be careful of. (And you would have to remember to select your headphones from 'System Preferences > Sound > Output' likely every time you go to use it this way.)