Rear view ears at speed How can you hear cars approaching over wind noise?

As I transition from a Sunday rider to a commuter I've gotten a good deal faster. This increases the wind noise in my ears, and it's gotten to the point were an unexpected car was surprisingly close as I glanced back prepping for a turn and I wobbled.

At low speeds I used my ears for a general sense of what is around me, is that expected to become unreliable at better speeds?
The stock advice is to put zero reliance on your ears ever. There's this issue, there's the related issue of wind direction keeping you from hearing a car even not at high speeds, and there are extremely quiet cars. Basically the choice is look back a lot and get good at it or use a mirror.