Is there software for transcribing and recognizing musical pitches in an audio recording? Does GarageBand or any other Mac software make it possible to determine the notes in a musical recording?

It doesn't have to be absolute, and it doesn't even have to guess the rhythm. Determining the intervals would be enough.
No, GarageBand cannot do this. You should investigate the commercial products Celemony Melodyne and Neuratron AudioScore.

Also have a look at Capo by SuperMegaUltraGroovy. It provides a good method for assisting a person in transcribing by ear.