How hot is it normal for a 45 watt MagSafe Adapter to get during use? I have a Mid-2011 MacBook Air (13"). Recently, the 45 watt MagSafe Adapter grew extremely hot (painful to touch) and smelled slightly singed while I was using the machine and plugged into the wall. It appears to still work.

I am using the extender cord, not the duck-head plug, and I live in the United States.

What is a normal high temperature for the MagSafe to reach? At what point do I visit my friendly local Genius Bar?
Painful to the touch - especially when the device hasn't been that hot since day one - is a great time to unplug and take your rig (computer and power adapter) in to have things checked out. You might power off everything and try hooking just the shut down mac and see if drawing less current helps, but keep a very close eye on things and don't leave it plugged in just in case if you have any concerns.

I've got an 85W adapter that started buzzing faintly and now it's getting hot as well so I've stopped using it until I can have things checked out.

Also, ambient temperature is a big factor. Since things are air cooled, if you are 85 degrees F then the charger can't dissipate heat as well as if you were in a server room cooled to 62 degrees or so.