Wheel rigidity, is it noticeable and should I care about it? I have a friend who shopped for Aero wheels recently and I heard a lot of talk about how this wheel was more rigid than that wheel and so on.

While I understand that a more rigid wheel should have less loss of energy, can it really be felt? I have entry level wheels and I don't feel any "softness". Is the rigidity of a wheel something that makes a difference only when you're a racer and want to shave off a few tenths of seconds? 

(I guess it can be something important for heavily loaded touring bikes, but I'm more interested in "racing" road bikes.)
For the average rider the answer to 'should I care?' is probably not.  If you are racing a lot of time trials then it's possible that it will make a difference.

Some of the primary benefits of a spoked-pneumatic wheel are traction and shock absorption. There is a trade off between increased rigidity (as well as higher air pressure) and both of those benefits.

So, a high rigidity wheel may increase energy transfer, and decrease rolling resistance. At the same time the harsher ride is going to increase fatigue, and the added vibration over anything but the smoothest surfaces will result in a decrease in traction.  Overall for the typical rider it's probably a wash.

It's largely the same story for the Aero benefits as well.  Unless you are primarily concerned about going fast in solo efforts the benefit would be minimal.