Is the first boot after a full system restore from TimeMachine supposed to take a long time? I have had the hard drive in my iMac replaced and now I'm trying to restore it from my TimeMachine back up. I thought the TimeMachine back up was corrupt but maybe I'm just not patient enough, or I'm misunderstanding the restore process.

I'm able to restore successfully, but on reboot the machine shows me the white screen with the Apple logo and the spinner. I can hear some occasional sounds from the hard drive in the machine.  The occasional hard drive sounds make me wonder if it's doing and fsck or similar operation on the drive before the system boots.

It's been at this screen for a good hour now.

I have not heard a chime (and have been in front of the machine working the entire time).

Is this just what happens after a TimeMachine restore or should it have booted at a near-normal speed?
Sounds like you have a problem.  Have a look at this apple support document on what to do if this happens.