if you don't use scaffolding, is ruby on rails still good for rapid development?
I have never seen Rails scaffold-generated view code used in a production app. The chances that it's going to create the look that you want is nearly zero. I use the generators for models and controllers all the time, as they are very useful.
To your question of frameworks:
If you know Python better, use Django.
If you know Ruby better, use Rails. 
If this is a hobby site, use whichever one interests you the most.
If you take out the scaffolding feature where it creates the model/controller, and CRUD pages for you, is ruby on rails still any faster to market than say, django?
It seems very similiar to be if you take away that step...(even though I believe django has similar auto-gen capabilities)
I am reading the starting guide on the rails site, and when it introduces the scaffolding feature, it says that many people prefer to hand code these types of areas.