According to the semantic hub hypothesis, a supramodal semantic hub is equally needed to deal with verbal and extraverbal “surface” representations. Damage to the supramodal hub is thought to underlie the crossmodal impairment observed in selective semantic deficits. In the present paper, we provide evidence supporting an alternative view: we hold that semantic impairment is not equal across domains but affects verbal behavior disproportionately. We investigated our hypothesis by manipulating the verbal load in an object decision task. Two pathological groups showing different levels of semantic impairment were enrolled together with their normal controls. The severe group included 10 subjects with semantic dementia and the mild group 10 subjects with Alzheimer’s disease. In keeping with our hypothesis, when shifting from the low verbal load to the high verbal load condition, brain-damaged individuals, as compared to controls, showed a disproportionate impairment as a function of the severity of their semantic deficit.