We assessed the relationship between brain structure and function in 10 individuals with specific language impairment (SLI), compared to six unaffected siblings, and 16 unrelated control participants with typical language. Voxel-based morphometry indicated that grey matter in the SLI group, relative to controls, was increased in the left inferior frontal cortex and decreased in the right caudate nucleus and superior temporal cortex bilaterally. The unaffected siblings also showed reduced grey matter in the caudate nucleus relative to controls. In an auditory covert naming task, the SLI group showed reduced activation in the left inferior frontal cortex, right putamen, and in the superior temporal cortex bilaterally. Despite spatially coincident structural and functional abnormalities in frontal and temporal areas, the relationships between structure and function in these regions were different. These findings suggest multiple structural and functional abnormalities in SLI that are differently associated with receptive and expressive language processing.