A number of recent studies utilizing both functional neuroimaging and lesion analysis techniques in neurologic patients have produced conflicting results with respect to the neural correlates of picture naming. Picture naming involves a number of cognitive processes, from visual perception/recognition to lexical-semantic retrieval to articulation. This middle process, the ability to retrieve a name associated with an object, has been attributed in some cases to posterior portions of the left lateral temporal lobe and in other cases, to anterior temporal cortex. In the current study, we used voxel-based lesion symptom mapping (VLSM) to identify neural correlates of picture naming in a large sample of well-characterized left hemisphere (LH) patients suffering from a range of naming deficits. We tested patients on the Boston Naming Test (BNT), a clinical, standardized measure of picture naming that is widely used in both clinical and research settings. We found that overall performance on the BNT was associated with a network of LH regions that included significant portions of the left anterior to posterior middle temporal gyrus (MTG) and superior temporal gyrus (STG) and underlying white matter, and extended into left inferior parietal cortex. However, when we added covariates to this analysis that controlled for deficits in visual recognition and motor speech in order to isolate brain regions specific to lexical-semantic retrieval, the significant regions that remained were confined almost exclusively to the left mid-posterior MTG and underlying white matter. These findings support the notion that a large network in left peri-Sylvian cortex supports picture naming, but that the left mid-posterior MTG and underlying white matter play a critical role in the core ability to retrieve a name associated with an object or picture.