Deficits in confrontation naming ability can occur after epilepsy surgery in the left temporal lobe. This study addresses the functional relationship between postoperative object naming and semantic and phonological speech processing in patients with epilepsy. Fifty-eight consecutive patients with temporal lobe epilepsy from our epilepsy surgery program (24 patients with left temporal lobe epilepsy, 34 patients with right temporal lobe epilepsy) were investigated using the Boston Naming Test and comprehensive semantic and phonological speech testing. Language dominance was evaluated in all patients with the preoperative intracarotid sodium amytal test. Naming decline was observed exclusively in patients with left temporal lobe epilepsy. Regression analysis with semantic processing and phonological input/output processing as independent variables, and naming change in the Boston Naming Test (preoperative–postoperative score) as a dependent variable, revealed a significant association between postoperative naming decline and impaired semantic functions. Accordingly, patients exhibited deficits in the category-related differentiation of objects. It is hypothesized that naming deficits arise from the functional specialization of the left temporal lobe for semantic interpretation of visual input.