Lesion and neuroimaging studies have demonstrated that the mesial temporal lobe is crucial for recognizing emotions from facial expressions. In humans, bilateral amygdala damage is followed by impaired recognition of facial expressions of fear. To evaluate the influence of unilateral mesial temporal lobe damage we examined recognition of facial expressions and functional magnetic resonance (fMRI) brain activation associated with incidental processing of fearful faces in thirteen mesial temporal lobe epilepsy (MTLE) patients (eight with right MTLE, five with left MTLE). We also examined the effect of early versus later damage, comparing subjects with hippocampal–amygdalar sclerosis (MTS) and seizures occurring before five years of age to epilepsy patients with late onset seizures. Fourteen healthy volunteers participated as controls. Neuropsychological testing demonstrated that the ability of right MTLE patients to recognize fearful facial expressions is impaired. Patients with early onset of seizures were the most severely impaired. This deficit was associated with defective activation of a neural network involved in the processing of fearful expressions, which in controls and left MTLE included the left inferior frontal cortex and several occipito-temporal structures of both hemispheres.