Extensive retrograde memory deficits have been detected in patients with temporal lobe epilepsy (TLE) and in patients who have undergone temporal lobectomy (TL). To date, no group study has used a prospective longitudinal design. We assessed memory for public and autobiographical facts and events in patients (n =15) with focal left- or right-sided TLE before and after TL using a battery of tests that covered the life span. Patients who underwent TL demonstrated material-specific memory decline in retrograde memory. Left TL was associated with a significant drop in recall of famous people’s names. Patients who underwent right TL exhibited a significant decline in the recall of autobiographical events from across the life span. Our study indicates for the first time that TL is associated with a risk of mild, material-specific retrograde memory decline. Verifications with a larger sample size and other measures of remote memory are needed.