In healthy humans, memory for words with emotional valence is better than memory for neutral words. At the same time, the word preceding the emotional word in a word list learning task is remembered less often than other neutral words. Both effects, enhanced memory for emotional words and retrograde amnesia for preceding words, are dependent on intactness of the amygdala. In this study we asked whether patients with temporal lobe epilepsy (TLE), a disease that often involves the amygdala as well, show altered memory for emotional words and the words presented in close temporal proximity. Whereas we found enhanced memory performance for emotional and decreased recognition performance for the preceding and successive word in our 19 control subjects, both effects were strongly reduced in our 21 patients. No group differences occurred in memory for perceptually deviant words. The lack of emotion effects on memory in the patients cannot simply be attributed to altered perception of emotions as the patients rated the emotionality of the words no different than control subjects. Hence, we conclude that patients with TLE have a specific deficit in the emotion-driven encoding enhancement mediated by the amygdala–hippocampus loop.