We assessed residual cognitive deficits in young people with idiopathic and cryptogenic epilepsy. In the setting of an ongoing prospective study, we invited participants initially diagnosed and enrolled in the cohort 8–9 years earlier to undergo standardized neuropsychological assessment. Sibling controls were invited when available. We analyzed 143 pairs in which cases had idiopathic or cryptogenic epilepsy and both case and control had normal intelligence. Compared with that for siblings, the Full Scale IQ for cases was 3.3 points lower (P =0.01) mainly due to slower processing speed, which was 5.6 points lower (P =0.0004). Word reading (P =0.04) and spelling (P =0.01), but not other scores, were also lower in cases. Remission status and drug use did not influence findings. In young people of normal intelligence with idiopathic or cryptogenic childhood-onset epilepsy, substantial residual effects of epilepsy appear to be confined largely to slower processing speed.