There is an ongoing debate concerning the extent to which deficits in reading and spelling share cognitive components and whether they rely, in a similar fashion, on sublexical and lexical pathways of word processing. The present study investigates whether the neural substrates of word processing differ in children with various patterns of reading and spelling deficits. Using functional magnetic resonance imaging, we compared written and auditory processing in three groups of 9–13-year olds (N = 104): (1) with age-adequate reading and spelling skills; (2) with reading and spelling deficits (i.e., dyslexia); (3) with isolated spelling deficits but without reading deficits. In visual word processing, both deficit groups showed hypoactivations in the posterior superior temporal cortex compared to typical readers and spellers. Only children with dyslexia exhibited hypoactivations in the ventral occipito-temporal cortex compared to the two groups of typical readers. This is the result of an atypical pattern of higher activity in the occipito-temporal cortex for non-linguistic visual stimuli than for words, indicating lower selectivity. The print–speech convergence was reduced in the two deficit groups. Impairments in lexico-orthographic regions in a reading-based task were associated primarily with reading deficits, whereas alterations in the sublexical word processing route could be considered common for both reading and spelling deficits. These findings highlight the partly distinct alterations of the language network related to reading and spelling deficits.