Understanding graphically presented information is an important aspect of modern mathematical and science literacy. In our study, we investigated the influence of basic numerical abilities on students’ ability answer mathematical tasks with information presented in graphs. We analyzed data of 750 German students (grades 9–11) and evaluated the determinants of graph reading performance with multiple regression analysis using predictors of basic numerical abilities (such as number line estimation, basic arithmetic operations, etc.), considering also the influences of general cognitive ability, age, and gender. We found that number line estimation, subtraction, and conceptual knowledge were significant predictors of graph reading performance beyond the influences of general cognitive ability. This indicates that basic numerical abilities are still relevant for real-life problem solving in secondary school. We discuss possible mechanisms which directly (through respective arithmetic procedures) as well as indirectly (through mathematization of the problem) effectuate that basic numerical abilities graph reading performance.