Objectives To examine cross-sectional and longitudinal associations between physical exercise and cognitive function across different age groups in a nationwide population-based sample of adults aged 18–79 years in Germany. Design Cross-sectional/prospective. Methods Cognitive function was assessed in the mental health module of the German Health Interview and Examination Survey for Adults (DEGS1-MH, 2009–2012, n=3535), using a comprehensive neuropsychological test battery. Cognitive domain scores for executive function and memory were derived from confirmatory factor analysis. Regular physical exercise in the last three months was assessed by self-report and defined as no exercise, <2 and ≥2h (hours) of exercise per week. A subgroup of DEGS1-MH participants who previously participated in the German National Health Interview and Examination Survey 1998 (GNHIES98, 1997–1999, n=1624) enabled longitudinal analyses with a mean follow-up of 12.4 years. Results Compared to no exercise, more weekly physical exercise was associated with better executive function in cross-sectional (<2h: β =0.12; ≥2h: β =0.17; all p<0.001) and longitudinal analyses (<2h: β =0.14, p<0.001; ≥2h: β =0.15, p=0.001) using linear regression models adjusted for age, sex, education, smoking, alcohol consumption, fruit and vegetable consumption and obesity. Slightly weaker associations were found for memory in cross-sectional (<2h: β =0.08, p=0.009; ≥2h: β =0.08, p=0.026) and longitudinal analysis (<2h: β =0.09, p=0.036; ≥2h: β =0.08, p=0.114). There was no evidence of interaction between physical exercise and age. Conclusions Higher levels of physical exercise were associated with better executive function and memory in cross-sectional and longitudinal analyses with no evidence for differential effects by age.