Background Depression is associated with cognitive impairment and dementia, but whether treatment for depression with antidepressants reduces the risk for cognitive decline is unclear. We assessed the association between antidepressant use and cognitive decline over 6 years. Methods Participants were 3714 adults aged 50 years or more who were enrolled in the nationally representative Health and Retirement Study and had self-reported antidepressant use. Depressive symptoms were assessed using the 8-item Center for Epidemiologic Studies Depression Scale. Cognitive function was assessed at 4 time points (2004, 2006, 2008, 2010) using a validated 27-point scale. Change in cognitive function over the 6-year follow-up period was examined using linear growth models, adjusted for demographics, depressive symptoms, comorbidities, functional limitations, and antidepressant anticholinergic activity load. Results At baseline, cognitive function did not differ significantly between the 445 (12.1%) participants taking antidepressants and those not taking antidepressants (mean, 14.9%; 95% confidence interval, 14.3-15.4 vs mean, 15.1%; 95% confidence interval, 14.9-15.3). During the 6-year follow up period, cognition declined in both users and nonusers of antidepressants, ranging from −1.4 change in mean score in those with high depressive symptoms and taking antidepressants to −0.5 change in mean score in those with high depressive symptoms and not taking antidepressants. In adjusted models, cognition declined in people taking antidepressants at the same rate as those not taking antidepressants. Results remained consistent across different levels of baseline cognitive function, age, and duration of antidepressant use (prolonged vs short-term). Conclusions Antidepressant use did not modify the course of 6-year cognitive change in this nationally representative sample.