Endocrine therapy in the setting of breast cancer has undoubtedly advanced clinical outcomes in this disease, but treatment with endocrine therapy is accompanied by a wide spectrum of side effects. It is of prime importance to understand and characterize these toxicities to facilitate clinical decision-making. Somewhat surprisingly, there is a relative paucity of data pertaining to cognitive changes associated with endocrine therapy. In this article we review cognitive associated with two classes of endocrine therapy: (1) selective estrogen receptor modulators (SERMs; tamoxifen and raloxifene) and (2) aromatase inhibitors (AIs; anastrozole, letrozole, and exemestane). Companion studies to the Multiple Outcome of Raloxifene Evaluation (MORE), the Study of Tamoxifen and Raloxifene (STAR) and National Surgical Adjuvant Breast and Bowel Project (NSABP) P-1 trials provide relevant data to understand the effect of SERMs on cognition. In contrast, substudies of the Arimidex, Tamoxifen Alone or in Combination (ATAC), Tamoxifen and Exemestane Adjuvant Multinational (TEAM) and Breast International Group (BIG) 1-98 trials juxtapose cognitive effects of AIs against those of tamoxifen. These and other studies are examined herein to provide a comprehensive overview of the effect of endocrine therapy on cognition.