Background Basic and clinical researches have suggested that type 2 diabetes (T2DM) is associated with cognitive impairment, and diabetes mellitus increases the risk of cognitive impairment and dementia. Recently, some reports found that undercarboxylated osteocalcin (ucOC) could affect brain functions, and decreased in patients with T2DM. We aimed to investigate the association of serum ucOC with cognitive impairment in T2DM patients. Methods A total of 196 male T2DM patients without medications known to affect bone metabolism or history of bone fracture, aged ≥18years were recruited and divided into impaired cognition group and normal cognition group. We use the scores of Minimum Mental State Examination (MMSE) to evaluate the subjects' cognitive function. Detailed cognitive performance was also evaluated by the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Serum ucOC was measured by Enzyme-Linked Immunosorbent Assay (ELISA) kit. Results Compared to male T2DM patients with normal cognition, the mean osteocalcin concentrations were significantly lower in male T2DM patients with impaired cognition (P <0.05). RBANS total and all indexes scores were also lower in patients with impaired cognition (all P <0.05). After adjusted effects of confounding factors, serum ucOC was positively correlated with a variety indexes of RBANS except visuospatial/constructional. Conclusions The serum ucOC is positively correlated with RBANS scores in male T2DM patients. It suggests that serum ucOC may be involved in the development and progression of cognitive dysfunction in T2DM patients.