Aim This study aimed to ascertain if performance on the Timed Up and Go (TUG) test is associated with indicators of brain volume and cognitive functions among community-dwelling older adults with normal cognition or mild cognitive impairment. Methods Participants were 80 community-dwelling older adults aged 65–89years (44 men, 36 women), including 20 with mild cognitive impairment. Participants completed the TUG and a battery of cognitive assessments, including the Mini-Mental State Examination (MMSE), the Logical Memory I and II (LM-I, LM-II) subtests of the Wechsler Memory Scale-Revised; and the Trail Making Test A and B (TMT-A, TMT-B). Bilateral, right- and left-side medial temporal area atrophy as well as whole gray and white matter indices were determined with the Voxel-based Specific Regional Analysis System for Alzheimer's Disease. We divided participants into three groups based on TUG performance: “better” (≤6.9s); “normal” (7–10s); and “poor” (≥10.1s). Results Worse TMT-A and TMT-B performance showed significant independent associations with worse TUG performance (P<0.05, P<0.01 for trend, respectively). After adjusting for covariates, severe atrophy of bilateral, right-, and left-side medial temporal areas were significantly independently associated with worse TUG performance (P<0.05 for trend). However, no significant associations were found between MMSE, LM-I, LM-II, whole gray and white matter indices, and TUG performance. Conclusions Worse TUG performance is related to poor performance on TMT-A and TMT-B, and is independently associated with severe medial temporal area atrophy in community-dwelling older adults.