Patients with subcortical ischemic vascular disease (SIVD) may exhibit a high risk of cognitive impairment (CI) by disruption of white matter (WM) integrity. Diffusion tensor imaging (DTI) is recommended as a sensitive method to explore whole brain WM alterations at an asymptomatic stage of the disease, which might be correlated with underlying cognitive disorders. We aim to investigate alterations in WM microstructures and evaluate the relationships between the mean values of diffusion metrics (FA, MD, AD, and RD) and cognitive assessments in SIVD patients. Fifty SIVD patients with (SVCI, N = 25) and without (pre-SVCI, N = 25) cognitive impairments and normal controls (NC, N = 23) underwent DTI and neuropsychological examinations. DTI data were analyzed via TBSS to detect significant changes in WM tracts. Spearman correlation analysis was performed to evaluate relationships between the mean values of diffusion indices and the cognitive assessments. In general, extensive symmetrically altered areas that involved approximately the entire cerebral WM were noted in the pre-SVCI group but were less distinct than that noted in the SVCI group compared with NCs. The genu of corpus callosum exhibited the most damaged WM fiber. Throughout WM, FA was decreased, whereas MD, AD, and RD were increased. Some specific WM tracts in patient groups were significantly correlated with the severity of white matter hyperintensity (WMH), cognitive assessments about executive functions and processing speed. WM integrity has already been damaged at the pre-SVCI stage, which would be associate with future cognitive dysfunction. DTI could potentially establish early biomarkers to detect underlying mechanisms of SIVD.