Understanding the genetic factors underlying brain structural connectivity is a major challenge in imaging genetics. Here, we present results from genome-wide association studies (GWASs) of whole-brain white matter (WM) fractional anisotropy (FA), an index of microstructural coherence measured using diffusion tensor imaging. Data from independent GWASs of 355 Swedish and 250 Norwegian healthy adults were integrated by meta-analysis to enhance power. Complementary GWASs on behavioral data reflecting processing speed, which is related to microstructural properties of WM pathways, were performed and integrated with WM FA results via multimodal analysis to identify shared genetic associations. One locus on chromosome 17 (rs145994492) showed genome-wide significant association with WM FA (meta P value = 1.87 × 10−08). Suggestive associations (Meta P value <1 × 10−06) were observed for 12 loci, including one containing ZFPM2 (lowest meta P value = 7.44 × 10−08). This locus was also implicated in multimodal analysis of WM FA and processing speed (lowest Fisher P value = 8.56 × 10−07). ZFPM2 is relevant in specification of corticothalamic neurons during brain development. Analysis of SNPs associated with processing speed revealed association with a locus that included SSPO (lowest meta P value = 4.37 × 10−08), which has been linked to commissural axon growth. An intergenic SNP (rs183854424) 14 kb downstream of CSMD1, which is implicated in schizophrenia, showed suggestive evidence of association in the WM FA meta-analysis (meta P value = 1.43 × 10−07) and the multimodal analysis (Fisher P value = 1 × 10−07). These findings provide novel data on the genetics of WM pathways and processing speed, and highlight a role of ZFPM2 and CSMD1 in information processing in the brain.