The mesoporous Li3V2(PO4)3@CMK-3 nanocomposite has been firstly synthesized by a sol–gel method. The X-ray diffraction (XRD), transmission electron microscopy (TEM) and nitrogen adsorption–desorption measurements show that the Li3V2(PO4)3@CMK-3 nanocomposite exhibits the pure monoclinic structure and mesoporous morphology. Li3V2(PO4)3 has particle sizes of <50 nm, and are embedded in the mesoporous channels as well as well dispersed on the CMK-3 surface. Electrochemical measurements demonstrate that the Li3V2(PO4)3@CMK-3 nanocomposite shows significantly better rate capability and cycling performance than the bulk Li3V2(PO4)3. In the potential range of 3.0–4.3 V, the Li3V2(PO4)3@CMK-3 nanocomposite delivers high initial discharge capacity of 130.0 mAh g−1 at 0.2 C, and maintain an initial discharge capacity of 119.5 and 107.8 mAh g−1 at 5 C and 10 C, respectively. After 300 cycles, it can still retain a discharge capacity of 95.4 and 73.5 mAh g−1 at 5 C and 10 C, respectively. The good electrochemical performance for the Li3V2(PO4)3@CMK-3 nanocomposite are related to the special mesoporous structure, nanosized particles, and the existence of conductive carbon matrix, thus leading to improvement in electron and lithium ion diffusivity.