Mesoporous silica SBA-15 was surface-modified by γ-glycidoxypropyltrimethoxy silane (GPTMS), and novel poly(ethylene oxide) (PEO)-based composite polymer electrolytes (CPE) using the silane-modified SBA-15 (SBA-15-GPTMS) as filler were prepared and characterized. The results of the low-angle X-ray diffraction (XRD) patterns and Fourier-transform infrared (FT-IR) spectroscopy indicated that GPTMS has been successfully attached to the surface of SBA-15 with a high degree of mesoscopic hexagonal pore structure. The incorporation of SBA-15-GPTMS in the PEO–LiClO4 matrix effectively reduced the PEO crystallinity and obviously improved the conductivity and electrochemical stability of the CPEs. The CPE with 10wt.% SBA-15-GPTMS provided the highest conductivity among all the tested CPEs, about 2–3 orders of magnitude higher than that of the PEO–LiClO4 matrix below the melting temperature of PEO. The reasons that the CPEs using SBA-15-GPTMS as filler showed higher conductivity than that with SBA-15 were discussed.