Li10GeP2S12 (LGPS) is incorporated into polyethylene oxide (PEO) matrix to fabricate composite solid polymer electrolyte (SPE) membranes. The lithium ion conductivities of as-prepared composite membranes are evaluated, and the optimal composite membrane exhibits a maximum ionic conductivity of 1.21 × 10−3 S cm−1 at 80 °C and an electrochemical window of 0–5.7 V. The phase transition behaviors for electrolytes are characterized by DSC, and the possible reasons for their enhanced ionic conductivities are discussed. The LGPS microparticles, acting as active fillers incorporation into the PEO matrix, have a positive effect on the ionic conductivity, lithium ion transference number and electrochemical stabilities. In addition, two kinds of all-solid-state lithium batteries (LiFeO4/SPE/Li and LiCoO2/SPE/Li) are fabricated to demonstrate the good compatibility between this new SPE membrane and different electrodes. And the LiFePO4/Li battery exhibits fascinating electrochemical performance with high capacity retention (92.5% after 50 cycles at 60 °C) and attractive capacities of 158, 148, 138 and 99 mAh g−1 at current rates of 0.1 C, 0.2 C, 0.5 C and 1 C at 60 °C, respectively. It is demonstrated that this new composite SPE should be a promising electrolyte applied in solid state batteries based on lithium metal electrode.