A thin, flexible, and safe composite polymer electrolyte membrane composed of lithium ion conductive nanofiber framework and polymer electrolyte matrix is demonstrated as an innovative solid-state electrolyte for potential use in high-performance rechargeable all-solid-state lithium ion battery. Due to this unique structural integrity, the composite solid polymer electrolyte indicates excellent mechanical stability, remarkable improvements in the ion conductivity (approximately 10−4 S cm−1 at room temperature) and the lithium ion transference number (approximately 0.5), and the significant battery performance in terms of good charge-discharge cycling behavior and moderate rate capability. Finally, a high-voltage multi-stacked battery with the solid polymer composite electrolytes is developed and can readily produce a doubled cell potential of 6.8 V, promising future batteries operated at high-voltage with high energy densities.