This paper presents the preparation and characterization of a series of PEO-PMMA composites containing different lithium salts (LiClO4 or LiTFSI) with or without nano-Al2O3. Nano-Al2O3 is added to the polymer electrolytes to alter the morphology, ionic conductivity, interfacial and thermal shrinkage properties. Nano-Al2O3 can effectively improve the stability of the polymer electrolyte membrane interface. The tensile strength of the PEO-PMMA-LiTFSI-Al2O3 electrolyte membrane is 3.24MPa at room temperature. The study shows that the ion conductivities at room temperature for PEO-PMMA-LiTFSI (EO/Li+ =10) and PEO-PMMA-LiTFSI-Al2O3 (EO/Li+ =10) are 6.71×10−7 S/cm and 9.39×10−7 S/cm, respectively. A good dimensional stability was observed for PEO-PMMA-LiTFSI with a thermal shrinkage of 8.7%. The electrochemical windows of PEO-PMMA-LiTFSI and PEO-PMMA-LiTFSI-Al2O3 are all stable up to 4.6V versus Li+/Li, which makes them good electrolyte candidates of lithium ion batteries with high-potential cathodes.