In this study, we propose a new filler material (Zn4O(1,4-benzenedicarboxylate)3 metal-organic framework, MOF-5), which is successfully prepared and incorporated in poly(ethylene oxide) (PEO) based composite polymer electrolyte (CPE) by an in-situ method for improving electrochemical properties in lithium ion battery. A highest ionic conductivity of 3.16 × 10−5 S cm−1 at 25 °C for the composition of PEO-LiN(SO2CF3)2 (ethylene oxide (EO):Li = 10:1)/10 wt% MOF-5 is obtained. The anodic steady window of the electrolyte is 4.57 V at 60 °C, which is decided by the decomposition voltage of MOF-5. The interfacial resistances of Li/CPE/Li cells become more stable upon storage compared for that without MOF-5. The reversible capacities of LiFePO4/Li solid cells increase at both 60 °C and 80 °C due to the decrease of the cell polarization. In addition, the capacity retention is 45% after 100 cycles at 1 C rate at 80 °C, while there is a sharp decay after 30 cycles in the cells without MOF-5 filling. These results show that MOF-5 is useful to enhance electrochemical properties of composite polymer electrolyte for solid lithium ion battery.