To improve the ionic conductivity as well as enhance the mechanical strength of the gel polymer electrolyte, poly(vinylidene fluoride-hexafluoroprolene) (P(VDF-HFP))-based composite polymer electrolyte (CPE) membranes doped with the organic-inorganic hybrid particles poly(methyl methacrylate) -ZrO2 (PMMA-ZrO2) are prepared by phase inversion method, in which PMMA is successfully grafted onto the surface of the homemade nano-ZrO2 particles via in situ polymerization confirmed by FT-IR. XRD and DSC patterns show adding PMMA-ZrO2 particles into P(VDF-HFP) can significantly decrease the crystallinity of the CPE membrane. The CPE membrane doped with 5 wt % PMMA-ZrO2 particles can not only present a homogeneous surface with abundant interconnected micro-pores, but maintain its initial shape after thermal exposure at 160 °C for 1 h, in which the ionic conductivity and lithium ion transference number at room temperature can reach to 3.59 × 10−3 S cm−1 and 0.41, respectively. The fitting results of the EIS plots indicate the doped PMMA-ZrO2 particles can significantly lower the interface resistance and promote lithium ions diffusion rate. The Li/CPE-sPZ/LiCoO2 and Li/CPE-sPZ/Graphite coin cells can deliver excellent rate and cycling performance. Those results suggest the P(VDF-HFP)-based CPE doped with 5 wt % PMMA-ZrO2 particles can become an exciting potential candidate as polymer electrolyte for the lithium ion battery.