Nano-sized fillers in a polymer matrix with good distribution can play a positive role in improving polymer electrolytes in the aspects of ionic conductivity, mechanical property and electrochemical performance of Li-ion cells. Herein, polyethylene oxide (PEO)/Li3PS4 hybrid polymer electrolyte is prepared via a new in-situ approach. The ionic conductivities of the novel hybrid electrolytes with variable proportions are measured, and the optimal electrolyte of PEO-2%vol Li3PS4 presents a considerable ionic conductivity of 8.01 × 10−4 S cm−1 at 60 °C and an electrochemical window up to 5.1 V. The tests of DSC and EDXS reveal that the Li3PS4 nanoparticles with better distribution, as active fillers scattering in the PEO, exhibit a positive effect on the transference of lithium ion and electrochemical interfacial stabilities. Finally, the assembled solid-state LiFePO4/Li battery presents a decent cycling performance (80.9% retention rate after 325 cycles at 60 °C) and excellent rate capacities with 153, 143, 139 and 127 mAh g−1 at the discharging rate of 0.1 C, 0.2 C, 0.5 C and 1 C at 60 °C. It is fully proved that it is an advanced strategy to preparing the new organic/inorganic hybrid electrolytes for lithium-ion batteries applications.