In order to meet the requirements of high specific energy and high specific power together and extend the service life of the energy storage system in temperature abusive conditions, a multi-power configuration with high specific energy lithium-ion battery and high specific power ultracapacitor is the best choice for the all-climate electric vehicle (ACEV). Aiming at real-time power management of a hybrid energy storage system (HESS), three power management strategies, which are respectively based on rules, dynamic programming algorithm, and real-time reinforcement learning algorithm, have been systematically compared in this study. To verify the performance of the control strategies, the hardware-in-loop (HIL) simulation test platform based on xPC Target has been built. The results show that the real-time power management strategy based on reinforcement learning algorithm is superior to the others. This strategy can reduce the charge and discharge ratio of the battery pack, which extends the life of battery pack and improves the efficiency of the system.