This paper proposes an adaptive mode switch strategy (AMSS) based on simulated annealing (SA) optimization of a multi-mode hybrid energy storage system (HESS) for electric vehicles. The proposed SA-AMSS is derived from a rule-based strategy to achieve the adaptive mode selection and energy management optimization. To improve the overall system efficiency of the multi-mode HESS, the state of charge (SOC) level of the supercapacitor (SC), the power level and the component efficiencies are discussed. On this basis, the objective function for the AMSS is established, focusing on selecting the most suitable mode. Furthermore, to accomplish a global energy management optimization based on the driving cycles, the SA approach is introduced into the optimizations of the reference SC SOC and battery power, rather than the direct power distribution optimization between the battery and SC. The AMSS is implemented based on the SA optimization. Simulations and experiments are presented to verify the effectiveness of the SA-AMSS for the multi-mode HESS. Results show that the SA-AMSS can not only reduce the frequency of the mode switching, but also avoid the sudden excessive power output of the battery. The SC can respond to all peak power demands and absorb all the braking energy. So the SA-AMSS is very flexible and effective, and the battery safety can be guaranteed. Compared with the rule-based strategy, the overall system efficiency of the multi-mode HESS is significantly improved.