Electric vehicles are becoming a more popular form of transportation, however their limited range has proven problematic. Battery-exchange stations allow the vehicles to swap batteries during their trip, but if a vehicle arrives at a station without a full battery available it may have to wait an extended period of time to get one. The vehicles can be routed so that they avoid stations without available batteries or to keep batteries available for other vehicles that need them in the future. The batteries can also be reserved during the routing process so that each vehicle is ensured the battery it plans to use is available. This paper provides a method of online routing of electric vehicles and making battery reservations that minimizes the average delay of the all vehicles by occasionally detouring them to the benefit of future ones. The system is modeled as a Markov chance-decision process and the optimal policy is approximated using the approximate dynamic programming technique of temporal differencing with linear models. The solution algorithm provides a quick way for vehicles to be routed using onboard vehicle software connected to a central computer. Computational results for the algorithm are provided using data on the Arizona highway network.