Rechargeable aqueous zinc ion batteries (ZIBs) have many advantages such as high abundance and low cost of Zn, as well as safer battery chemistry in aqueous electrolytes. Cryptomelane-type manganese dioxides (α-MnO2s) are promising cathodes for ZIBs due to their high theoretical capacity, voltage, and abundance. However, α-MnO2 suffers from severe capacity fading as the battery is put through extended charge-discharge cycles. Here, we show that galvanostatic cycling performance of α-MnO2 can significantly be improved at ZnSO4 concentrations below 0.2 M. Low-electrolyte concentrations demonstrated higher discharge capacities and capacity retentions at long battery cycling tests. At 0.1 M ZnSO4, α-MnO2 delivered 62 and 103 mAh/g capacities with 0.3 A/g rate at the 1st and 100th cycles, respectively (166% retention). At a higher charge/discharge rate of 0.9 A/g, α-MnO2 delivered 85 mAh/g capacity at the 500th cycle, where the retention was 167% with respect to the 1st cycle in 0.1 M ZnSO4. On the other hand, at high electrolyte concentrations (>1.0 M), the capacity retentions were below 30% and discharge capacities were below 60 mAh/g after 100 cycles. Better electrochemical performance of α-MnO2 at low electrolyte concentrations is attributed to the suppression of cathode dissolution and the mitigation of the formation of impurity phases such as ZnMn2O4 and ZnMn3O7·xH2O.