Synthesis of materials with desirable nanostructures is a hot research topic owing to their enhanced performances in contrast to the bulk counterparts. Herein, dumbbell-shaped cobalt carbonate (CoCO3) nano architectures and the bulk counterpart of CoCO3 rhombohedra are prepared via a facile hydrothermal route in the presence and absence of ascorbic acid (AA), respectively. By comparison, it has been found that: the addition of AA in the hydrothermal crystallization system changes the shape of the building blocks from Co2CO3(OH)2 nanosheets to CoCO3 nanoparticles, and then further influences the final configuration of the products. When applied as anodes of lithium ion batteries, CoCO3 dumbbells deliver a 100th capacity of 1042 mAh g−1 at 200 mA g−1 and even exhibit a long-term value of 824 mAh g−1 over 500 cycles at 1000 mA g−1, which are much higher than the rhombohedral counterparts with corresponding 540 and 481 mAh g−1 respectively. The much higher capacity, better cycling stability and enhanced rate performance of CoCO3 dumbbells can be attributed to the higher specific surface area, smaller charge transport resistance and better structure stability resulting from the slight doping (∼4.6 wt%) of AA, and also relate with a novel lithium storage mechanism in CoCO3.