A novel composite of magnetite (Fe3O4) nanoparticles (NPs) grown on reduced graphene oxide (rGO) has been synthesized by a facile hydrothermal technique without any surfactants or templates. In this method, the growth of Fe3O4 NPs and the reduction of graphene oxide (GO) are completed in one single step. Moreover, we have prepared Fe3O4 microcubes and Fe3O4/carbon nanotubes (CNTs) composite under the same condition. The electrochemical properties of the as-prepared samples are investigated as advanced electrode materials for supercapacitors. It is found that the Fe3O4/rGO nanocomposite displays much higher specific capacitances and better cycle stability than those of pure Fe3O4 and Fe3O4/CNTs composite. Specifically, it exhibits a high specific discharge capacitance of 220.1 F g−1 at 0.5 A g−1 and remains stable after 3000 charge/discharge cycles. The improvement of the electrochemical performances of Fe3O4/rGO may be attributed to the chemical interaction between rGO and Fe3O4, lower agglomeration and smaller particle size of Fe3O4.