 Binary composites comprised of metal oxides and graphene with novel microstructure prepared via a simple, rapid and high-efficiency strategy is of key significance for the development of high-performance supercapacitor. In this work, Co3O4 nanoparticles with average size of 10 nm were in situ grown on reduced graphene oxide (RGO) nanosheets via a time-saving sonochemical treatment, to yield Co3O4/RGO composites with a unique sandwich structure. The RGO skeleton supported the uniform growth of Co3O4 nanoparticles onto it, inhibiting the restacking of graphene nanosheets and affording a large surface area for more effective ion accessibility. Moreover, the excellent electrical conductivity of RGO network ensured the fast electron transfer at Co3O4/RGO interfaces to facilitate the pseudo-capacitive reaction of Co3O4. As a supercapacitor electrode, Co3O4/graphene composite exhibited a high-specific capacitance of 276.6 F g−1 at current density of 0.5 A g−1 and retained 210 F g−1 at 10 A g−1, indicating an excellent rate capability. Furthermore, the composite electrode demonstrated a good cycling stability with 92.4% capacitance retention after 5000 cycles at 5 A g−1. The superior capacitance performance of the composite electrode was attributed to the pseudo-capacitance effect of Co3O4 nanoparticles combined with the electrical double-layer capacitor contribution of RGO nanosheets.