A three-dimensional (3D) self-standing MgMoO4/reduced graphene oxide (rGO) nanosheet arrays (MMO-G NSAs) are fabricated for lithium and sodium ion storage via a facile bottom-up strategy. In the synthesized process, graphene oxide (GO) was intelligently employed as structure-directing agents to guide the interconnection of magnesium molybdate nanosheets (MMO NSs) along the vertical direction and confine the overgrowth of MMO NSs. The formation mechanism of MMO-G NSAs is proposed based on the illustrations of thermodynamic and dynamic driving forces. As an anode material for lithium ion battery, the MMO-G NSAs nanocomposite exhibits enhanced capacity (848.2mAhg−1 at 100mAg−1) than the MMO NSs electrode (582.5mAhg−1 at 100mAg−1). For sodium ion battery, the MMO-G NSAs nanocomposite also exhibits encouraging sodium ion storage capability (201.1mAhg−1 after 300 cycles at 100mAg−1). These results demonstrate that the special array architecture not only improves the charge transfer efficiency during cycling but also facilitates the electrolyte penetration and alleviates the stress induced by volume changes. Furthermore, the morphologies of the solid–electrolyte interphase (SEI) layer at different cycle stages were captured, which reveals that a dense, flexible SEI layer is crucial to the long-life cycling performance.