In the present paper, the hierarchical three-dimensional Fe3O4@porous carbon matrix (PCM) and Fe3O4@PCM/graphene are synthesized by a facile annealing of Mil-53(Fe) templates. Their electrochemical performances are evaluated as anode materials for lithium ion batteries. In comparison with Fe3O4@PCM, the Fe3O4@PCM/graphene electrode exhibits a much better cycling performance with the reversible capacity high up to 1000 mAh g−1 after 500 cycles at 1 C. The improved performance can be ascribed to the high conductivity and structure stability provided by the unique graphene based hierarchical three-dimensional nanostructure.