A graphene loading heterogeneous hydrated forms iron based fluoride (abbreviated as FeF3·xH2O/G) nanocomposite is successfully designed and synthesized for the first time by a sol–gel method. It found that the FeF3·xH2O nanoparticles distribute randomly on the surface of the graphene, stacking together to form a nanocomposite with high specific surface and abundant mesporous structure. The FeF3·xH2O was consisted of FeF3·3H2O and FeF2.5·0.5H2O with pyrochlore phase structure and FeF3·0.33H2O with hexagonal-tungsten-bronze-type structure (HTB). The FeF3·xH2O/G was used as cathode materials of rechargeable lithium/sodium batteries, respectively. It has been found that it can deliver a large reversible capacity exceeding 200 mAh g−1 and excellent cyclic performance with a residual capacity of 183 mAh g−1 after 100 cycles at 0.2C and 149 mAh g−1 after 200 cycles at 1C, especially, an outstanding rate performance exceeding 130 mAh g−1 at 5C in the voltage range of 1.5–4.5 V for Li-ion batteries. Moreover, when FeF3·xH2O/G is used as cathode material of Na-ion batteries, it exhibits also a high reversible capacity of 101 mAh g−1 after 30 cycles in the voltage range of 1.0–4.0 V at 0.1C. Therefore, FeF3·xH2O/G will a promising cathode material for high-performance lithium/sodium ion batteries.