Complex aluminum hydrides have been widely studied as potential hydrogen storage materials but also, for some time now, for electrochemical applications. This review summarizes the crystal structures of alkali and alkaline earth aluminum hydrides and correlates structure properties with physical and chemical properties of the hydride compounds. The crystal structures of the alkali metal aluminum hydrides change significantly during the stepwise dehydrogenation. The general pathway follows a transformation of structures built of isolated [AlH4]− tetrahedra to structures built of isolated [AlH6]3− octahedra. The crystal structure relations in the group of alkaline earth metal aluminum hydrides are much more complicated than those of the alkali metal aluminum hydrides. The structures of the alkaline earth metal aluminum hydrides consist of isolated tetrahedra but the intermediate structures exhibit chains of corner-shared octahedra. The coordination numbers within the alkali metal group increase with cation sizes which goes along with an increase of the decomposition temperatures of the primary hydrides. Alkaline earth metal hydrides have higher coordination numbers but decompose at slightly lower temperatures than their alkali metal counterparts. The decomposition pathways of alkaline metal aluminum hydrides have not been studied in all cases and require future research.