The residence was built in the 1820s, during the reign of Sultan Moulay Abd ar-Rahman, by Sulayman as-Siyadmi, a qaid of the Chiadma tribe. Moulay Abd ar-Rahman's son and successor, Muhammad IV, confiscated the residence and gifted it to his brother Ali (Moulay Ali), after whom the residence is now named. During the French Protectorate in Morocco (1912-1956), it was used as the residence of various military officials.