He arrived in September 1817 with a cargo of "useful plants" including two olives from Provence. Five years later Macarthur made an equally favourable impression on Commissioner John Bigge who was in NSW to investigate all aspects of colonial administration, including the development of agriculture and trade. Bigge observed that Macarthur's olive trees had assimilated well to the climate and that olive oil from NSW could prove a successful export product. The settlement's British cultural background and lack of experience growing olives (commercially) and competition from the wool production industry kept olive production in the shade. Two olive trees remain at Elizabeth Farm on the front (northern) lawn. It is unclear whether these date from 1805 or 1817 introductions. However they appear to be the earliest olive trees surviving in Australia and thus, despite Suttor's introduction in 1800, Macarthur could be credited with introducing or at least successfully establishing olives in Australia, leading (at least in the late 20th century) to a growing agricultural industry. In the 1820s Macarthur added a separate two storey section to the rear, and built stables. The house additions used open planning with French doors leading to the verandahs and gardens. A few years later the verandahs may have been remodelled. A Doric columned north verandah in contrast to the treillage of the eastern verandah was added during 1826. Further extensions were made during 1826 and 1827, designed by Henry Cooper. Refacing of the servants quarters, possibly by John Verge, took place sometime around 1833. In this year John Macarthur moved to their property at Camden and died there in 1834. In 1872 Edward Macarthur died leaving Elizabeth Farm to his niece Elizabeth Onslow, the daughter of his brother James, but allowing his wife Sarah a lifetime interest in it. The Macarthur family sold Elizabeth Farm Estate in 1881. From 1852 Elizabeth Farm was occupied by various tenants and agents, including William Billyard, Crown Solicitor of NSW from 1875 until 1883. The new owner, Septimus Stephen, subdivided the land and put the house block up for auction. There were further subdivisions in 1884. The house was leased variously as a boarding house and glue factory. It was purchased as a house and six acres by William and Elizabeth Swann and family in 1904 for the land value only. The house was in a state of dilapidation. They immediately proceeded to have it cleansed, disinfected and repaired.