Atkinson indulged his passion for horticulture by often planting rare species, for example in the gardens of the villa which he built for himself about 1818 at Grove End in Paddington and, later, in the 170 acres estate he purchased about 1830 at Silvermere, near Cobham, Surrey, where he also built himself a house.