Holland arrived at Fort Amherst in summer 1764 with his family but realized the fortification was unsuitable for his use and chose instead to build a dwelling approximately 1 kilometers south of the garrison at a location he named Observation Cove (present-day Holland Cove).