His father, James Taylor IV, bought 2700 acres of land in Northern Kentucky from his friend George Muse, which was part of the land Muse had been awarded for his military service in the French and Indian War.