 His mother was quite wealthy as her father eventually became the sole heir to estates held by James Hamilton, 1st Earl of Clanbrassil, whose male line had died out with his son.In 1799, James' father died, leaving him to inherit the baronetcy as his elder brother, Robert Blackwood, had been killed in 1785 in a fall from his horse.Although commonly referred to as Anglo-Irish, the Blackwoods are a Scottish family.