Hay was a great-grandson of the 2nd Marquess of Tweeddale, and in 1787 he inherited the titles of his first cousin once-removed, the 6th Marquess. He then became a Burgess of Edinburgh a year later, Lord Lieutenant of Haddingtonshire in 1794, and a Scottish representative peer in 1796. On 18 April 1785, he married Lady Hannah Maitland (a daughter of the 7th Earl of Lauderdale) and they had (with two other unmarried daughters):