bertie was the third son of willoughby bertie, 4th earl of abingdon and charlotte warren, a daughter of peter warren (royal navy officer) and a descendant of the schuyler family, the van cortlandt family, and the delancey family of british north america. as his two elder brothers predeceased their father, on the latter's death on 26 september 1799 he succeeded him as 5th earl of abingdon. his father died insolvent, having sold off much of the estates at westbury, wiltshire, but he retained control of the burgages and hence the disposal of the parliamentary seats for westbury. the 4th earl had sold the nomination of members there since 1786 to raise money; his trustees and successor continued the practice until 1810, when he sold the manor and control of the seats to sir manasseh lopes.