bertie was the eldest son of montagu bertie, 2nd earl of lindsey by his second wife, bridget bertie (née wray), 4th baroness norreys. the earl of lindsey was a royalist of impeccable credentials, and the head of an influential lincolnshire family. while james' position as a sixth son might have limited his prospects, upon his mother's death c.1657, he became the 5th baron norreys of rycote. together with the peerage, he inherited from his mother considerable estates, including the manors of rycote, albury, wendlebury, chesterton, dorchester, thame, beckley and horton in oxfordshire, and wytham, cumnor and frilsham, all then in berkshire. these extensive estates would provide him with a base of political power in oxfordshire. the marriage in the early 1650s of his half-sister bridget to sir thomas osborne would also prove of great advantage to the berties, as osborne rose to become duke of leeds and one of the eminent politicians of the later stuart era. osborne was on particularly good terms with lord norreys, and often stayed and hunted with him at rycote.