 One of 19 children, he was the third son and fifth child of Robert Walpole, a member of the local gentry and a Whig politician who represented the borough of Castle Rising in the House of Commons, and his wife Mary Walpole, the daughter and heiress of Sir Geoffrey Burwell of Rougham, Suffolk.Horatio Walpole, 1st Baron Walpole was his younger brother.As a child, Walpole attended a private school at Massingham, Norfolk.