He was described as ‘a lover of letters and liberal knowledge, affectionate and affable to a numerous family’. He had two daughters by his first wife and five sons and ten daughters by his second wife. His eldest son, Peter succeeded to Clarendon but it eventually passed to the descendants of his eldest daughter, Selina.