the third construction, a castellated mansion, was ordered by lady catherine darnley, duchess of buckingham, illegitimate daughter of james ii of england and then wife of john sheffield, 1st duke of buckingham and normanby. the property passed to the phipps family in 1718, when the duchess's daughter lady catherine annesley married william phipps. a summer house was also built on the grounds, according to legend on the spot of a hermitage which william de percy established in 1150. in 1858 duleep singh, the last maharajah of the punjab, took a lease on mulgrave castle.