Phipps purchased an old 160 acres Quaker farm on Long Island where he built a large mansion with magnificent gardens that, following his death, became a non-profit organization that today is known as Old Westbury Gardens and is open to the public.