It was built in 1938-39 by real estate developer Charles V. Paterno on the 7 acres site of what had been the castle that was his residence, and was designed by George F. Pelham, Jr., whose father, George F. Pelham, had designed the nearby Hudson View Gardens.