In addition, large sums of money were spent improving and extending the 'big house' at Caledon (variously known as Caledon House, Caledon Hill and Caledon Castle), and on laying out the richly ornamental demesne and gardens of over 600 acres .