 Lord Cawley married Elizabeth Smith, daughter of John Smith and Fanny Robson, in 1876.They had four sons, of whom the three youngest, Harold Cawley MP, John and Oswald Cawley MP, were killed in the First World War, and one daughter, Hilda Cawley.In memory of his three dead sons, Cawley endowed a ward at Ancoats Hospital, Manchester, in 1919 at a cost of £10,000.