He was widowed in April 1871, and in January 1873 remarried to Janet Marian Snodgrass, the daughter of the Victorian pastoralist and politician Peter Snodgrass. He had two sons and two daughters by his first wife, and another four sons and four daughters by his second; he was survived by Janet and nine of his children. Clarke was a household name in Victoria. He made a few large donations but his help could constantly be relied on by hospitals, charitable institutions, and agricultural and other societies. He divided one of his estates into small holdings and was a model landlord, and he showed much foresight in allying science with agriculture by employing MacIvor as a lecturer. His second wife, Janet, who had been associated with him in philanthropic movements, kept up her interest in them, especially in all matters relating to women, until her death on 28 April 1909. One of their sons, Sir Francis Grenville Clarke, went into politics and was a member of several Victorian ministries. He became president of the Legislative Council in 1923 and held that position for almost 20 years and was created K.B.E. in 1926. His son Rupert succeeded him as the 2nd Baronet. The baronetcy of Clarke of Rupertswood is one of only two active hereditary titles in an Australian family. His second son, Ernest Edward Dowling Clarke (1869–1941), was a noted racehorse owner, closely associated with trainer James Scobie.