gifford was born in edinburgh on 29 february 1820 to katherine ann (née west) (1786–1873) and james gifford (1780–1862), an affluent grocer and treasurer of the merchant company of edinburgh. his twin brother was john gifford (1820–1895). his childhood home was at 22 union place in the east end of the new town.