filley was born on may 23, 1806 in bloomfield, connecticut. he was the eldest of six children, five sons and one daughter, born to oliver filley and annis (née humphrey) filley. his siblings included marcus lucius filley, jay humphrey filley, joseph earl filley, giles franklin filley, jennette annis filley and john eldridge filley, who all became prominent.