born in champion, new york, spencer was the son of gordon percival and deborah mallory spencer. he educated at montreal college in canada. after relocating to iowa he engaged in the study of law. during the pike's peak gold rush he briefly relocated to colorado where in november 1859 he founded the town of breckenridge.