His parents were immigrants from Denmark. His father, Jens Hans Jeppesen, was an architect and builder trained in Denmark. Elrey grew up on a farm that his father had cleared and homesteaded in Odell, Oregon, before moving to Portland.