Born in Wilkes-Barre, Pennsylvania, he came with his parents to Green Bay, Michigan Territory. Arndt graduated from Rutgers College, studied law in Easton, Pennsylvania and was admitted to the bar. In 1836, Arndt moved back to Green Bay and was admitted to the bar in Michigan Territory.