 He immigrated to Georgia at the age of seventeen where he constructed shipmasts for the French government with his brother John.Together, they built a steamboat to work the Chattahoochee, but suffered a major setback when it sank, leaving Gilmer almost penniless.In the late 1840s, he migrated to Orange, Texas where he entered the shipbuilding business with his cousin George C. Gilmer.