 Charlton was a member of North Steyne Surf Life Saving club in his teens, before transferring to Manly Life Saving Club in the mid 1920s

Charlton first came to public attention in 1921 when he won a 440 yd freestyle race in the open division at a New South Wales Swimming Association competition in 5 min 45 s. It was his youth that led to his nickname "Boy".In 1922 Bill Harris, the bronze medallist in the 100m freestyle at the 1920 Summer Olympics, came to Australia from Honolulu to compete against the likes of Frank Beaurepaire and Moss Christie.Charlton defeated Harris at the New South Wales Championships, winning the 440 yd in 5 min 22.4 s. He then set a world record of 11m 5.4s in the 880yd event, as well as winning the one mile race in 23 min 43.2 s. Charlton used a trudgen stroke which embodied characteristics of the modern crawl stroke, which was at the time in its infancy.