In many ways, 1949 was a notable season for Gumpert. Even though he posted a losing, 13–16 record for a team that went only 63–91, he set personal bests in victories, innings pitched (234), complete games (18), and shutouts (three). His elbow miseries persisted, however, and Gumpert led all American League hurlers in most home runs allowed (22). The season saw Gumpert split time between the rotation and the bullpen, and his performance fell off: he won only five games and his ERA climbed to 4.75. He rebounded in ; he won nine of 17 decisions for the ChiSox and a fast start earned him a spot on the 1951 American League All-Star team. However, he didn't appear in the midsummer classic, played July 10 at Detroit's Briggs Stadium and an 8–3 romp for the rival National League. On May 1, 1951, Gumpert became part of baseball history when he allowed Mickey Mantle's first home run, during a relief appearance at Comiskey Park. His value improved by his All-Star selection, Gumpert was traded that offseason, on November 13, 1951, along with outfielder Don Lenhardt to the Boston Red Sox for infielder Mel Hoderlein and pitcher Chuck Stobbs. After getting into ten games for the Red Sox, with one start, he was traded again on June 10, this time to the Washington Senators with fellow hurler Walt Masterson for Sid Hudson, another right-hander. He finished the 1952 campaign with Washington, working in 20 games, with 12 starting assignments. His final MLB appearance came September 23, 1952, a scoreless, two-inning relief stint against the Athletics at Shibe Park. Over his ten-season career, which spanned 17 years, Gumpert won 51 games, lost 59, and compiled a 4.17 earned run average. Of his 261 games pitched, 113 were starts and 148 came out of the bullpen, and he was credited with 47 complete games, six shutouts and seven saves. He allowed 1,099 hits and 346 bases on balls, with 352 strikeouts, in 1,052 innings pitched. Gumpert then played three more seasons, 1953–1955, at the highest levels of minor league baseball before hanging up his glove. Gumpert remained in baseball for another 39 years, managing in the Yankees' farm system (1956–1958 and 1960) and working as a minor league pitching instructor. He served as a temporary member of the Bombers' coaching staff, when, in April, Bill Dickey stepped down due to ill health; Gumpert eventually ceded his coaching post to Charlie Keller.