Perry played semi-professionally in Alpine, Texas at Kokernot Field in the early 1950s for the Alpine Cowboys. Bobby Biedermann was his catcher and roommate. Perry was signed by the San Francisco Giants on June 3, 1958 for $90,000, which was a big contract at the time. He spent 1958 with the St. Cloud, Minnesota team in Class A Northern League, compiling a 9–5 record and a 2.39 ERA. In 1959 he was promoted to the Class AA Corpus Christi Giants, where he posted a less impressive 10–11 record and 4.05 ERA. He remained with the team as they became the Rio Grande Valley Giants in 1960, and an improved ERA of 2.82 earned him a promotion to the Class AAA Tacoma Giants for the 1961 season. At Tacoma, he led the Pacific Coast League in wins and inning pitched in 1961. He had a brief call-up to the Major Leagues in 1962, making his debut on April 14 against the Cincinnati Reds. He appeared in 13 games in 1962, but had a 5.23 ERA and was sent back down to Tacoma for the remainder of the year. With the addition of Perry, Bill James called that 1962 Tacoma squad, which featured numerous future major league players, the best minor league lineup of the 1960s. After his brief call-up in 1962, Perry joined the Giants in 1963 to work mostly as a relief pitcher that year, posting a mediocre 4.03 ERA in 31 appearances. Nevertheless, in 1964 he was given the opportunity to join the starting rotation, finishing with a 2.75 ERA and a 12–11 record, both second best for the Giants that year behind Juan Marichal. In 1965 his record was 8–12, and with two full seasons as a starter, his 24–30 record attracted little national attention. Perry's breakout season came in 1966 with a tremendous start, going 20–2 into August. Perry and Marichal became known as a "1–2 punch" to rival the famous Koufax/Drysdale combination of the Los Angeles Dodgers. While Marichal was NL Player of the Month in May, Perry was so named in June (5-0, 0.90 ERA, 31 SO). He played in his first All-Star game, but after August, he slumped the rest of the season, finishing 21–8, and the Giants finished second to the Dodgers. Marichal missed much of the 1967 season with a leg injury, and Perry was thrust into the role of team ace.