Prolific run-scoring at age group levels saw him getting elevated to the senior team squad of Hyderabad. He made his first-class debut for Hyderabad in January 2002, at the age of 16, during the 2001–02 Ranji Trophy. Batting at 4, he scored 33 in that match, his only first-class appearance of the season. Later that year, he was made captain of India Under-17s and also made appearances for India Under-19s. On India Under-19 tour of England, his maiden Under-19 assignment, Rayudu batted as an opener and made a total of 291 runs from 3 innings and finished as the highest run-getter of the series which India swept 3–0. His best efforts of the series came in the third ODI in which he scored a 169-ball 177 to single-handedly take his team to the target of 304 from a precarious position of 137 for 6. Rayudu appeared in all matches for Hyderabad in the 2002–03 Ranji Trophy and amassed a total of 698 runs at an average of 69.80 to finish third on the leading run-scorers list. During the tournament, playing only his third Ranji game, Rayudu scored 210 and 159 not out against Andhra in the same match and "pressed his claims for an India cap". He became the youngest player in the history of Ranji Trophy to score a double century and a century in the same match. In 2003, Rayudu played extensively for India A on its West Indies and England tours. He averaged 87 on the England tour following which experts opined that he is "sure to play for India in the near future". "ESPNcricinfo" wrote about the 17-year-old: "Touted as the next big hope for Indian middle-order batting, Rayudu has had the critics gushing about his scintillating strokeplay and his composure under pressure." However he had a disappointing stint with India Seniors at the Challenger Trophy in September 2003, where his perceived weakness against the short ball was exploited. He then failed to make an impression in the Emerging Players Tournament after which he played four matches in the following Ranji season with an average of over 54. He captained India Under-19s to the title at the Asia Under-19 tournament, with an unbeaten fifty in the final. Rayudu captained the Indian team at the 2004 ICC Under-19 Cricket World Cup in Bangladesh where India finished as semifinalists.