in august 2006 he signed for stevenage borough, scoring his first goal for them in a 1–1 draw at exeter city. with jeff kenna, guppy became the first player to play at both the new wembley and the old wembley. this was achieved on 12 may 2007 when kidderminster harriers played borough in the 2007 final of the fa trophy at the new wembley stadium. borough won 3–2 despite being 2–0 down at half time; craig dobson, who replaced guppy on 63 minutes, scored the equalizing goal. due to stevenage winning, guppy became the first ever person to win medals at both the old and new wembley stadium.