 Handley joined Aston Villa in September 1957 from Goole Town, but had to wait until the 1959–60 season for his league debut.He moved to Torquay United in September 1960, scoring 32 goals in 82 games for Eric Webber's side before dropping into non-league football.In February 1966, he joined Rochdale from Bridgwater Town, but played only three times before leaving league football once more.