Doherty was unhappy with the directors who opposed his plan to secure his future by taking over the Arboretum Hotel and an earlier dispute over FA Cup Final tickets.[13] The "Terriers" boasted a powerful front five of Albert Bateman, Jimmy Glazzard, Alf Whittingham, Doherty and Vic Metcalfe.[14] However a weak defence saw them struggle in the lower reaches of the First Division throughout the 1946–47 season, 1947–48 and 1948–49 seasons.[14] At Leeds Road he scored 33 goals in 83 league appearances, finishing as the "Terriers" top-scorer in his final two seasons under George Stephenson.