He was born, educated and started his career in Fife, but his peak playing years were in the early 1960s with the Glasgow club Rangers, whom he helped to win ten trophies between 1960 and 1965, and where he became known as "Slim Jim". However, he started drinking heavily during a four-month layoff caused by a leg fracture in December 1964, his fitness suffered, and he was transferred to Sunderland in summer 1965. In two and a half years at Sunderland he played 98 games and scored 12 goals, becoming known for drinking himself unconscious the night before a match and playing well the next day.