Koller played his entire career for Swiss club Grasshoppers Zürich. In these 24 years he won seven Swiss championships and five Swiss cups. Koller was capped 56 times for the Swiss national team, scoring 3 goals.