Litmanen opened the scoring for Finland
Group A leaders Serbia had to wait nearly an hour to break the deadlock before easing to a 3-0 win over 10-man Armenia.
Marko Pantelic missed a first-half penalty before Dejan Stankovic showed him how it should be done from the spot on 54 minutes.
Danko Lazovic added a second before Rafik Nazaryan was sent off, and Nicola Zigic added a third in injury time.
Belgium's Moussa Dembele scored and was sent off as Belgium beat Azerbaijan 3-0.
Timmy Simons' first-half penalty and a Kevin Vandenbergh chip set up the win.
Substitute Dembele added a third before he saw red for kicking Aslan Kerimov.
Finland captain Jari Litmanen led by example as his team bounced back from an embarrassing draw with Armenia, beating Kazakhstan 2-0.
Birmingham's Mikael Forsell set up the former Liverpool player for the first on 29 minutes, with Sami Hyypia heading another early in the second half.
And two early goals by Ebi Smolarek gave Poland a 2-1 win over Portugal, with Nuno Gomes pulling one back in injury time.
Shevchenko (left) scored Ukraine's second goal
Scotland's fantastic start to their Group B campaign came to a sudden halt as they lost 2-0 to Ukraine in Kiev.
Olexander Kucher's close-range shot broke their resistance late on.
Kenny Miller came close to an equaliser as he hit the crossbar with a speculative effort, and headed a wonderful chance wide on 80 minutes.
But late on Steven Pressley was sent off, shortly before Ukraine captain Andriy Shevchenko wrapped up the points with a penalty.
Elsewhere, World Cup winners Italy eased to a 3-1 win over Georgia.
Daniele De Rossi opened the scoring with a long-range bouncing shot, before George Shashiashvili's shot from outside the box on 24 minutes levelled things.
Dzhaba Kankava was sent off for his second yellow card shortly before Mauro Camoranesi headed Italy's second, and Simone Perrotta made sure of the points.
And David Trezeguet scored twice as World Cup runners-up France thrashed the Faroe Islands 5-0.
Premiership trio Louis Saha, Thierry Henry and Nicolas Anelka claimed the other goals
Defending European champions Greece enjoyed their third successive victory as they won 4-0 in Bosnia-Herzegovina.
Angelos Charisteas' early penalty silenced the home fans, but Greece had to wait until the last 10 minutes for Christos Patsatzoglou, Giorgos Samaras and Costas Katsouranis to add to the scoreline.
Rangers defender Sasa Papac was sent off shortly after half-time for a foul on Bolton's Stelios Giannakopoulos, but television pictures suggested the foul was committed by Emir Spahic.
Hakan Sukur scored four times as Turkey romped to a 5-0 win over Moldova.
His first-half hat-trick included a penalty, with Tuncay Sanli adding a fourth after the hour mark, before Sukur finished off the visitors.
Malta made history as they beat Hungary 2-1 for their first European Championship qualifying win in 24 years.
Andre Schembri opened the scoring on 13 minutes, only for Sandor Toghelle to level six minutes later.
Hungary's Vilmos Vanczak was sent off shortly before half-time, and seven minutes into the second half, Schembri netted his and Malta's second goal.
Killbane's goal sparked delight in Dublin
Kevin Kilbane's goal against the Czech Republic looked to have put the Republic of Ireland on their way to a first win.
But Jan Koller's equaliser meant they had to settle for a draw, although the improved performance will have come as some help to under-fire manager Steve Staunton.
Wales bounced back from Saturday's mauling by Slovakia by beating Cyprus 3-1.
A Jason Koumas header and Robert Earnshaw shot put them ahead at the break, with captain Craig Bellamy adding a fine goal 72 minutes.
But Yiannis Okkas denied Wales a clean sheet with superb lob late on.
Chelsea's Michael Ballack was on target for Germany as they eased past Slovakia 4-1.
Bayern Munich's Lukas Podolski also claimed a brace, with Bastian Schweinsteiger scoring the other.
Stanislav Varga of Sunderland headed the consolation goal.
Croatia's second goal was a nightmare for keeper Robinson
England suffered the first defeat in Group E, as they went down 2-0 to Croatia.
Eduardo da Silva's looping header put the hosts in control, before Gary Neville put through his own net when the ball bobbled as Paul Robinson went to clear his back pass.
Two late goals gave Russia their first win, 2-0 over Estonia, who had Raio Piiroja sent off at the death.
Pavel Pogrebnyak's 78th-minute close-range strike broke the deadlock, with Dmitry Sychev adding the second.
FYR Macedonia followed up their draw with England by beating 10-man Andorra 3-0, with first-half goals by Goran Pandev, Nikolce Noveski and Ilco Naumoski.
Andorra's Antoni Sivera was sent off between the second and third goals.
Healy's goal was his 24th for his country
David Healy was Northern Ireland's hero once again as his goal out of nothing earned them a 1-0 win over Lativa at Windsor Park.
Keith Gillespie flicked the ball forward to Healy, who broke away from marker Maris Smirnovs and slotted the ball low into the bottom corner.
Maik Taylor's save from Girts Karlsons' close-range shot was the most dangerous moment for the home side.
Christian Wilhelmsson scored Sweden's winner as they recovered from conceding an early goal to beat Iceland 2-1 and maintain their 100% record.
Arnar Vidarsson put the home side ahead on six minutes, but within two minutes Kim Kaellstroem put Sweden level.
That was how things stayed until just before the hour mark, when Wilhelmsson struck.
Jon Dahl Tomasson scored twice as Denmark beat Liechtenstein 4-0 to get back to winning ways after their draw with Northern Ireland.
Daniel Jensen and Michael Gravgaard were their other scorers.
In Group G, Holland needed an own-goal by Arjan Beqaj to beat Albania 2-1.
Arsenal's Robin van Persie opened the scoring, with Beqaj doubling the lead before half-time.
Debatik Curri pulled one back in the second half to ensure a nervous finish, but the Dutch remain unbeaten and top of the group.
Sergei Kornilenko scored twice as Belarus came from behind to beat Slovenia 4-2.
Denis Kovba gave Belarus an early lead but within 30 seconds Bostjan Cesar equalised, with Klemen Lavric's header putting Slovenia ahead at half-time.
A quick Kornilenko brace early in the second half restored Belarus' lead.
Substitute Vladimir Korytko made it 4-2 to Belarus late on with a free kick in the 85th minute following a foul on two-goal hero Kornilenko.
And Aleksandar Tunchev's first-half goal was enough to give Bulgaria a 1-0 win in Luxembourg.