Danish goalkeeper Thomas Sorensen fumbled for England's first and third goals, while the Danes lost track of Michael Owen for the Liverpool forward's first goal in the tournament.
I made a mistake on the corner and it cost us an
Danish goalkeeper Thomas Sorensen
"We played a good game but we made a couple of mistakes," Olsen said.
"After the first goal it was difficult because they have a very good defence, then after a quarter-of-an-hour they are 2-0 in front with a mistake in our defence.
And Olsen paid tribute to an England defence, which has now conceded just one goal in four games.
"We reacted well and we played some good football.
"But just before half-time we made a mistake and the match is over because to score three goals against England is almost impossible."
Saturday's game - Denmark's 13th in the World Cup finals - proved unlucky for Olsen.
In those previous 12 games in the finals the Danes had never failed to score in a World Cup match.
But after playing so authoratively in Denmark's previous three games, Sorensen proved unable to maintain that consistency against players he faces week-in, week-out.
"I made a mistake on the corner and it cost us an early goal," he admitted.
"That's an example of the tough luck you can have as a goalkeeper.
"I'll hold my hands up, but yes it was a very sad moment.
"I've got the character to get over this. I'll take it like a man, and just get on with it."