Media playback is not supported on this device
Simon Mignolet replaced 23-year-old Karius, who was criticised by pundits, including ex-Manchester United players Gary and Phil Neville, after errors against West Ham and Bournemouth.
"I'm not interested in public pressure, I'm interested in the boy," said Klopp.
"There's no reason to push him through the situation, he loses confidence."
The German boss added: "Karius has a lot of things that other goalkeepers want to have. We want to develop the players - we don't want to convince people that he's not bad.
"I know him really well and he's much better than he has been in the last two games. It's a situation which Liverpool have as a long-term project."
Klopp signed Karius from his former club Mainz for £4.7m in the summer and the German has played 10 Premier League games, keeping three clean sheets.
However, a fumble for Bournemouth's late winner on 4 December prompted comments from Gary Neville, while former Liverpool defender Jamie Carragher said the keeper was "yet to convince" and needed "massive improvement".
Media playback is not supported on this device
Karius defended himself in an article and Klopp also questioned the pundits, comments he now says were "not serious".
"I only want to show how it feels when you get criticised without doing anything wrong," added the 49-year-old. "The pundits need to know how it feels."
Klopp said he has "no time frame" for when Karius will return to the starting XI, leaving open the possibility Mignolet may play again in Monday's Merseyside derby at Everton (20:00 GMT kick-off).
The Belgium international, 28, made three saves as Liverpool dominated at the Riverside Stadium on Wednesday, including turning a powerful Viktor Fischer shot round the post when the score was 1-0.
The Reds moved second in the Premier League table after the victory, which featured two goals from Adam Lallana and Divock Origi's fifth goal in as many games.
Gary Neville's brother Phil told BBC Radio 5 live: "I don't think we are the only ones that have criticised Karius. My criticism of him was not for his performance on Sunday or over the last couple of games, it was for the newspaper interview he did and the comments he made about Carragher and Gary.
"Maybe when you are going through a tough period, I would give him the same advice I got which is to keep your head down, don't do any press interviews until you have a good few games and wins under your belt. Then you can do your interviews.
"My advice was not that this goalkeeper is no good. The decision to leave him out last night - to take him out the firing line - was the right one and it was justified in the performance of Mignolet. Karius is a young goalkeeper. He will bounce back."

Without a national uniform to wear because he was competing under the Olympic flag following Kuwait's suspension from the Rio Games, he decided to don an Arsenal shirt.
Whether he knows the football club's nickname is the Gunners is doubtful.
Nevertheless, the top helped him win Olympic bronze in the men's skeet.
Asked why he had worn it, he replied: "I don't know, I just bought it."
Some didn't waste any time cracking a few jokes at Arsenal's expense.
Christopher Flux suggested on Facebook that they could do with a sharp-shooter of their own, while Ash Dolan ‏tweeted his surprise that Al-Rashidi finished third and not fourth, which has often been the side's fate in recent times.
The Gunners were second to champions Leicester City in May but have finished fourth on six occasions in the last 11 seasons.
Liverpool manager Jurgen Klopp has backed Loris Karius to be a "long-term" success, despite dropping the keeper for the 3-0 win at Middlesbrough.
[NEXT_CONCEPT]
It seemed like the ideal solution but the significance was perhaps lost on Abdullah al-Rashidi.