The Ibrox side lost 2-1 on aggregate to crash out in the first qualifier.
Their part-time opponents, who are ranked 440th by Uefa, secured their first win in European competition.
"We need to apologise to our fans because they gave us support and loyalty, they don't deserve this," Caixinha told Rangers TV.
"I assume all the responsibility."
Kenny Miller's winner in the first leg in Glasgow had put Rangers 1-0 ahead going into the match in Luxembourg.
And up until Tuesday night Progres, who finished fourth in Luxembourg's top flight last season [21 points behind champions F91 Dudelange], had not won any of their 13 games in Europe.
But it is the side who are only two places below Prestatyn Town of Wales in Uefa rankings - and from a town with a population of 3,000 - who progress to the second qualifying round.
Progres had chances before Emmanuel Francoise netted at the near post in the second half, scoring his side's second ever goal in European competition.
The second came from a Sebastian Thill free-kick that evaded everyone in the box and curled in at the far post.
At 0-0, Rangers had hit the bar through Niko Kranjcar's header and at 2-0 Josh Windass and Kenny Miller also saw efforts come back off the bar.
The defeat goes down as one of the worst results in the club's history.
"We could not do what we were here to do - win the game," Caixinha added. "It's something that happens once in a lifetime - it happens to us today.
"For the second goal we knew we could not commit unnecessary fouls and we did, but as I said, I assume all the responsibility and the disappointment."
The Portuguese manager, who has overseen a significant summer rebuild added: "I'm always a positive guy, I never give up. It's a strong moment that we are living and we need to face it, we need all our character, personality and strength.
"I believe in this process, I believe in the players and the work we are going to do.
"It's a strong lesson that we need to learn from now on, and we need to focus on the positives regarding the future. We need to keep working harder and looking forward."
For Progres, this was the greatest result in their history.
Head coach Paolo Amodio told BBC Scotland: "It's a great event, amazing. I cannot believe that we won 2-0 against Rangers. It's not possible.
"We made history and I can't believe it. It's incredible."
Rangers manager Pedro Caixinha apologised to the club's fans for his side's Europa League embarrassment by Luxembourg's Progres Niederkorn.