The visitors lost the Test series 2-0 and the one-day international series 3-0 with only Tuesday's Twenty20 left.
The closest they have come to a win was in the opening ODI at Trent Bridge where England secured a tie by hitting the last ball of the match for six.
"There will be some who will hate us, some will still love us. Sorry for letting you down," Mathews tweeted.
"It is tough times but will come back hard."
After losing Saturday's final ODI in Cardiff, Sri Lanka also slipped to sixth in the International Cricket Council's world rankings, with England moving a place above them.
Having drawn their warm-up games against Essex and Leicestershire, the tourists' only victories have come in their two ODIs against Ireland in Dublin, shortly after the Tests in England.
Their tour has also been marked by a raft of injuries, with pace bowlers Dhammika Prasad (shoulder) and Dushmantha Chameera (lower back stress fracture) and batsman Lahiru Thirimanne (lower back strain), all flying home injured.
Several others have played on despite injury - Mathews and vice-captain Dinesh Chandimal suffered hamstring niggles, while Farveez Maharoof suffered a fractured finger on his left hand.
Sri Lanka captain Angelo Mathews has apologised for his team's disappointing tour of England.