Location, Location, Location is among the programmes shown by Channel 4
It is "difficult to see" how any merger between UK commercial broadcasters Channel 4 and Five could take place, a report by independent analysts said.
The pair first discussed a tie-up in 2004 and recent reports suggested Five owner RTL was mulling a takeover bid.
But Enders Analysis said they had such a big share of the advertising market that any merger was unlikely to be approved by the Competition Commission.
Ofcom has said Channel 4 will need up to £100m to meet its existing remit.
Funding from other sources would be required to fulfil Channel 4's public-service commitments, the media regulator said in September.
Ultimately, the government will decide how the broadcaster is funded.
But Ofcom has said it opposes the so-called "top-slicing" of the licence fee, where some of the revenue generated for the BBC is redistributed to Channel 4.
The analysts' report said any merger between Channel 4 and Five was likely to be considered by Peter Freeman, chairman of the Competition Commission.
Last month he decided plans by the BBC, ITV and Channel 4 to launch an on-demand video service - known initially as "Project Kangaroo" - could "hurt competition".
They would also lead to a "substantial lessening" of competition in the UK video-on-demand market, he added.
Among the most popular series on Five's schedules is CSI: New York
Enders Analysis said it was likely Mr Freeman would come to the same sort of conclusion about a tie-up between Channel 4 and Five.
"He is almost bound to be as severe as he has been on [Project] Kangaroo," the report said.
If either Channel 4 or Five were deemed to be "failing" financially, then a merger might be allowed without being referred to the commission, the analysts added.
"But this is a stringent test, one which would apply if imminent bankruptcy were an issue rather than a sense of foreboding about the distant future," they wrote.
"Neither Channel 4 nor Five would pass this test at present."
RTL, the owner of Five and more than 40 other TV channels in 10 countries, has expressed its desire to increase its hold on the UK market and was previously linked to a bid for ITV.
Ofcom is due to report back early this year with proposals on how the commercial TV industry can reshape itself at a time when advertising revenues are falling.
There has also been unhappiness among broadcasters about the amount of news, children's, religious and arts programmes they must screen under their "public service" remit.