Ministers are planning to publish on the internet the names of absent parents who refuse to pay maintenance for their children.
The CSA is to be scrapped as part of a major shake-up of the system
Work and Pensions Secretary John Hutton said he aimed to "come down like a ton of bricks" on absent parents.
New enforcement powers to be outlined in a White Paper this week include the removal of passports, curfews and electronic tagging.
Opposition parties said the plans were a "knee jerk reaction" and a "gimmick".
The White Paper will include details of the smaller body which is to replace the troubled Child Support Agency (CSA), which has been dogged by problems and is owed £3.5bn.
Mr Hutton said the reforms would make the system simpler and more streamlined so the new agency "can get right in there quickly, end the delays, get the money into the families that need it most and make a contribution to supporting family life and tackling child poverty."
Some critics say the new measures could cause hostility between partners at a difficult time in their relationship.
But Mr Hutton told Andrew Marr on Sunday AM that life should be made "as uncomfortable as possible" for parents neglecting their responsibilities.
"When they don't pay, we have to pay, and that is not a proper thing to happen," he said.
Former Tory leader Iain Duncan Smith said the CSA had shown it has a limited value but said the government's plans were a "knee jerk reaction".
"Incompetent and failing as it is, it has had a marginal impact on fathers staying up with their responsibilities," he said.
Liberal Democrat spokesman on work and pensions David Laws said the plans were "yet more daft 'government by gimmick' on the CSA".
"Listing people's names on a website is likely to be totally ineffective and could be seen by some people more as a badge of pride than a badge of shame."
He said the CSA's existing powers to take away driving licences had been little used and were of little effect.
Only wholesale reform and aggressive enforcement would make any difference, he said.
Labour MP, Frank Field, former minister for welfare reform, said he supported the plans but measures would need to be in place to ensure the information posted on the internet was accurate.
He said no debts should be written off when a new system is brought in.
Nick Woodall of the Centre for Separated Families, a support group for families going through separation, said the government "unwittingly contributed" to conflict between parents.
He told BBC Radio Five Live: "It seems like it's just another attempt by the government to sound tough on parents, when really what it should be doing is creating services to help them."
Set up in 1993 to ensure parents who do not live with their children pay for their upkeep
Spends 70p to collect every £1 of child support
£3.5bn in payments not collected since 1993
Reforms begun in 2003 cost £539m but scheme worked no better than predecessor
Third of non-resident parents pay nothing despite their maintenance being assessed
Source: National Audit Office, published June 2006
But Janet Allbeson from the charity One Parent Families said lone parents were desperate for the government to "put some welly" into the collection of unpaid maintenance.
"We talk to lone parents all the time and they are desperate for the (Child Support Agency) to really take strong action against non-resident parents, to make sure they pay.
"And they're incredibly angry that people who don't pay child support, really, they've been treated a bit too lightly."
On Saturday it emerged that the CSA had been increasingly using private companies to collect unpaid money, which had so far enabled it to recover about £320,000 which it would not otherwise have recovered.
However, some campaigners fear that more than £1bn owed to parents will be written off when the agency is replaced.