More long-term unemployed people could have their benefits cut - or stopped altogether - under new proposals.
Work secretary John Hutton said he wanted to end the "can work but won't work" culture, saying society should expect more in return for benefits.
He said a "hardcore" of claimants was not competing with east European migrants for jobs.
But the Conservatives and some Labour MPs have questioned what stricter sanctions would achieve.
Government figures show 950,000 people claimed Jobseekers' Allowance last month. About 100,000 of them had spent six of the past seven years on benefits.
In his speech at the Institute for Public Policy Research (IPPR) in London, Mr Hutton compared the long-term unemployed with new migrants to Britain.
He added: "The next challenge we face is to ensure the hardcore of 'can work but won't work' benefit claimants take advantage of the opportunities out there and compete for jobs alongside growing numbers of migrants who arrive in Britain specifically to look for work rather than to settle for the long term."
There was no shortage of vacancies for low-skilled workers, even in areas of relatively high unemployment, he argued.
"Economic migration from the EU has only served to highlight this issue.
"If workers from Poland can take advantage of these vacancies in our major cities - why can't our own people do so as well?," he said.
He suggested there could be moves to cut - or even stop - benefits for those who did not do enough to find work.
At the moment, only a very small percentage of jobseekers face such sanctions.
"We need to ask whether we should expect more from some in return for the help we provide," Mr Hutton said.
People had to take "more active steps to get back into the labour market" and become more involved in programmes that could help them get a job.
"And for those who won't do so, then there should be consequences, including less benefit or no benefit at all," he told the IPPR.
Mr Hutton said it was also important to help people "up the career leader" and that "the labour exchange of the past must become the skills exchange of the future".
But initiatives which compel people off benefits may face resistance from the Tories and some Labour MPs.
Shadow Work and Pensions Secretary Philip Hammond told the BBC: "The Labour Party knows it has created the next prime minister in Mr Brown who isn't going to deliver, who isn't going to continue, with the Blairite programmes and what Mr Hutton and Mr Blair and his colleagues are trying to do is set the agenda for the next 10 years and lock Mr Brown in.
"If this was a serious proposal why isn't it included in the Welfare Reform Bill that's currently going through Parliament?"
The Liberal Democrats denounced the speech as "spin" said the government was "talking tough" on welfare reform, but doing little.
"This government has had a decade to come up with serious policy on welfare reform, but all we hear are the same old recycled cliches," said work and pensions spokesman David Laws.
More than half a million have arrived from Poland and other countries since 2004, far more than the Home Office prediction of about 13,000.