In a speech in the Wirral, in the north-west of England, Mr Miliband repeated his admission Labour had made mistakes in immigration policy, highlighting the party's failure to ensure there were maximum transitional controls when new countries joined the European Union in 2004.
But he said his party had "listened, learned and changed".
He said it was not "prejudiced" to be concerned about immigration. "As prime minister, I will always address concerns not ignore them. As prime minister, I will always put working families first and that's why we'll have proper controls and rules," he said.
He set out five principles to help tackle the issue: "Securing our borders; restoring the principle that you contribute before you claim; achieving integration in our communities; ending the undercutting of wages of local workers through the exploitation of migrant workers and rebuilding trust by only making promises we can keep and proposing solutions that will work."
Mr Miliband announced plans to create a 100-strong new task force to target firms who exploit migrant workers and do not pay the minimum wage. The enforcement unit based in the Home Office would fine firms that exploited low paid workers, he said.
HMRC, which enforces the minimum wage, said it employed "selective and exemplary criminal investigation" for "deliberate non-compliance or obstructive behaviour". Most cases of failure to pay are settled without recourse to the civil or criminal courts.
Responding to Mr Miliband's task force plan, the CBI warned about "burdening" firms with inspections, saying any changes should target criminals.
"Mixing up employment rules with criminal offences like trafficking and illegal migration issues should be avoided," added deputy director-general Katja Hall.
Labour said the unit would bring together teams from the Gangmasters Licensing Authority and specialist police units with extra Home Office staff.
The coalition has increased penalties on employers who pay less than the legal minimum and has placed a cap on non-EU migration.
UKIP has said the the new policy on gangmasters is necessary only because of Labour's lax control of Britain's borders in the past.
But Steve Murphy, general secretary of the building workers union Ucatt, said: "This commitment demonstrates that Labour is serious about ending the misery caused by the exploitation of migrant workers and rightly targets the employers who profit through the mistreatment of workers."
'Must speak English'
Mr Miliband also said in his speech that "everyone in Britain" should know how to speak English and this was "especially important" for those working in public services.
He acknowledged the "crucial contribution" people from overseas played in the NHS, adding in future "all healthcare professionals will be required to speak English to a sufficient standard so they can care effectively for patients".
The coalition government has given some health regulators the power to impose language tests, but Labour says the process has been too slow and does not cover all staff, such as paramedics, social workers and physiotherapists.
Home Office Minister James Brokenshire, for the Conservatives, said: "Nothing Ed Miliband is proposing today would help control immigration. Labour sent out 'search parties for people' and under them net migration increased more than five-fold."
He said the Conservatives had cracked down on illegal working and would seek to change EU free movement rules if they won the election.
The Liberal Democrats also mention English language in their manifesto, saying people who cannot speak good English and want to claim Jobseeker's Allowance should have to attend courses.
The Conservatives say EU migrants should have to wait four years before they can claim certain benefits or social housing and that child benefit payments should not be made for children living outside the UK.
Recently, some Labour figures complained about the party's messages on immigration. David Lammy warned that the party should not be "trying to out-UKIP UKIP".
The Conservatives' key announcement on Saturday is that pensions campaigner Ros Altmann would be made a Tory peer and given responsibility for financial education and consumer protection.
David Cameron says he wants the "country's leading expert" on these issues to be at the heart of government, helping give people "more power to save, to access their pension, to pass their pension on to their children".
The Liberal Democrats focused on apprenticeships, with a promise the party would double the number of employers offering apprenticeships to young people. Business Secretary Vince Cable said this would mean 360,000 firms offering on-the-job training.
Companies would be offered exemptions from National Insurance and apprenticeship grants as extra incentives. The policy would mean the creation of "more apprenticeship starts per year than Germany", he said.