The centre at Milton, Stoke-on-Trent, is licensed to hold about 500 tonnes of recyclable waste, but last December the Environment Agency estimated there were about 6,000 tonnes.
The agency wants excess waste gone by July and said progress has been made.
Owner Hanbury Plastics Recycling Ltd (HPR) said it was applying to increase its licence to hold 3,000 tonnes.
Resident June Grimes said: "[We] shouldn't have to put up with the smells and the issues as these sites bring when [they're] put on your doorstep.
"What we want to see happen is this site completely cleared and relocated to a suitable industrial site."
Ms Grimes filmed images on her mobile phone of drivers trying to negotiate Milton's narrow lanes and bridges.
She added: "We have got heavy goods [vehicles], left hand drive, coming on our roads where our children walk to go to school and they're that big, they're mounting the pavement as they're going down."
A spokesman for the owners said it was the best way to deal with plastics, otherwise they would end up in landfill.
HPR said it was planning to double the workforce to 60 over the next few years.
An agency spokesman said it issued the company with a notice in December to remove excess waste from the firm's site.
He said: "Since that time, we have carried out regular checks to ensure that the company is working to remove the waste by the end of June 2014.
"To date, the company have made progress and moved waste from the site. We will continue to monitor progress and then decide on the best course of action once the deadline has been reached."

That was just below its record performance achieved the previous year, although 2015 included a one-off gain from licensing fees.
Turnover was up 14% to £447m while employee numbers were up by 12% to almost 4,000.
Almac operates across the world, with more than half its staff based in Craigavon.
The company said 2016 had seen "significant investment" on expanding its US HQ at Souderton, Pennsylvania.
Almac also recently announced a £5m investment at its global HQ campus in Craigavon to build an additional lab and office facility.
Further investment is being planned as part of its global expansion strategy and will be announced throughout the year.
In January the firm bought a factory in the Republic of Ireland as a direct result of Brexit.
A major part of the Craigavon-based business is making products for big drugs firms.
It told a Commons committee its customers need it to have a guaranteed presence in the EU for regulatory reasons.
Group chief executive Alan Armstrong said: "2016 was a very successful year for Almac as we continued to deliver upon our global expansion strategy including investment of around £31m in new facilities, equipment and resources."

9 October 2016 Last updated at 10:18 BST
But one of them, Donald Trump, has landed himself in some big trouble after comments he made about women 11 years ago were released.
In the past Donald Trump has used words like "pig", "fat" and "ugly" to describe women he disagreed with.
Now a 2005 video has come out, in which he said even worse things about women.
Since the recording was made public, Donald Trump has released a video statement where he apologised for the comments.
But Hillary Clinton has said that Trump should not be president after what he has said.
She says that his words will have a negative effect on women and girls.
People in Donald Trump's own party have said they strongly disagree with the things he has said. Some of them have said they will no longer be supporting him in the election.
The president of the United States is one of the most powerful and important jobs in the world, and the winner of this year's election will take over from current president Barack Obama.
These big presidential debates are shown on TV to try to help people make their mind up about who they want to vote for in the election.
Residents are calling for a recycling collection centre to be closed after it was told to remove excess waste.
[NEXT_CONCEPT]
Almac, the Craigavon-based pharmaceutical firm, made a pre-tax profit of almost £27m in 2016.
[NEXT_CONCEPT]
A big debate takes place on Sunday night in America between Hillary Clinton and Donald Trump, the two people trying to become the next US president.