The department, which employs 247 staff, said it plans to shut the office by 2018 but has not said how many, if any, redundancies there may be.
It said the decision was aimed at cutting costs by creating a combined central HQ and policy centre in London.
PCS, the union, say a further 100 jobs in the UK are threatened by the move.
Permanent secretary Martin Donnelly said the move was intended to "modernise the way we work, reduce operating costs and deliver a simpler, smaller department that is more flexible and responsive to stakeholders and businesses.
"The decision to close Sheffield by 2018 has not been taken lightly.
"We will provide comprehensive support to all those facing a potential change or loss of job."
However, he said the department was planning to create "around six business centres" across the country.
The department currently employs about 2,500 people in total across 10 locations.
Mark Serwotka, general secretary of the Public and Commercial Services union, said: "The business department is supposed to lead the way in helping local economies grow after the recession, yet it is retreating from towns and cities across the country.
"We do not accept the need for these offices to close and for jobs to be at risk and will be opposing these moves."
The union has not said where the other 100 jobs could be lost.
Julie Dore, leader of Sheffield City Council, described the plan as "appalling" and said she would be writing to the Secretary of State to "make it clear that this is unacceptable".
Nick Clegg, Liberal Democrat MP for Sheffield Hallam, said the decision "flies directly in the face of the Northern Powerhouse agenda".

Richard Ferrand, a minister in Emmanuel Macron's government, rented office space from his own partner for a fund he managed, it has emerged.
The revelation comes as Mr Macron's government prepares a law to tackle political corruption.
Both Mr Ferrand and the government say there was nothing illegal or unethical in the deal.
No public money appears to have been involved in the deal, which took place when Mr Ferrand was head of a health insurance fund, the Mutuelles de Bretagne.
However, tackling corruption in public life was a major campaign pledge from Mr Macron, who opposed MPs working as consultants or employing family members. The new justice minister is also currently drafting a bill on ethics in public life.
The allegations were first unveiled in the satirical French newspaper Le Canard Enchaîné.
In its report, it said the organisation led by Mr Ferrand had decided to rent office space from a building his partner was involved in.
On the basis of that guaranteed rental income, she was then able to secure a business loan worth about €402,000 (£347,000), the paper reports. Renovations carried out by the fund would also have increased the property's value, it added.
But Mr Ferrand was quick to address the issue on national television, saying the decision was made with full knowledge of his connection to the building.
"The administrators of the board, on which I did not sit, picked the best offer... which was a building owned by my partner," he told French broadcaster BFMTV.
He also said the buildings were still being used years later - indicating satisfaction with the arrangement.
He described the story as a "pseudo-scandal" and a "welcome present" for the beginning of his public life.
A government spokesman, Christophe Castaner, said that while the disclosure comes as a "bad time", there was no question of the minister resigning, or any suggestion of illegality.
Le Canard Enchaîné also alleged that Mr Ferrand's son had worked as a parliamentary assistant in 2014, which the minister dismissed as minor work which lasted for only four months.
Mr Ferrand has been appointed by President Macron as minister for territorial cohesion - a job he was picked for after helping the president to electoral victory as secretary general of the political party La République en Marche (Republic on the move).
Financial dealing between family members has become a political hot potato in France since the revelation last year - also by Le Canard Enchaîné - that presidential hopeful Francois Fillon had paid his wife for parliamentary work.
Media reports questioned whether she had actually performed the work she was paid for, and the controversy damaged his campaign, arguably clearing the way for Emmanuel Macron's victory.
Nearly 250 jobs have been put at risk after the Department for Business, Innovation and Skills announced it will close its office in Sheffield.
[NEXT_CONCEPT]
A close ally of France's new president is facing accusations over a property deal involving his partner.