The Northside regeneration scheme planned to revamp an area between Royal Avenue and Carrick Hill with the help of private investors.
Proposals included as many as 3,000 apartments and houses, mostly for students, as well as retail outlets, offices and a hotel.
The scheme would have taken five to seven years to complete.
The development consortium appointed by the DSD in 2014, Northside Regeneration Limited, expressed "surprise and disappointment" at the move.
In a letter seen by the BBC, the DSD said that the decision to pull out came after an "assessment of the scheme".
"As part of this process, minister committed to taking the views of local stakeholders into account as well as a number of areas, including where the developer had not met requirements set by the department.
"The minister has decided that the scheme as proposed by Northside Regeneration Limited should no longer benefit from the potential use of his department's statutory powers."
The DSD added that it "has no alternative plans at this time".
However, it added that "other options can now be explored" and said it is committed to "the regeneration the area needs".
The development consortium - a partnership between local developer Kevin McKay and international firm Balfour Beatty - is now seeking an urgent meeting with Social Development Minister Lord Morrow.
A statement from them said that "significant financial resources" had been invested in the project up to this point.
They added that "a number of factors beyond our control" have affected the timeframe.
A collaboration agreement between the DSD and the consortium expired on 31 March and has not been extended.
The scheme had faced community opposition.
Responding to the DSD move, Frank Dempsey of the Carrick Hill Residents' Association said the project was "ill-conceived from the start".
Stormont's Department of Social Development (DSD) has pulled out of a £300m development project in Belfast.