Norfolk and Suffolk NHS Foundation Trust (NSFT) was rated as "inadequate" in February 2015 when the Care Quality Commission (CQC) reported it was "not a safe... service".
Its overall rating has since risen to "requires improvement".
The trust said it would continue to make improvements.
However, the trust is still rated as "inadequate" for safety, which the Campaign to Save Mental Health Services in Norfolk and Suffolk described as "worrying".
Last year's CQC report expressed concerns about a lack of beds and said "urgent action" was needed.
If found insufficient staffing levels to safely meet patients' needs, inadequate arrangements for medication management and concerns regarding seclusion and restraint practice
Despite collection of data by the trust "there was little evidence of the use of intelligence and data to inform performance".
The lack of beds "meant that people did not always receive the right care at the right time and sometimes people were moved, discharged early or managed within an inappropriate service".
But following a new inspection, England's chief inspector of hospitals Prof Sir Mike Richards recommended to NHS Improvement (NHSI) that special measures now be removed.
He said: "Our return to Norfolk and Suffolk NHS Foundation Trust showed significant improvement had taken place… The trust's staff and leadership should be proud of their achievement so far…"
"The trust leadership knows what it must do now to ensure those changes take place."
Michael Scott, chief executive of the trust, said the announcement was "great news", adding: "It is a real testament to all of the hard work and commitment of our staff, and it's reassuring for our patients.
To have made this positive shift in just over a year and a half is a real achievement and we are incredibly proud of the progress we have made together in all parts of our trust."
NHSI said: "Today's removal of the trust from special measures reflects significant improvement at the trust and a confidence in the leadership."
A spokesman for the Campaign to Save Mental Health Services in Norfolk and Suffolk said: "It is worrying, 18 months after NSFT's failure, that the CQC inspectors have found that NSFT still lacks the leadership, beds, doctors and other staff to deliver safe mental health services.
"The CQC shares our concern about the alarmingly high number of patient deaths which have continued to rise during NSFT's time in special measures. MPs, health commissioners and NHS England must, as a matter of great urgency, ensure that NSFT has the leadership and resources to stop the dramatic rise in patient deaths and deliver safe and comprehensive mental health services in Norfolk and Suffolk."
A mental health trust has been taken out of special measures, 20 months after it became the first in England to have the conditions imposed.