Four attackers were seen entering the office, followed by heavy gunfire and several explosions.
Five security officers - including a police chief - were among the dead and 26 government officials among the hurt.
A spokesman for the Taliban said they carried out the attack.
The siege ended seven hours after it began and all four gunmen were killed by the Afghan National Security Forces which had surrounded the area.
Police said they rescued the attorney general from his office, which is just 200m (650ft) from the office of the governor of Balkh province.
The governor's aide told the BBC that the governor, Atta Muhammad Nur, was at home when the attack began but was taken to his office to monitor the situation.
In a statement seen by the BBC, Taliban spokesman Zabihullah Mujahid said: "Breaking News: This noon suicide [attackers] stormed office of the attorney and appeal court of Balkh province. Heavy casualties for enemy, battle continues."
The Taliban has targeted Afghanistan's legal system in the past. In June 2013, the group detonated a car bomb in front of the compound of Afghanistan's Supreme Court, killing 17 and wounding 39.
And in December of 2014, Atiqullah Raufi, the chief of the secretariat of Afghanistan's Supreme Court, was assassinated by Taliban gunmen in the west of Kabul.
The attack in Mazar-e-Sharif comes a day after an American soldier was killed in a fire fight between US and Afghan troops in eastern Afghanistan.

Patrick Mugadza announced last week that the 92-year-old head of state would die on 17 October this year.
His lawyer, Gift Mtisi, told the BBC that he was relaying a "message from God. Police would have to prove that God didn't say it".
Mr Mugabe mocks frequent rumours of his death, saying he has been resurrected more often than Jesus Christ.
Mr Mtisi said his client had initially been charged with undermining the authority of the president, then "criminal nuisance" and finally "insulting people of a certain race or religion".
He said Mr Mugadza was laughing about the charges and would plead not guilty.
"I'm still at pains to find the criminal part of it," Mr Mtisi told the BBC.
He added the pastor had no regrets about making the prophesy:
"He's admitting to the facts. He says he didn't lie - that's a message from God. Police will have to prove God didn't say it."
Mr Mugadza of the Remnant Church held a press conference last week at which he announced his prophecy of Mr Mugabe's forthcoming demise.
He was arrested at the magistrate's court in Harare where he was appearing on separate charges of wearing the national flag.
This was made an offence last year, after another Pastor, Evan Mawarire, launched a campaign urging Zimbabweans to reclaim their flag.
This is not Pastor Mugadza's first brush with the law.
In 2005, he was arrested and detained for nearly a month after holding a placard telling Mr Mugabe his people were suffering under his rule.
And on Zimbabwe's independence day last year, he gave a sermon while tied to a lamppost in Harare's main shopping mall, saying the act symbolised the lack of freedom in the country.
Last September, the president was rumoured to have died after he reportedly cut short his attendance of an AU summit to fly to Dubai for a health check.
Mr Mugabe later joked about the rumours, saying he indeed died but was only resurrected.
Gunmen dressed in military uniforms have stormed the office of the attorney general in the northern Afghan city of Mazar-e-Sharif, killing at least 10 people and injuring more than 65.
[NEXT_CONCEPT]
A Zimbabwean pastor who predicted the death of President Robert Mugabe has been arrested, his lawyer says.