Michelle Barnes, 33, from Cumbria, was serving a two-year sentence for drug offences at Low Newton Prison, Durham, when she died in December.
The inquest heard that Ms Barnes had hoped to keep the baby with her.
However, Low Newton has no mother and baby unit, and she was unable to get a transfer to a prison which does.
Ms Barnes - who found out she was pregnant while serving her sentence - had a history of mental health problems and self harming, the inquest jury was told.
Cumbria County Council had begun care proceedings as it had "concerns" about her as a mother.
Her solicitor, Victoria Rowson, said: "Her intention was to be a good mother to her child".
However, an interim care order was made after the baby was born in hospital.
Social worker, Sara Driscoll, said:  "It was not a decision taken lightly at all.
"We believed there was no safe decision for the child, other than removing the child from birth.
"There was one occasion during one of my visits that she said 'If you take the baby, I will go too'.
"I passed that on to her mental health worker."
She agreed with Durham Coroner Andrew Tweddle's suggestion that this was something said "in the heat of the moment".
Earlier, Alan Richer, the most senior governor at Low Newton said the prison experienced 20 to 35 self-harm incidents a month, and it appeared to be a "coping mechanism" for some of the women prisoners.
He said there were around 200 babies born to inmates every year in the UK, and only five prisons which have mother and baby units, after one unit closed due to flooding.
The hearing, at Crook Civic Centre, continues.

The club has fined the pair for the incident, which happened prior to Tuesday evening's defeat by Brentford.
"I'd like to apologise to the Bolton fans for what we did before the game," Trotter, 27, told BBC Radio Manchester.
"We didn't realise it was live, it wasn't aimed at the fans or club, it was a silly gesture as we walked in."
A club statement said: "Bolton Wanderers takes the conduct of its players and staff extremely seriously and will not tolerate such behaviour."
The defeat, Bolton's 22nd in the Championship this season, means they could be relegated to League One on Saturday when they face Derby County.
The club have spent much of the season fighting a relegation battle, as well as off-the-field financial problems, and are 17 points adrift of safety at the bottom of the table with six games left to play.
"It's too hard to pinpoint one thing, there's a lot of things gone wrong. If we knew what the problem was then we would be doing better," said Trotter.
"There's a lot of factors, a lot of outside influence with what's been going on at the club, but it's not just that, the players haven't been good enough this season.
"We haven't done enough on the pitch. We're in the position we're in because we deserve to be."
An inmate found hanging in her prison cell days after giving birth, told a social worker "if you take my baby I go too", an inquest has heard.
[NEXT_CONCEPT]
Bolton Wanderers midfielder Liam Trotter has apologised for making an offensive gesture with David Wheater in a pre-match live club broadcast.