The Planet Earth presenter said his work took him away for months at a time when his two children, Susan and Robert, were growing up.
Sir David told the Radio Times: "If you have a child of six or eight and you miss three months of his or her life, it's irreplaceable. You miss something.
"Perhaps you can't have your cake and eat it."
Attenborough said his wife, Jane, who died in 1997, had been very "understanding", however.
A month after celebrating his 91st birthday, Sir David told interviewer Louis Theroux that he thinks about his own mortality "all the time", because "it's more and more likely that I'm going to die tomorrow."
Asked if he expects anything to happen afterwards, he said simply: "No."
On the subject of global warming, Sir David warned: "We should be very, very worried about it.... the land is being scorched, deserts are spreading, and the seas are warming - all those factors cause great changes in our fortunes."
When asked which animal he felt a kinship with, Sir David replied: "That has to be an ape. Because our kinship is a reality. I don't feel it with a mosquito or, indeed, a whale."
Follow us on Facebook, on Twitter @BBCNewsEnts, or on Instagram at bbcnewsents. If you have a story suggestion email entertainment.news@bbc.co.uk.

Michael Redpath, 62, was pulled from the burning property in Albion Street, Swindon, but died in hospital as a result of smoke inhalation and burn injuries, Winchester Crown Court heard.
Patricia Louise Hindle, 31, from Swindon, will be sentenced on Friday.
Supt Sean Memory said Mr Redpath's final moments "must have been terrifying".
Speaking after Hindle was convicted, he described how she "ran away from the scene and made attempts to cover up her crime by disposing of his keys and bank cards".
"Today the court found that she did kill him and that she intended to kill him."
Hindle, of Crombey Street, carried out the fatal attack in September 2014.

Among them were customers at the Salt House Bar in Galway, who were celebrating when something out of the ordinary happened.
"I was cleaning up glasses and felt a drip on my head and we noticed that it was raining from the roof," said Liam McEvoy, who was working in the bar.
He went upstairs to the storeroom and discovered a keg had burst.
"When 20 litres of beer comes out of the keg there's a lot of it, so it spread across the floor upstairs and started to come through in a few different spots," he said.
Liam said staff cleared up the spill pretty quickly, but that did not stop some of the beer from raining down on customers in the bar.
"It was unfortunately just over two tables of girls, so we just gave them umbrellas and let them get on with their day," he said.
"At first everyone was a bit unsure what was happening, but it was St Patrick's Day so nobody was really going to complain or go home and they got a bit of a laugh out of it, getting served with umbrellas up in the pub even if it's bad luck.
"I think beer from the roof was strange enough, let alone having superstitions."
Eventually the drips subsided, but the female drinkers had got quite used to the novelty of using umbrellas indoors.
"I think the girls started to enjoy it so they kept them up for about half an hour, even though they probably only needed them for a few minutes," he said.
Sir David Attenborough says his only career regret is missing out on time with his children.
[NEXT_CONCEPT]
A woman has been found guilty of murdering a man by setting a house on fire and locking him inside.
[NEXT_CONCEPT]
Irish people around the world raised a glass or two in honour of their patron saint on St Patrick's Day.