Michael Howard has called for a change in the law which would give parents equal access to their children when their relationships break down.
A minister says family courts have a 'gender bias' toward mothers
At a summit on custody battles on Monday, the Tory leader said the current system was unfair and often "fairly chaotic".
It is thought ministers are considering law changes to give divorced fathers a better deal on custody and access.
The Tory summit is considering shared parenting - a norm in many countries.
Mr Howard said the courts dealt with more than 65,000 applications for parental contact in 2002. Many of them were formalities but almost half of all fathers lost contact completely two years after separating from their mother.
He said mediation, rather than court action, should be the first step as far as possible after relationship breakdowns.
"There should be a strong presumption in favour of equal rights for parents to have an influence on the upbringing of their children," said Mr Howard.
The summit is looking at 'shared parenting' in custody battles
"The absence of such a presumption has meant that parents with residence have found it far easier to obstruct the other parent's access to their children and their ability to have a say in how those children are brought up. We must redress that imbalance."
Children's Minister Margaret Hodge said it would be very dangerous to challenge the principle that the interests of the child should be paramount.
She told BBC Radio 4's World At One: "The courts are not institutionally unfair. The courts don't begin with any presumption of access in favour of either mother or father."
The government is due to publish a consultation paper before MPs start their summer break.
Ms Hodge said she wanted to get access agreed more often outside the courts through mediation; to speed up the court system; and ensure better implementation of court orders.
But she stressed that in nine out of 10 cases access was agreed outside the courts and she said she was "saddened" the issue was now becoming party political.
Judges are currently only allowed to impose fines or prison sentences on mothers who deny fathers access, but both are seen as damaging to the family and are
Newspaper reports claim the government is considering forcing mothers who flout court orders on giving ex-husbands
access to do community service.
Ms Hodge said there might be sanctions but there could also be a bigger role for Cafcass (the Children and Family Court Advisory and
Support Service) in ensuring court orders worked through conciliation.
Liberal Democrat spokeswoman Annette Brooke said the focus had to be on children, who could become "pawns in a terrible game between parents".
"Where there are broken relationships, we need a system of mentoring and counselling to make sure the children's interests are put first," said Ms Brooke.
"In principle there should be equal access available for both parents, unless there is any question of domestic violence."
The issue of fathers' rights has been pushed up the political agenda as divorce rates continue to rise.
At the weekend campaign group Fathers 4 Justice staged a protest at York Minister at a service attended by the Church of England's General Synod.
A total of 14 people were arrested and bailed following the demonstration.
The group hit the headlines in May with a purple powder bomb attack on Tony Blair in the House of Commons.
The group's Matt O'Connor told the World At One its protests had raised awareness, although politicians did not want to admit its contribution.