Plans for a £41m revamp of the central London street, including a daylight ban on cars six days a week, were approved by Camden Council in 2014.
The Licensed Taxi Drivers Association (LTA) said it was "madness" and would hinder customer pick-up and drop-off.
It sought a judicial review of the decision that was refused on 1 June.
The council said the redesign would make the street safer and support businesses ahead of the opening of a Crossrail station in 2018.
It undertook an assessment of the impact the partial car ban might have and said allowing taxis to use the street would lead to more traffic congestion, worse air quality and increased traffic collisions.
Only buses, bicycles and local access would be allowed on the street from 08:00 to 19:00 Monday to Saturday, but side streets should still give other vehicles 60% access to Tottenham Court Road, the council added.
Black cab drivers have been denied permission to mount a legal challenge against the decision to limit car access to Tottenham Court Road.