Two bombs exploded in the Mulberry Bush and the Tavern in the Town in November 1974, killing 21 people and injuring 182 others.
Six men were jailed in 1975 but their convictions were quashed in 1991.
The service, at the city's St Philips Cathedral, paid tribute to other terrorism victims.
As part of the service, Birmingham Library was lit up in blue, white and red to also remember the victims of the recent Paris terrorist attacks, the council said.
Justice 4 the 21 campaign group, set up by victims' relatives, is calling for another memorial to remember the victims.
There is already a monument off St Philip's Place in the city.
But the University of Birmingham and victims' families are discussing plans for a more prominent tribute - possibly in New Street near to where one of the pubs used to be.
The families are also lobbying the government for inquests into the 21 deaths.
Relatives met Home Secretary Theresa May and Northern Ireland Secretary Theresa Villiers earlier this year to make a fresh call for the hearings.
An inquest was opened days after the bombings - at the time the worst ever terrorist attack on English soil - but it closed without hearing evidence in 1975 after the guilty verdicts.
After the convictions were quashed, the former Director of Public Prosecutions, Barbara Mills, placed a 75-year embargo on files relating to a Devon and Cornwall Police inquiry into the West Midlands Police investigation, meaning those files are not due to be released until 2069.

Paul Kelly, deputy leader of North Lanarkshire Council, was speaking the day after plans for 500 redundancies at the Uddingston base were made public.
Belgium owners Ageas said it was looking to close the site by the end of March next year.
The Scottish government said it was "ready to assist" if jobs do go.
Mr Kelly told BBC Radio Scotland's Good Morning Scotland programme that closure would have a "direct impact on the Lanarkshire area and the local community".
He added: "This is a very significant number of jobs and our immediate concern is with the workforce and their families. This is the lead up to Christmas, it is devastating news.
"But we are wanting action, we are wanting to immediately act on this that is why we have called on the establishment of a taskforce."
Mr Kelly urged the Scottish government to get round the table with other partners "to do all we can" and "as soon as possible".
Scottish Enterprise and the Scottish government are understood to be seeking another company which could use the insurance firm's contact centre skills.
Five years ago, the company had more than 900 staff in North Lanarkshire. It now employs 521 people at the Tannochside Business Park site.
An Ageas spokeswoman said Kwik Fit Insurance Services had been "under pressure for some time" as a result of changes in the way people buy insurance and the way the personal insurance market now operates.
Mr Kelly said he did "not disagree" with the reasons Ageas had given for the planned closure but he believed the current workforce was "highly skilled and a huge asset" to the wider business community.
He explained that the council had written to First Minister Nicola Sturgeon asking for the establishment of a taskforce.
Business minister Paul Wheelhouse said on Tuesday that he was "very disappointed" to learn of the proposed closure.
He added: "Should any job losses proceed, we stand ready to assist those affected through our initiative for responding to redundancy situations, Partnership Action for Continuing Employment (PACE) and our Finance Sector Jobs Taskforce.
"I am keen to meet with North Lanarkshire Council to work together with them to ensure we support the local community and secure investment at the site."
Ageas said it would now enter into a consultation period ahead of the planned shut down next year.
Relatives of the victims of the IRA Birmingham pub bombings have attended a memorial service to mark the 41st anniversary of the attacks.
[NEXT_CONCEPT]
A taskforce to save jobs at the Kwik Fit insurance office in Lanarkshire should be set up "as soon as possible", a local councillor has said.