The report said ethnic minorities largely performed better academically than white Scots.
But they were more likely to be unemployed or in low-paid work, and were under-represented in senior management roles.
This was despite 40 years of legislation and equality policies.
The report by Holyrood's Equal Opportunities Committee followed a six-month investigation into the obstacles encountered by people from ethnic minority communities in Scotland.
It found that there were "significant barriers facing people from ethnic minorities in gaining employment and developing a career".
The report said: "The evidence indicated a situation that is not acceptable and we were extremely concerned to hear of how discrimination and lack of access to opportunities are holding back many talented and committed individuals."
The Scottish government said it was "completely unacceptable" that anyone should face barriers to training or employment as a result of their race or religion.
And it said it would be "fully responding" to the committee's recommendations as part of efforts to eradicate the problem.
Born in Togo, Joseph moved to Glasgow aged four and is fluent in English and French. He is also Scotland's under-20s triple jump champion.
He said: "I don't feel like I encountered any barriers during my school years. There were lots of people at Bellahouston Academy from a range of different ethnic backgrounds and I never felt we were treated differently.
"The problems started when I left school. I was at college, but getting a job was definitely my goal.
"A white, Scottish friend and I would go out together looking for work. We would hand in our CVs, but even though we had the same qualifications, he got the calls.
"I thought putting my picture on my CV would show I'm smart and presentable. But then I started to wonder if having my picture  - and name - on my CV made the difference."
Read more here
The committee called on the Scottish government to tackle "defective" employment and recruitment practices in order to "tackle discrimination and promote diversity" in Scotland.
Among its recommendations were:
Committee convener Margaret McCulloch, a Labour MSP, said: "Achieving equality in the workplace is a vital part of ensuring Scotland as a nation is fair and inclusive to all.
"We can only progress if we refuse to accept current defective recruitment practices and challenge segregation within employment. Without confronting existing practices, we cannot address any underlying racism and discrimination that the evidence confirms exists."
Failure to act risked placing an "ethnic penalty" on Scotland's young people, she added.
A spokeswoman for the Scottish government said it "values the diverse communities who enrich Scotland socially, culturally and economically" and was committed to eradicating any barriers faced by people from ethnic minority backgrounds.
The government is currently developing a Race Equality Framework for Scotland which will be in place by the spring and will set out its approach to promoting race equality and tackling racism and inequality between 2016 and 2030.
People from ethnic minority backgrounds in Scotland are still being held back by workplace discrimination, according to a report by a committee of MSPs.