And 34% of blacks, and 29% of Asians, say they have personally experienced racial or religious discrimination at work.
One quarter of blacks also say they have observed racism in shops or the provision of other services, while one in ten (11%) saying they have experienced or seen racism in banking services.
The negative attitudes to ethnic minorities in the workplace may be related to the belief that they have not made much of a contribution to British society as a whole.
Overall, a majority of people in Britain judge that immigration has damaged British society over the past 50 years.
Just 30% say it is beneficial, while 44% say it has damaged society and 26% didn't know.
Black and Asians disagree with this assessment, with 43% of blacks and 50% of Asians citing positive achievements.
People in London and the Southeast, where ethnic minorities are concentrated, also disagree, with 41% citing benefits and only 32% damage.
Working with others
One measure of racism in employment is whether people would work with people from other races.
There appears to be widespread concern about having a black person as your boss.
One in three say that "people mind working for someone of a different race."
And one in four say that people would mind working in the same workplace as someone of a different race.
A higher percentage of black and Asian respondents believe that others are uncomfortable working with those of a different race.
However, a majority of those working full-time (48%) believe that the colour of a person's skin makes a difference in the way they are treated at work.
Getting a job
Many blacks and Asians also believe that they have not been hired for a job because of racial discrimination.
One-third of ethnic minorities, as compared to only 6% of whites, say they have been discriminated against.
However, many whites believe that ethnic minorities are given extra advantages when it comes to hiring.
More than one-quarter of whites (28%), as opposed to only 13% of Asians and 14% of blacks, say there has been positive discrimination in hiring.
Overall, the survey suggests that the racial divide - both in perception and perhaps in reality - is greater in the sphere of employment than elsewhere.