The prison's Independent Monitoring Board (IMB) says these drugs are one of the biggest issues facing staff.
In its annual report, the IMB also says around 1,000 prisoners have been diagnosed with mental health problems over the past year.
But it concludes staff are doing a good job under "difficult circumstances".
The IMB was set up to ensure that prisoners are treated humanely and reports back to the Ministry of Justice every year.
Trained sniffer dogs and closer observation of visitors are methods being used to cut the supply of legal highs, said chairman of Cardiff IMB, Stephen Cocks.
"Every prison in the country is suffering to one degree or another from drug misuse and particularly with the use of legal highs which, because of its unpredictable effects, can cause violent reactions among prisoners," he added.
The report also states there are not enough staff to cover basic duties, prisoner complaints are not responded to quickly enough and there is not enough privacy for inmates using the toilets.
Legal highs smuggled into Cardiff Prison have led to "horrific, self-inflicted injuries", according to a new report.