The Academy of Medical Royal Colleges and Faculties said staffing and leadership were particular concerns.
It recommended the Scottish government and NHS boards agree minimum safe staffing levels for all professions in hospitals.
The Scottish government said NHS staffing was at its highest-ever level.
The academy set up a working group to look at how the health service could learn from past failures.
It followed three previous reports into serious failings of care in the Scottish health service.
They looked at problems of staffing and leadership at Aberdeen Royal Infirmary (ARI); above-average mortality rates at NHS Lanarkshire; and the worst-ever outbreak of C-difficile infection, at Vale of Leven Hospital.
The academy's report, Learning from Serious Failings in Care, identified key issues to be addressed, including:
The working group was chaired by Prof Alan Paterson of Strathclyde University and involved senior experts from the Royal Colleges in Scotland.
Prof Paterson said: "It is clear that serious failings in team working between clinical staff and NHS management played a significant role in the failings in care identified.
"These failings are deep-rooted and systemic. They must not be ignored if we are to learn from them and to prevent repetition.
"It is also clear that a combination of factors led to some appalling failings in care, a loss of basic compassion and the prioritisation of inappropriate targets over patient care. In addition, leadership and accountability were all too often sadly lacking and bullying endemic.
"While there have been responses to the individual published reports of inquiries and reviews into failings in care, there is little evidence to suggest that we are tackling the underlying systemic failings which exist."
The report recommended that hospital staffing levels should be based on "best evidence". It said priority should be given to developing minimum safe staffing for acute medicine and medicine for the elderly wards.
Among its 20 recommendations was a suggestion that there should be less reliance on locum and agency staff, and a call for quality of care to be put ahead of meeting targets,
It also raised concerns about staff sickness, saying this could be an indication of deteriorating morale and should be used as "an early warning system".
Royal College of Nursing Scotland Director Theresa Fyffe said: "These are hard-hitting recommendations and will make uncomfortable reading for many managers and leaders in our health service."
She added: "Too often, failures in care are down to unsafe staffing levels. We know that quality of care is inextricably linked to having the right number of suitably skilled staff in the right place at the right time, with the right balance between registered nurses and unregistered staff.
"Having appropriately trained and experienced staff must be a priority for all health boards if they are to deliver safe, high quality care to patients."
The Scottish government's national clinical director for healthcare quality, Prof Jason Leitch, said: "We are committed to driving up standards in our NHS which is why we have led the way in responding to the reports detailed in the Scottish Academy's report.
"We are absolutely clear about the value we place on staff and the safe and effective service they deliver to patients.
"We currently have the highest staffing levels across our NHS than ever before, with increases under this government of over 1,300 more consultants, and 2,300 more qualified nurses and midwives."
Prof Leitch added: "We are constantly striving to further improve health care in Scotland and will review the recommendations within this report and take forward any learning that will benefit NHS Scotland, either as part of the follow up work to the Vale of Leven, Grampian and Lanarkshire reviews."

He posted three different photos along with the message: "Today's exhilarating spacewalk will be etched in my memory forever - quite an incredible feeling!"
He later sent a second tweet which said: "Wrapping up today's spacewalk activities. Huge thanks to the ground teams who make it all possible & keep us safe out there - you guys rock!"
Tim Peake made history on Friday when he became the first British person ever to walk in space.
He and fellow astronaut Tim Kopra were outside the space station, carrying out repairs, for four hours and 43 minutes.
However, they were forced to finish the walk early, when Tim Kopra found water in his helmet.
A body representing health professionals in Scotland has said major changes are needed to address "systemic failures" in the NHS.
[NEXT_CONCEPT]
UK astronaut Tim Peake has tweeted a selfie of his historic spacewalk.