The Care Quality Commission (CQC) blames a "lack of urgency" to treat people because there are not always enough senior doctors or nurses.
The emergency department has been rated inadequate and the trust as requiring improvement.
The trust claimed it had addressed many of the areas identified as needing improvement.
Click here for live updates on this story
Overall Torbay and South Devon NHS Foundation Trust, which includes community health services and adult social care services, was rated outstanding for caring.
Mental health services and adult social services were both rated good, while patient transport services were outstanding.
Community health dental care, special care dental services and community health services were rated outstanding.
The trust was created from two separate trusts seven months ago.
Chief inspector of hospitals, Professor Sir Mike Richards, said: "I appreciate that it is sometimes hard to manage change in large organisations yet, despite challenges we have found that there is a strong vision for the future.
"However, there are some areas where improvements are required.
"On our inspection we found there was an apparent lack of urgency to assess or treat some patients promptly by at Torbay Hospital."
Mairead McAlinden, the trust's chief executive, said: "We are a new organisation at the beginning of a massive change programme to deliver an exciting new model of care and tackle the long standing challenges we face as a health and care system.
"We have had longstanding pressure on our urgent care service and at the time of our inspection our urgent and emergency care service was at maximum escalation.
"We have already addressed many of the areas identified as needing improvement."

Typhoon Nida hit Hong Kong on Tuesday with high winds and torrential rain, forcing schools, businesses and transport services to shut.
But the storm was reported to be weakening as it moved to the mainland.
Guangzhou in Guangdong province issued a red alert, its highest weather warning, and people have been advised to stockpile food and essentials.
The southern cities of Zhuhai and Shanwei are also on red alert, with transport, industry and public services largely grinding to a halt.
Nida, which earlier passed over the Philippines, is set to be the strongest typhoon in the region since 1983, one official told Chinese state media, warning it could bring severe flooding.
In Hong Kong, thousands of workers were evacuated from an offshore oil platform and from a series of tunnels and bridges being built to link the territory with two other cities.
Hundreds of people took refuge in government shelters in the city, amid torrential rain and gusts of over 150km/h (93mph).
Although Hong Kong officials ended their rainstorm alert at midday on Tuesday, they warned residents there was still a risk of flooding in low-lying areas.
Southern China is hit by heavy rains every monsoon season, but this year has been particularly bad.
In July, Typhoon Nepartak killed dozens of people in Fujian province and forced hundreds of thousands of Chinese people from their homes. It also caused deaths and damage in Taiwan.
Patients in Torbay Hospital's emergency department are "at risk of avoidable harm" say inspectors.
[NEXT_CONCEPT]
Large areas of southern China are on emergency alert as a powerful typhoon batters the region.