The UAE has held Pakistan's home Tests since 2010, after a terrorist attack on the Sri Lanka cricket team in 2009.
The first day of the first Test at Abu Dhabi's Zayed Cricket Stadium was marked by poor attendance, with banks of empty seats visible.
"You have to get people watching and they aren't here," said Vaughan.
"It's either too hot or they aren't interested. Forget the money - I'd play this series in Australia."
England bowler Stuart Broad, who is touring the UAE for a second time, admitted the lack of atmosphere does present a different challenge for players who were accustomed to sold-out stadia during last summer's Ashes series against Australia.
Speaking to Sky Sports, he said: "When you are playing an Ashes series in front of a lot of crowds at home, a lot gets you through the day but here it is about your 'inner ticker'... as Vaughany called it."
Vaughan added: "Pakistan played Australia in England and it was quite well supported. Why not take this series to a part of the world where it would be the same?
"There would be ex-pats. You could go to some of the lesser (smaller) grounds in Australia."
Pakistan have not been beaten in seven Test series in the UAE, and Vaughan believes the slow, spin-friendly conditions give them an excessive advantage.
"Home advantage has swayed too far," he said. "There's a fine margin making sure it's not too much."
Pakistan's Test series against England should have been moved from the United Arab Emirates to Australia, says former England captain Michael Vaughan.