The Spaniard, 31, lost the first set but fought back to win 6-7 (3-7) 6-3 6-1 6-4 after three hours and 16 minutes.
Nadal saw 13 break points go begging before finally breaking the world number 59 after nearly two hours.
He dropped just five more games on his way to a last-16 meeting with Ukraine's Alexandr Dolgopolov.
"All the matches have been very tight - I am very happy," said Nadal.
"I don't know how many break points I had before I got the first one. Mentally, I fought a lot until that moment, then I think I played better."
Nadal could not land a telling return as the break points came and went against Mayer, and the Argentine took advantage by playing a fine tie-break to take the first set.
I am in the fourth round now. I don't think much about that.
When Nadal finally made the breakthrough at the 14th attempt, at 3-3 in the second set, he screamed "Vamos!" and his family and team celebrated in the stands.
The Spaniard had won all three previous matches against Mayer and went on to dominate after finally breaking.
There was a late flash of resistance when Mayer broke serve with Nadal trying to close out the match, but two games later the world number one sealed victory at 8.40pm local time.
Asked about a potential first-ever meeting with 19-time Grand Slam champion Roger Federer at the US Open in the semi-finals next week, he said: "I am in the fourth round now. I don't think much about that.
"It's a long way away. All the matches have been very tough.
"The only thing I am looking for is the practice tomorrow to keep working hard and focus on the match against Dolgopolov."
Third seed Federer booked his place in the last 16 with a clinical 6-3 6-3 7-5 victory over Spain's Feliciano Lopez.

A member of staff at Glenshee Ski Centre alerted SAIS to the slide on Glas Tulaichean, a 3,448ft (1,051m) Munro.
SAIS Southern Cairngorms said the full depth avalanche was filled with mud, rocks and large blocks of snow.
Another large slide was recorded at Creag Meagaidh in February.
The Sportscotland Avalanche Information Service (SAIS) provides avalanche hazard information reports for walkers, climbers and skiers from December to early April.
The service covers hills and mountains in Lochaber, Glencoe, Creag Meagaidh, Southern Cairngorms, Northern Cairngorms and Torridon.

The MP was suspended by Labour last week and apologised to the Commons for endorsing calls in 2014 for Israel to be moved to the United States.
The comments triggered a wider row over alleged anti-Semitism within the party, with an inquiry now under way.
Keith Vaz, the chair of the committee, said Ms Shah would not take part in its meetings for the time being.
The committee is beginning an inquiry into the extent of anti-Semitism in the UK and will call Labour leader Jeremy Corbyn and Ken Livingstone, the former mayor of London who was suspended from Labour for suggesting Adolf Hitler was a Zionist, among its witnesses.
Mr Vaz said Ms Shah - the MP for Bradford West who was elected to Parliament in May 2015 - had requested not to be involved in this inquiry or the committee's other activities for the time being.
"Naz Shah has informed me and the committee that she wishes to stand aside from all her duties until current matters are resolved," he said.
"We have accepted her decision. She will not take part in any future matters concerning the Home Affairs Select Committee with immediate effect."
Mr Corbyn has asked the former head of Liberty, Shami Chakrabarti, to lead an inquiry into the extent of anti-Semitism and "other forms of racism" in the party. No timeframe has been given for when its findings will be published.
Ms Shah told MPs last week that she regretted a series of Facebook posts in April 2014, including one endorsing a suggestion that Israel should be relocated to the United States as a "solution" to the Israeli-Palestinian conflict.
Top seed Rafael Nadal overcame stubborn resistance from Argentina's Leonardo Mayer to reach the fourth round at the US Open in New York.
[NEXT_CONCEPT]
The largest avalanche recorded by the Sportscotland Avalanche Information Service so far this season has been discovered in the Southern Cairngorms.
[NEXT_CONCEPT]
Ex-Labour MP Naz Shah has "stood aside" from the Commons Home Affairs select committee "until further notice".