The world record holder led with two hurdles to go at the US trials but missed the required top-three place by a hundredth of a second.
The 30-year-old had only competed in three previous meets this year.
"To be where I am is a miracle, but it's a pity because in six weeks I'll be in much better shape," he said.
"I thought I had finished second or third. When it came up fourth I was shocked."
Merritt, who tore his groin in June, asked for the photo finish to be reviewed but the result stood.
Devon Allen was the surprise winner in 13.03 seconds, with Ronnie Ash and Jeff Porter both clocking 13.21secs to claim the other two Olympic slots.
Merritt won bronze at the World Championships last August and days later had an operation to receive a kidney from his sister.
A "complication" led to further surgery and signficantly affected his preparations for this season.
At the trials in Eugene, Oregon, world silver medallist Justin Gatlin held off LaShawn Merritt to win the 200m in 19.75secs and complete the sprint double after claiming the 100m title in a world-leading 9.80 seconds on Monday.
And 41-year-old Bernard Lagat qualified for his fifth Olympics by winning the 5,000m.
Lagat represented Kenya in the 1500m at Sydney 2000 and Athens 2004 - winning bronze and silver respectively - before travelling to Beijing and London as part of the US team.
Olympic 110m hurdles champion Aries Merritt has failed in his attempt to qualify for the Rio Games - less than a year after having a kidney transplant.