The Lesedi la Rona, almost the size of a tennis ball, was unearthed in Botswana in November.
The rough diamond is 1,109 carats and believed to be more than 2.5 billion years old.
It had been expected to sell for more than £52m ($70m) but the highest bid was about £45m ($61m).
The auction at Sotheby's was the first time a rough diamond of such a size has gone on public sale.
Why have so many huge diamonds been found recently?
The auction house had called its discovery "the find of a lifetime" and earlier said that "every aspect of this auction is unprecedented".
The Lesedi la Rona, which means "our light" in the Tswana language spoken in Botswana, was discovered by Lucara Diamond Corp's Karowe mine.
After it failed to sell, the Canada-based company - which has retained possession of the diamond - saw its stock fall more than 14% on the Toronto Stock Exchange.
According to a study by the Gemological Institute of America, the rough diamond's colour and transparency "exemplify" type IIA diamonds.
Stones in this group are said to be "the most chemically pure and often show extraordinary optical transparency".
In terms of its size the rough is exceeded only by the Cullinan Diamond, mined in South Africa in 1905 and presented to King Edward VII.
That 3,106-carat diamond was cut into nine stones, many of which are in the British Crown Jewels.

But the 31-year-old was disappointed with his time of 51.90 seconds ahead of the British Championships in Birmingham on 3-5 July.
Williams is continuing his comeback from a four-month ban for failing a drugs test in July 2014.
"I didn't expect to run so slow today... but with a few tweaks I should run a bit quicker next week," he said.
"I'm expecting to run under 50 seconds but it's no good me talking about it, but I've trained hard and I can only do what I can do."
Williams is bidding for a place in the Great Britain team for August's World Championships in Beijing.
But he would have to finish in the top two in the British Championships to be selected and also needs to meet the qualifying time of 49.50sec at an event before Beijing.
Williams only returned to competitive action in May after his ban, which he served after UK Anti-Doping found that he and fellow Welshman Gareth Warburton had inadvertently taken a contaminated food supplement.
Elsewhere at the Cardiff Athletics Stadium, 17-year-old Hannah Brier claimed her first senior women's 100m title in a wind-assisted 11.40sec - just outside the national record of 11.39 sec and all-comers record of 11.27sec.
"I just wanted to concentrate on my start as I've been chasing times a lot and that's not very good," Brier said.
"I'd forgotten that I had to concentrate on my start and I was really happy with that - 11.40 was up there with some of the fastest times I've run this season."
Brier, who added the 200m title on Sunday in 23.62sec, now hopes to be selected for the World Youth Championships, which begin in Colombia on 15 July.
"I've run the qualifying standards quite comfortably, so it's just whether the selection panel now think I'm worthy enough of going out there," she added.
Sam Gordon took the men's 100m title in 10.36sec, with defending champion Dewi Hammond having to withdraw after picking up an injury when winning his heat.
Navid Childs won the triple jump with 14.68m, while Sian Swanson retained her women's triple jump title with 11.06m.
Olly Sheppard was the 3000m champion in 10min 47.52sec, with Vicki Cronin running a personal best of 11.56.23 in the women's event.
Brett Morse successfully defended his discus title with a throw of 58.69m, as did Awen Rosser in the women's event with 39.54m.
There was also success for Cardiff-based pole vaulter Holly Bradshaw, returning to action after 16 months out injured, whose 4.50m led the field and also achieved the mark necessary to qualify for the World Championships.
The world's largest uncut diamond has failed to sell at a London auction after bids did not reach the minimum reserve price.
[NEXT_CONCEPT]
Rhys Williams successfully defended his 400m hurdles title at the Welsh Championships in Cardiff on Saturday.