Panama Papers: How to tackle 11m documents
Nobody in the Panorama team had ever seen anything quite like it.
One of our producers, James Oliver, is a member of the International Consortium of Investigative Journalists (ICIJ) and the Belfast office where he works has investigated big data leaks in the past.
The Luxembourg leaks that Panorama investigated with French TV in 2012 contained 28,000 documents. And the HSBC files that we investigated with the ICIJ last year held 60,000 documents.
But the Panama Papers were on a completely different scale. Where do you begin when you have 11 million documents to look at?
Needle in the haystack
This was a project that would take 10 months and the budget was stretched, so most of the searching fell to just two researchers.
David Thompson and Conor Spackman spent hundreds of hours going through document after document, as they searched for the journalistic needle in the haystack.
The data was fascinating and frustrating in equal measure.
One of Conor's first searches found Prime Minister David Cameron's late father and the offshore investment fund he helped set up.
But the vast majority of documents in the database were of little value. There were endless company formations, appointments of nominee directors, invoices for fees - the routine rubber stamping of the offshore industry at work.
Days, sometimes weeks, would go by with little of interest to report.
And it was easy to get lost in the information. A search on one individual might yield a couple of hundred documents. Halfway through searching through those, another lead might appear on someone else with another few hundred documents attached.
The opportunities to get diverted and distracted were limitless.
But slowly the stories started to emerge. There were 1,464 documents about four mysterious offshore companies run by Bank Rossiya in Russia.
They included dozens of suspicious contracts and loans that appeared to have been set up to siphon money from the state.
The owner of two of the companies was one of President Vladimir Putin's oldest friends - a cellist called Sergei Roldugin.
It was still a big task to unpick what was going on and it might have taken Panorama years to do it alone.
But the ICIJ has introduced a new, collaborative way of working. Almost 400 journalists around the world had agreed to help each other, rather than compete for exclusive stories.
This co-operation transformed the way we worked. David Thompson talked regularly to our colleagues in Germany and in Switzerland. They divided up the documents and the tasks between them.
If we needed some additional help in Russia, there were ICIJ colleagues we could call for help. The Guardian and the ICIJ shared their findings too.
This small team put the pieces together and uncovered a suspected billion-dollar money laundering scheme that led to the door of the Kremlin.
Similar collaborations sprang up all round the world. Johannes Kristjansson from Reykjavik Media helped journalists working on the story about the secret offshore company owned by Icelandic Prime Minister Sigmundur Gunnlaugsson with his wife.
When we needed an expert to explain the difference between tax avoidance and evasion in Panama, our Panamanian partner was happy to help.
Investigative 'candy shop'
The ICIJ set up a forum where all members were encouraged to share their stories and their ideas.
Journalists with mutual interests set up threads which any member could join. The titles give some idea of the range of stories being covered - Three UK Crooks, Oil for Fuel Scandal and Yachts, Vessels and Cruises to name just a few.
It was like being given the keys to the investigative candy shop. You could pick which stories you were interested in, take some advice and then plunge back into the documents to find the evidence.
Panorama doesn't take stories from third parties. We always investigate and verify every aspect of our programmes ourselves.
But through the collaboration of the ICIJ we had an extra layer of protection. As we came closer to broadcast, trusted partners started to exchange and check each other's stories.
We were grateful to the Swiss journalist Oliver Zihlmann and the German journalist Petra Blum, who spotted a potential problem in a particularly complicated part of the Russian story at the eleventh hour.
Extra pairs of eyes were needed, as we have rarely made so many complicated allegations against so many people.
We wrote to 11 people and organisations in the Russian story alone to offer them a right of reply. Most were extremely wealthy and more than capable of taking legal action if we made a mistake.
The pressure intensified last weekend. BBC News was fully behind the story and that meant a huge demand for material from radio, TV, online and social media.
The international collaboration paid off. The volume and quality of stories from our partners ensured that the Panama Papers dominated headlines around the world.
It feels like a sea change in the way investigative journalists work.
The reporters on the Suddeutsche Zeitung who obtained the leak could have kept the documents for themselves, but they may never have been fully investigated.
By sharing - rather than scooping - they created the biggest story about tax havens the world has ever seen.