text:
January 23, 2017 by dominiccummings
Specialist maths schools – some facts
The news reports that the Government will try to promote more 'specialist maths schools' similar to the King's College and Exeter schools.
The idea for these schools came when I read about Perelman, the Russian mathematician who in 2003 suddenly posted on arXiv a solution to the Poincaré Conjecture, one of the most important open problems in mathematics. Perelman went to one of the famous Russian specialist maths schools that were set up by one of the most important mathematicians of the 20th Century, Kolmogorov.
I thought – a) given the fall in standards in maths and physics because of the corruption of the curriculum and exams started by the Tories and continued by Blair, b) the way in which proper teaching of advanced maths and physics is increasingly limited to a tiny number of schools many of which are private, and c) the huge gains for our civilisation from the proper education of the unusual small fraction of children who are very gifted in maths and physics, why not try to set up something similar.
Gove's team therefore pushed the idea through the DfE. Dean Acheson, US Secretary of State, said, 'I have long been the advocate of the heretical view that, whatever political scientists might say, policy in this country is made, as often as not, by the necessity of finding something to say for an important figure committed to speak without a prearranged subject.' This is quite true (it also explains a lot about how Monnet created the ECSC and EEC). Many things that the Gove team did relied on this. We prepared the maths school idea and waited our chance. Sure enough, the word came through from Downing Street – 'the Chancellor needs an announcement for the Budget, something on science'. We gave them this, he announced it, and bureaucratic resistance was largely broken.
If interested in some details, then look at pages 75ff of my 2013 essay for useful links. Other countries have successfully pursued similar ideas, including France for a couple of centuries and Singapore recently.
One of the interesting aspects of trying to get them going was the way in which a) the official 'education world' loathed not just the idea but also the idea about the idea – they hated thinking about 'very high ability' and specialist teaching; b) when I visited maths departments they all knew about these schools because university departments in the West employ a large number of people who were educated in these schools but they all said 'we can't help you with this even though it's a good idea because we'd be killed politically for supporting "elitism" [fingers doing quote marks in the air], good luck I hope you succeed but we'll probably attack you on the record.' They mostly did.
The only reason why the King's project happened is because Alison Wolf made it a personal crusade to defeat all the entropic forces that elsewhere killed the idea (with the exception of Exeter). Without her it would have had no chance. I found few equivalents elsewhere and where I did they were smashed by their VCs.
A few points…
1) Kolmogorov-type schools are a particular thing. They undoubtedly work. But they are aimed at a small fraction of the population. Given what the products of these schools go on to contribute to human civilisation they are extraordinarily cheap. They are also often a refuge for children who have a terrible time in normal schools. If they were as different to normal kids in a negative sense as they are in a positive sense then there would be no argument about whether they have 'special needs'.
2) Don't believe the rubbish in things like Gladwell's book about maths and IQ. There is now very good data on this particularly in the form of the unprecedented SMPY multi-decade study. Even a short crude test at 11-13 gives very good predictions of who is likely to be very good at maths/physics. Further there is a strong correlation between performance at the top 1% / 1:1,000 / 1:10,000 level and many outcomes in later life such as getting a doctorate, a patent, writing a paper in Science and Nature, high income, health etc. The education world has been ~100% committed to rejecting the science of this subject though this resistance is cracking.
This chart shows the SMPY results (maths ability at 13) for the top 1% of maths ability broken down into quartiles 1-4: the top quartile of the top 1% clearly outperforms viz tenure, publication and patent rates.
3) The arguments for Kolmogorov schools do not translate to arguments for selection in general – ie. they are specific to the subject. It is the structure of maths and the nature of the brain that allows very young people to make rapid progress. These features are not there for English, history and so on. I am not wading into the grammar school argument on either side – I am just pointing out a fact that the arguments for such maths schools are clear but should not be confused with the wider arguments over selection that involve complicated trade-offs. People on both sides of the grammar debate should, if rational, be able to support this policy.
4) These schools are not 'maths hot houses'. Kolmogorov took the children to see Shakespeare plays, music and so on. It is important to note that teaching English and other subjects is normal – other than you are obviously dealing with unusually bright children. If these children are not in specialist schools, then the solution is a) specialist maths teaching (including help from university-level mathematicians) and b) keeping other aspects of their education normal. Arguably the greatest mathematician in the world, Terry Tao, had wise parents and enjoyed this combination. So it is of course possible to educate such children without specialist schools but the risks are higher that either parents or teachers cock it up.
5) Extended wisely across Britain they could have big benefits not just for those children and elite universities but they could also play an important role in raising standards generally in their area by being a focus for high quality empirical training. One of the worst aspects of the education world is the combination of low quality training and resistance to experiments. This has improved since the Gove reforms but the world of education research continues to be dominated by what Feynman called 'cargo cult science'.
6) We also worked with a physicist at Cambridge, Professor Mark Warner, to set up a project to improve the quality of 6th form physics. This project has been a great success thanks to his extraordinary efforts and the enthusiasm of young Cambridge physicists. Thousands of questions have been answered on their online platform from many schools. This project gives kids the chance to learn proper problem solving – that is the core skill that the corruption of the exam system has devalued and increasingly pushed into a ghetto of of private education. Needless to say the education world also was hostile to this project. Anything that suggests that we can do much much better is generally hated by all elements of the bureaucracy, including even elements such as the Institute of Physics that supposedly exist to support exactly this. A handful of officials helped us push through projects like this and of course most of them have since left Whitehall in disgust, thus does the system protect itself against improvement while promoting the worst people.
7) This idea connects to a broader idea. Kids anywhere in the state system should be able to apply some form of voucher to buy high quality advanced teaching from outside their school for a wide range of serious subjects from music to physics.
8) One of the few projects that the Gove team tried and failed to get going was to break the grip of GCSEs on state schools (Cameron sided with Clegg and although we cheated a huge amount through the system we hit a wall on this project). It is extremely wasteful for the system and boring for many children for them to be focused on existing exams that do not develop serious skills. Maths already has the STEP paper. There should be equivalents in other subjects at age 16. There is nothing that the bureaucracy will fight harder than this and it will probably only happen if excellent private schools decide to do it themselves and political pressure then forces the Government to allow state schools to do them.
Any journalists who want to speak to people about this should try to speak to Dan Abramson (the head of the King's school), Alison Wolf, or Alexander Borovik (a mathematician at Manchester University who attended one of these schools in Russia).
It is hopeful that No10 is backing this idea but of course they will face determined resistance. It will only happen if at least one special adviser in the DfE makes it a priority and has the support of No10 so officials know they might as well fight about other things…
This is the most interesting comment probably ever left on this blog and it is much more interesting than the blog itself so I have copied it below. It is made by Borovik, mentioned above, who attended one of these schools in Russia and knows many who attended similar…
'There is one more aspect of (high level) selective specialist mathematics education that is unknown outside the professional community of mathematicians.
I am not an expert on "gifted and talented" education. On the other hand, I spent my life surrounded by people who got exclusive academically selective education in mathematics and physics, whether it was in the Lavrentiev School in Siberia, or Lycée Louis-le-Grand in Paris, or Fazekas in Budapest, or Galatasaray Lisesi (aka Lycée de Galatasaray) in Istanbul — the list can be continued.
The schools have nothing in common, with the exception of being unique, each one in its own way.
I had research collaborators and co-authors from each of the schools that Ilisted above. Why was it so easy for us to find a common language?
Well, the explanation can be found in the words of Stanislas Dehaene, the leading researcher of neurophysiology of mathematical thinking:
"We have to do mathematics using the brain which evolved 30 000 years ago for survival in the African savanna."
In humans, the speed of totally controlled mental operations is at most 16 bits per second. Standard school maths education trains children to work at that speed.
The visual processing module in the brain crunches 10,000,000,000 bits per second.
I offer a simple thought experiment to the readers who have some knowledge of school level geometry.
Imagine that you are given a triangle; mentally rotate it about the longest side. What is the resulting solid of revolution? Describe it. And then try to reflect: where the answer came from?
The best kept secret of mathematics: it is done by subconsciousness.
Mathematics is a language for communication with subconsciousness.
There are four conversants in a conversation between two mathematicians: two people and two their "inner", "intuitive" brains.
When mathematicians talk about mathematics face-to-face, they
* frequently use language which is very fluid and informal;
* improvised on the spot;
* includes pauses (for a lay observer—very strange and awkwardly timed) for absorbtion of thought;
* has almost nothing in common with standardised mathematics "in print".
Mathematician is trying to convey a message from his "intuitive brain" directly to his colleagues' "intuitive brain".
Alumni of high level specialist mathematics schools are "birds of feather" because they have been initiated into this mode of communication at the most susceptible age, as teenagers, at the peak of intensity of their socialisation / shaping group identity stream of self-actualisation.
In that aspect, mathematics is not much different from arts. Part of the skills that children get in music schools, acting schools, dancing school, and art schools is the ability to talk about music, acting, dancing, art with intuitive, subconscious parts of their minds — and with their peers, in a secret language which is not recognised (and perhaps not even registered) by uninitiated.
However, specialist mathematics schools form a continuous spectrum from just ordinary, with standard syllabus, but good schools with good maths teachers to the likes of Louis-le-Grand and Fazekas. My comments apply mostly to the top end of the spectrum. I have a feeling that the Green Paper is less ambitious and does not call for setting up mathematics boarding schools using Chetham's School of Music as a model. However, middle tier maths school could also be very useful — if they are set up with realistic expectations, properly supported, and have strong connections with universities.'
A Borovik
Posted in Education, My essay 'Some thoughts on education and...'	| Tagged Cameron, Gove, IQ, kolmogorov, maths schools, Osborne, perelman | 21 Comments
January 9, 2017 by dominiccummings
On the referendum #21: Branching histories of the 2016 referendum and 'the frogs before the storm'
'Politics is gambling for high stakes with other people's money… Politics is a job that can be compared with navigation in uncharted waters. One has no idea how the weather or the currents will be or what storms one is in for. In politics, there is the added fact that one is largely dependent on the decisions of others, decisions on which one was counting and which then do not materialise; one's actions are never completely one's own. And if the friends on whose support one is relying change their minds, which is something that one cannot vouch for, the whole plan miscarries… One's enemies one can count on – but one's friends!' Bismarck.
'The most important thing is not to fool yourself and you are the easiest person to fool.' Feynman.
'He lies like an eyewitness.' Russian proverb.
In January 2014 I left the Department for Education and spent the next 18 months away from politics. A few days after the 2015 election I wrote a blog about Michael Gove's new job touching on the referendum. When I wrote it I assumed I would carry on studying and would not be involved in it. About ten days later I was asked by an assortment of MPs, rich businessmen, and campaigners including Matthew Elliott to help put together an organisation that could fight the referendum. I was very reluctant and prevaricated but ended up agreeing. I left my happy life away from SW1 and spent eight weeks biking around London persuading people to take what was likely to be a car crash career decision – to quit their jobs and join a low probability proposition: hacking the political system to win a referendum against almost every force with power and money in politics. In September we had an office, in October 'Vote Leave' went public, in April we were designated the official campaign, 10 weeks later we won.
Why and how? The first draft of history was written in the days and weeks after the 23 June and the second draft has appeared over the past few weeks in the form of a handful of books. There is no competition between them. Shipman's is by far the best and he is the only one to have spoken to key people. I will review it soon. One of his few errors is to give me the credit for things that were done by others, often people in their twenties like Oliver Lewis, Jonny Suart, and Cleo Watson who, unknown outside the office, made extreme efforts and ran rings around supposed 'experts'. His book has encouraged people to exaggerate greatly my importance.
I have been urged by some of those who worked on the campaign to write about it. I have avoided it, and interviews, for a few reasons (though I had to write one blog to explain that with the formal closing of VL we had made the first online canvassing software that really works in the UK freely available HERE). For months I couldn't face it. The idea of writing about the referendum made me feel sick. It still does but a bit less.
For about a year I worked on this project every day often for 18 hours and sometimes awake almost constantly. Most of the 'debate' was moronic as political debate always is. Many hours of life I'm never getting back were spent dealing with abysmal infighting among dysfunctional egomaniacs while trying to build a ~£10 million startup in 10 months when very few powerful people thought the probability of victory was worth the risk of helping us. (Two rare heroes who put up a lot of their own money and supported the team were Peter Cruddas and Stuart Wheeler.) Many of those involved regarded their TV appearances as by far the most important aspect of the campaign. Many regarded Vote Leave as 'the real enemy'.
It is hard to explain the depth of TV derangement that gobbles up SW1 souls. Much of politics involves very similar tragi-comic scenes re-created by some of the basic atoms of human nature – fear, self-interest and vanity. The years, characters, and contexts change, the atoms shuffle, but the stories are the same year after year, century after century. Delusions and vanity dominate 'rationality' and 'public service'. Progress, when it comes, is driven by the error-correcting institutions of science and markets when political institutions limit the damage done by decision makers at the apex of centralised hierarchies. It rarely comes from those people, and, when it does, it is usually accidental or incidental to their motives.
Discussions about things like 'why did X win/lose?' are structured to be misleading and I could not face trying to untangle everything. There are strong psychological pressures that lead people to create post facto stories that seem to add up to 'I always said X and X happened.' Even if people do not think this at the start they rapidly construct psychologically appealing stories that overwrite memories. Many involved with this extraordinary episode feel the need to justify themselves and this means a lot of rewriting of history. I also kept no diary so I have no clear source for what I really thought other than some notes here and there. I already know from talking to people that my lousy memory has conflated episodes, tried to impose patterns that did not actually exist and so on – all the usual psychological issues. To counter all this in detail would require going through big databases of emails, printouts of appointment diaries, notebooks and so on, and even then I would rarely be able to reconstruct reliably what I thought. Life's too short.
I've learned over the years that 'rational discussion' accomplishes almost nothing in politics, particularly with people better educated than average. Most educated people are not set up to listen or change their minds about politics, however sensible they are in other fields. But I have also learned that when you say or write something, although it has roughly zero effect on powerful/prestigious people or the immediate course of any 'debate', you are throwing seeds into a wind and are often happily surprised. A few years ago I wrote something that was almost entirely ignored in SW1 but someone at Harvard I'd never met read it. This ended up having a decisive effect on the referendum.
A warning. Politics is not a field which meets the two basic criteria for true expertise (see below). An effect of this is that arguments made by people who win are taken too seriously. People in my position often see victory as confirmation of ideas they had before victory but people often win for reasons they never understand or even despite their own efforts. Cameron's win in 2015 was like this – he fooled himself about some of the reasons why he'd won and this error contributed to his errors on the referendum. Maybe Leave won regardless of or even despite my ideas. Maybe I'm fooling myself like Cameron. Some of my arguments below have as good an empirical support as is possible in politics (i.e. not very good objectively) but most of them do not even have that. Also, it is clear that almost nobody agrees with me about some of my general ideas. It is more likely that I am wrong than 99% of people who work in this field professionally. Still, cognitive diversity is inherently good for political analysis so I'll say what I think and others will judge if there's anything to learn.
Apologies for the length but I didn't have time to make it shorter. The next ones will be short.
Reality has branching histories, not 'a big why'
Much political analysis revolves around competing simple stories based on one big factor such that, in retrospect, 'it was always clear that immigration would trump economic interest / Cameron's negotiation was never going to be enough / there is an unstoppable populist tide', and so on. Alternatives are quickly thought to have been impossible (even if X argued the exact opposite repeatedly). The big event must have had an equally big single cause. Confirmation bias kicks in and evidence seeming to suggest that what actually happened would happen looms larger. People who are quite wrong quickly persuade themselves they were 'mostly right' and 'had a strong feeling' unlike, of course, the blind fools around them. Soon our actual history seems like the only way things could have played out. Brexit had to happen. Trump had to win.
You see these dynamics all the time in historical accounts. History tends to present the 1866 war between Prussia and Austria as almost inevitable but historians spend much less time on why Bismarck pulled back from war in 1865 and how he might have done the same in 1866 (actually he prepared the ground so he could do this and he kept the option open until the last minute). The same is true about 1870. When some generals tried to bounce him into a quick preventive war against Russia in the late 1880s he squashed them flat warning against tying the probability of a Great Power war to 'the passions of sheep stealers' in the Balkans (a lesson even more important today than then). If he had wanted a war, students would now be writing essays on why the Russo-German War of 1888 was 'inevitable'. Many portray the war that broke out in August 1914 as 'inevitable' but many decisions in the preceding month could have derailed it, just as decisions derailed general war in previous Balkan crises. Few realise how lucky we were to avoid nuclear war during the Cuban Missile crisis (cf. Vasili Arkhipov) and other terrifying near-miss nuclear wars. The whole 20th Century history of two world wars and a nuclear Cold War might have been avoided if one of the assassination attempts on Bismarck had succeeded. If Cohen-Blind's aim had been very slightly different in May 1866 when he fired five bullets at Bismarck, then the German states would certainly have evolved in a different way and it is quite plausible that there would have been no unified German army with its fearsome General Staff, no World War I, no Lenin and Hitler, and so on. The branching histories are forgotten and the actual branch taken, often because of some relatively trivial event casting a huge shadow (perhaps as small as a half-second delay by Cohen-Blind), seems overwhelmingly probable. This ought to, but does not, make us apply extreme intelligent focus to those areas that can go catastrophically wrong, like accidental nuclear war, to try to narrow the range of possible histories but instead most people in politics spend almost all their time on trivia.
We evolved to make sense of this nonlinear and unpredictable world with stories. These stories are often very powerful. On one hand the work of Kahneman et al on 'irrationality' has given an exaggerated impression. The fact that we did not evolve to think as natural Bayesians does not make us as 'irrational' as some argue. We evolved to avoid disasters where the probability of disaster X happening was unknowable but the outcome was fatal. Rationality is more than 'Bayesian updating'. On the other hand our stories do often obscure the branching histories of reality and they remain the primary way in which history is told. The mathematical models that illuminate complex reality in the physical sciences do not help us much with history yet. Only recently has reliable data science begun to play an important role in politics.
Andrew Marr wrote recently about the referendum with a classic post facto 'big event must be caused by one big factor' story:
'Connected to this is the big "why?". I don't think we voted to leave the EU because of clever tacticians or not-quite-clever-enough pollsters, or even because Johnson decided that one of his columns was better than another. I think we voted to leave because so many British people had been left behind economically and culturally for so long, and were furious about it; and because, from the 2008 financial crisis onwards, they had accumulated so much contempt for the political elites. In these circumstances any referendum narrows down to a single question: "Are you happy with the way things are?" The answer was "no".' Andrew Marr, October 2016.
'The big why?' is psychologically appealing but it is a mistake. In general terms it is the wrong way to look at history and it is specifically wrong about the referendum. If it were accurate we would have won by much more than we did given millions who were not 'happy with the way things are' and would like to be out of the EU reluctantly voted IN out of fear. Such stories oversimplify and limit thinking about the much richer reality of branching histories.
Branching histories in 2016: three powerful forces, many possible campaigns
Sometimes the outcome of a vote is clear before a campaign starts such that it is reasonable to say 'the campaign didn't matter' other than in the negative sense that, provided it avoids huge disasters, the twists and turns, the exact messages and adverts, thousands of decisions taken and so on very likely had no impact on the binary outcome. For example, Reagan's re-election campaign in 1984 or Blair's re-election campaign in 2001 were campaigns like this. Both won by so much and were clearly predicted by very large and historically very unusual poll leads well in advance. It is not plausible to say that the weeks of campaigning affected who won. At most the campaigns affected the scale of victory.
The referendum was not like this. Throughout the second half of 2015 and the beginning of 2016 the averages of polls – the only sensible way to look at polls – showed clear IN leads. All polls showed significant shifts towards Leave in the last five weeks (then a shift towards Remain at the end but this was at least partly because London-based pollsters changed their methodology thinking that they were making them more accurate – they fooled themselves). Polls tracking deeper attitudes that had been consistent for years suddenly changed in the last few months in ways that were significant given the close outcome. Recent claims that the polls 'really' showed Leave ahead all the time should be taken with very large pinches of salt given their dodgy statistical claims, charlatan authors like Matt Goodwin (who treats data dishonestly), and the inherent impossibility of discovering the truth of such a question.
One example from our private ICM polls (I will post the data tables for all these): Vote Leave asked people to choose between these options regularly to probe attitudes to the EU that are more informative than just the referendum question. The 11 point gain for 'strong out' is much bigger than the margin of error, is supported by other data, and is clearly significant.
The cold reality of the referendum is no clear story, no 'one big causal factor', and no inevitability – it was 'men going at it blind'. The result was an emergent property of many individual actions playing out amid a combination of three big forces (see below). Many of these actions were profoundly nonlinear and interdependent and the result that we actually witnessed was very close. If about 600,000 people – just over 1% of registered voters – had decided differently, IN would have won. This is a small enough margin that it could easily have happened if quite a few specific events and decisions had turned out differently. If just one person had behaved differently the dominant story now would be 'the economy was always going to trump a revolt against the elites, the status quo and "the economy stupid" always win' – which is what the overwhelming majority of pundits said before 23 June and in some cases had drafted for their columns after the vote.
For example, if Michael Gove had stayed out of the campaign then Vote Leave would almost certainly have either collapsed (which it nearly did anyway) or been forced into fighting the campaign on a losing message like 'Go Global', a firm favourite for many years among a subset of MPs and Farage's inner circle (Leave.EU adopted this as its first slogan) and a total loser with the public. (Therefore another counterfactual: why did Cameron and Osborne not try very hard to get a clear commitment from Gove that all he would do is issue a statement but would carry on with his day job and would not campaign? I hope he would have refused but it was worth a shot and they didn't try very hard.)
Without Boris, Farage would have been a much more prominent face on TV during the crucial final weeks, probably the most prominent face. (We had to use Boris as leverage with the BBC to keep Farage off and even then they nearly screwed us as ITV did.) It is extremely plausible that this would have lost us over 600,000 vital middle class votes.
Without Victoria Woodcock, an absolutely phenomenal manager and by far the single most important person in the management of Vote Leave (and who would have been running Downing Street now but for the Gove-Boris debacle – more branching histories), we would not have been able to build anything like the structure we did and this could easily have cost us the winning margin of votes.
Anybody who says 'I always knew X would win' is fooling themselves. What actually happened was one of many branching histories and in many other branches of this network – branches that almost happened and still seem almost real to me – we lost.
Problems with Vote Leave
This is not a claim that 'we won because of the Vote Leave campaign'. Our campaign failed to do much that we should have done. There were powerful connections between:
infighting over who appeared on broadcast and strategy,
the lack of resources (many kept clear because of the infighting and many used infighting as an excuse to keep clear of something they thought was doomed),
the extreme difficulty of finding a governance system that could work,
four crucial posts held by the wrong people (including the disastrous John Mills as first Chairman),
the fundamental structure of how the media works (see below),
the extreme difficulty of getting prominent people to say on TV what research showed was necessary to win, and
the lack of anything resembling a well-organised mass movement.
Despite many years to prepare, the eurosceptic community had built remarkably little to prepare for the battle. On the ground were many small ineffective and often warring little groups and essentially no serious machinery (though Business for Britain had begun to build a business network). All this had to be built almost entirely from scratch in an environment in which many of those in charge of the small groups were sure we would lose, were less interested in winning than they were in 'preserving our group's identity Dominic', and were keen to get their hands on cash being handed out by Leave.EU on condition that they not contribute to the campaign with Vote Leave. At various points UKIP HQ sent out emails to UKIP activists telling them not to work with Vote Leave and some senior activists were told by Farage's gang that they would lose their UKIP jobs if they helped our ground campaign (luckily most of those out on the ground ignored these instructions but they were disruptive).
The office implemented the winning message in ~125 million leaflets and nearly a billion targeted digital adverts regardless of all complaints. We recruited more active volunteers (~12,000) in 10 months than UKIP in 25 years (~7,000 according to Farage). Our GOTV effort targeted crucial voters identified by traditional polling, a new type of experimental polling, the ground campaign, and the social media campaign, all overseen by the data science team. But until the last 4-5 weeks we had a big problem getting those going on TV to give the same message. The office could only do so much. If Boris, Gove, and Gisela had not supported us and picked up the baseball bat marked 'Turkey/NHS/£350 million' with five weeks to go, then 650,000 votes might have been lost. In the awful weekly campaign committee meetings, there were constant complaints and arguments for variations on 'Go Global' (until all the polls swung our way and people remembered 'I've always said stick with 350 million'.) The Big Three knocked this back despite great pressure.
Some people had spent a quarter of a century talking about things that appealed to about 10% of the population and they would not pay attention to what millions of normal people actually knew and thought ('I've spent years trying to ignore the NHS in elections Dominic and I'm not going to change now' said many like Peter Bone). Media planning was extremely hard. Paul Stephenson's media team of half a dozen, massively outnumbered by hundreds of officials, did a fantastic job but we could have done so much more if more MPs had been more determined and more supportive.
It should be remembered that the net effect of Conservative MPs was strongly supportive of IN. We won despite the net effort of Conservative Party MPs, not because of them, though the support from a small fraction was vital. Although Leave voters were more enthusiastic and determined than Remain voters, Cameron and Osborne were more focused on winning than most Leave MPs were. (Almost all Labour MPs seemed to be in a parallel universe until they got intelligence from their constituencies about postal votes after which they panicked ineffectually.)
Most of the MPs we dealt with were not highly motivated to win and lacked extreme focus, even those who had been boring everybody about this for decades. They sort of wanted to win but they had other priorities. They were very happy having dinner parties and gossiping. They were very happy coming to meetings with people they thought were important. This wasted enormous amounts of time as we had to create a string of Potemkin committees for people to attend while the core team actually did the campaign, then reinvent them as people became convinced that there were other secret meetings that they were being excluded from. They were very happy to be on the Today Programme. But they didn't want to win that much. Not enough to work weekends. Not enough to stop having all their usual skiing holidays and winter beach holidays. Not enough to get out on the streets day after day. Not enough to miss a great shooting weekend. Not enough, most of them, to risk annoying a Prime Minister who they thought would still control their next job after 23 June.
This lack of motivation is connected to another important psychology – the willingness to fail conventionally. Most people in politics are, whether they know it or not, much more comfortable with failing conventionally than risking the social stigma of behaving unconventionally. They did not mind losing so much as being embarrassed, as standing out from the crowd. (The same phenomenon explains why the vast majority of active fund management destroys wealth and nobody learns from this fact repeated every year.)
Our core campaign team were not like this. They sacrificed weekends, holidays, and family events. They worked like dogs week in week out for little money often treated with appalling rudeness by people calling from their beach loungers (Boris, Gisela and Gove were three notable exceptions and all three were liked by junior staff partly because of their good, therefore rare, manners). We were happy to risk looking stupid to win. We knew that almost nobody in SW1 understood or agreed with what we were doing. We also knew we had more chance of winning if we did not explain a lot of it – most importantly the entire digital and data science element which (combined with the ground campaign and GOTV) gave us a chance to exploit strong network effects (and which we hid from the Board and MPs, see HERE).
Example… We were urged by everyone to hire a big advertising agency and do traditional posters. 'When can we discuss our posters?' I was asked constantly by people who would then try to explain to me their creative ideas ('we need another Labour Isn't Working, Dominic, I've got an idea for a picture of the globe and arrows…'). One of the few reliable things we know about advertising amid the all-pervasive charlatanry is that, unsurprisingly, adverts are more effective the closer to the decision moment they hit the brain. Instead of spending a fortune on an expensive agency (with 15% going to them out of 'controlled expenditure') and putting up posters to be 'part of the national conversation' weeks or months before the vote, we decided to 1) hire extremely smart physicists to consider everything from first principles, 2) put almost all our money into digital (~98%), 3) hold the vast majority of our budget back and drop it all right at the end with money spent on those adverts that experiments had shown were most effective (internal code name 'Waterloo'). When things are digital you can be more empirical and control the timing. The world of advertising agencies and PR companies were sure we had screwed up because they did not see what we were doing. (Tim Bell told everybody we were doomed because we hadn't hired one of his companies.) This points to another important issue – it is actually hard even for very competent and determined people to track digital communication accurately, and it is important that the political media is not set up to do this. There was not a single report anywhere (and very little curiosity) on how the official Leave campaign spent 98% of its marketing budget. There was a lot of coverage of a few tactical posters.
There were some MP heroes.
Example… Steve Baker often disagreed with me, sometimes very strongly, but he was a rare person in the campaign – an honest man. Not only did Steve win some important Parliamentary battles he also played a vital role during the attempted coup of 25 January. If he had thrown in his lot with the coup, it might have proved fatal. Instead he spoke honestly about the situation. We did not agree and we were both under pressure from a set of people who thought that 'if they [HQ/MPs] control the campaign we will lose, we [HQ/MPs] must control it'. We came to an agreement that we both stuck to. With five weeks to go, there was an attempt to revive the coup by a couple of VL Board members working with players from the January coup like Malcolm Pearson. The demand was to replace the Big Three (Boris, Gisela, Gove) and the core campaign team with Farage, and replace £350 million / NHS with 'go global' trade babble. This didn't get past the usual weekend boozy chats partly because of Steve Baker telling them he thought it a mad plan. This also shows how volatile the situation was right until the end and how few prominent eurosceptics even then understood that a) the £350 million / NHS argument was necessary to win and b) their 'go global' message was a total loser.
Other MPs also made significant personal sacrifices – backbenchers like Anne Marie Trevelyan and Graham Stringer, and ministers like George Eustice and Dominic Raab.
Rough balance of forces
The IN side started with huge structural advantages.
IN started in 2015 well ahead in the polls and had the advantage of having the status quo on its side which is intrinsically easier to explain than change is, as lots of historical data around the world shows. Usually the 'change' campaign has to start considerably ahead in order to win as it loses support as the campaign goes on. This argument was even stronger with something so much bigger and more complex like the EU. VL had to persuade millions of people to risk a profound change. Those on the IN side made this point repeatedly for many months. They were right then. After 23 June many of them say the exact opposite – it's so complex to explain all the wonders of the EU, they say, and so easy to argue for change. This is laughable.
IN had the government at its heart including the Downing Street machine, the Cabinet Office, and Government departments and agencies all of which added up to thousands of people including hundreds of press officers. Cameron and Heywood also instructed Permanent Secretaries not to share EU material with Secretaries of State supporting Vote Leave in order that they did not have access to new information about all the ways in which EU law affected policy. (In general Whitehall has made great efforts to hide the scope of EU control. It also preserves, Potemkin-style, old processes like circulating Cabinet papers 'for approval' where the only acceptable response is 'approve' – it is not actually legally possible not to 'approve' but still the papers are sent round via the absurd red box system daily.) VL had a few dozen effective people and no access to the official machine other than some leaks. We had a research team of about five. MPs proved largely useless in helping this team.
IN controlled one side of the renegotiation and its timing. VL was at the mercy of events and could not get any ministers supporting us until the process ended.
IN controlled the timing of the referendum. VL had to plan resources on the basis of many scenarios.
IN controlled the Cabinet and junior ministers – bribes for support and threats to deter. They had the chance to set the terms for how ministers engaged in the campaign (though they partly blew this). VL had to meet ministers in secret, could guarantee them no jobs, and (as was pointed out to me by many) could not dodge the basic truth that purely from a personal career perspective it was usually better to support the PM.
IN controlled the governing party and the Parliamentary timetable and procedures. VL had to work with a small number of MPs many of whom had spent many years in constant opposition to their own leadership and were unused to any sort of discipline or collective action.
IN set the legal rules. VL faced a huge imbalance in how these worked. For example, Cameron even during the official campaign could do huge events at places like the British Museum and the IN campaign did not have to account for such events as part of their £7 million. Meanwhile VL was told by the Electoral Commission that if people we did not even know put up huge signs that appeared on TV we might get billed for them. There were many other consequences of the imbalance. E.g. the Government's legal timetable meant we had to commit before the official start of the campaign to a load of activity that would occur after the official start of the campaign without knowing if we would be the official campaign and therefore legally entitled to spend this money. We therefore had to choose between either a) not do various things, be sure we would not break the law, and lower the chances of winning or b) do the right thing for the campaign and riski being judged to have broken the law. Obviously we did (b) though we had to hide this choice from some of those on our Board as this was exactly the sort of thing some of them were very weak about.
IN had access to huge resources – financial, personnel etc. IN had the support of almost every entity with power in Britain, Europe, and the world from the senior civil service to the CBI to the big investment banks, to Obama and the world bureaucracy (G20, UN, IMF etc). Very few senior people were prepared to risk supporting us. Those who did mostly did so in a small way and on their own terms without getting involved in our campaign. While IN could send out name after name to deliver their message, we could depend on very few names who would deliver our message. The Government machine, the Commission, and the Cabinet Office were effective in scaring off prominent people from supporting us; many of them told us (some embarrassed) about the phone calls they'd had and their 'duty to shareholders' and so on. Advanced media planning was almost impossible and we had to shuffle things around at short notice constantly. IN had millions more than us before the campaign ever started and used this money for direct voter communication. We could not afford this. We sent out one 10 million voter mailing to people identified by the physicists just before the spending limits started and we could only do this by tricking some of those on our Board about the numbers. (I was also helped by Peter Cruddas saying, 'Don't worry about the fundraising situation, don't listen to everybody panicking, just do whatever it takes to do the campaign, if the money doesn't come I guarantee I'll put in whatever you need'. I knew I could trust him. This gave us vital flexibility and also meant we could ignore some of those on the Board who were more focused on whether they may be liable for a bill post-23/6 than they were on winning.)
IN had the support of most journalists and senior management in the main broadcasters. The broadcasters let the Government set the agenda on TV for almost the entire campaign, apart from ten crucial days after the immigration numbers on 26 May. VL had the support of some powerful papers but we were overwhelmed on TV news. (Two broadcast journalists who were conspicuous by their unusual professionalism and determination to act fairly despite the behaviour of some of their management were Laura K and Allegra Stratton.)
IN started with legal access to vast amounts of electoral data from at least three political parties, unofficial / illegal access to vast amounts of data from things like CCHQ data and the Crosby/Messina models built during the campaign, and vast amounts of commercial data. (CCHQ laughably claimed that there were 'Chinese walls' that prevented any abuse of Party data.) VL had none of these things. We could not even afford to buy standard commercial datasets (though the physicists found ingenious ways around this). We had no way even to acquire the electoral roll until the official process allowed us in early 2016, after which we had to wait a couple of months for LAs to fulfil their legal obligations to provide us with the data (which they did patchily and often late).
IN had a great boost to its fortunes in the form of a network linking Nigel Farage, Aaron Banks, assorted peers (e.g. Malcolm Pearson), MPs (e.g. Bill Cash), businessmen (e.g. Richard Smith), and a handful of Vote Leave Board members (including the one-time Chairman John Mills) and some staff foisted on us (one of whom won the title of the most repellent person I've met in politics – Nigel Griffiths, an ex-MP who some female staff refused to be in the same room with). Farage put off millions of (middle class in particular) voters who wanted to leave the EU but who were very clear in market research that a major obstacle to voting Leave was 'I don't want to vote for Farage, I'm not like that'. He also put off many prominent business people from supporting us. Over and over they would say 'I agree with you the EU is a disaster and we should get out but I just cannot be on the same side as a guy who makes comments about people with HIV'.
On 25 January 2016 a network of these characters launched a coup. But for the actions of Stephen Parkinson, Paul Stephenson, and Victoria Woodcock (supported by most but not all of the office) it would have succeeded. This would have given control of the official campaign to the Farage crowd. They ran with vapid slogans like 'Be in the know'. Ironically for a group of people who claim to be anti-SW1 they rehashed the classic losing SW1 eurosceptic trope for 25 years – 'Go Global' – showing how little they understood the electorate and mass communication. They rejected the connection between immigration, £350 million and the NHS, which was absolutely vital, as the IN side has said after 23 June (see below). They published dumb offensive videos. They talked about privatising the NHS. They built little grassroots organisation and their claims about social media were (and remain ) laughable. Farage himself admitted after 23 June that they did not have the organisation to run the campaign if they had won designation: 'quite what we would have done if we had got it I'm not really sure!', which sums them up (Shipman, Location 4,150). The media would have covered this gang's official campaign as a version of their own book – a bunch of childish dodgy boozers on an ego trip.
Before the 2015 election Farage said to me at Stuart Wheeler's that he knew he could not be the leading face of the campaign – 'I'm one of the generals but I can't lead the army' he said, to my relief. When I next saw him in the summer, I was amazed at how his tune had changed, his obsession with the debates, and his pessimism. One can only understand some of the behaviour from those around Farage if you realise that much of their operation was about positioning Farage for what they assumed would be defeat.
One of the biggest problems during the campaign and biggest misconceptions after concerns this issue. Those who argued 'we need one campaign' were wrong. Those who argue now 'we would have won by more if there'd been one campaign' are wrong. One campaign would have meant total bedlam and 60-40 defeat.
If MPs had had extreme focus on winning then they would not have used Farage as leverage against us viz official designation and therefore much of the infighting could have been avoided as Farage would have done a sensible deal with us early, realising much earlier that we would not compromise over him running the campaign under any circumstances. By encouraging Farage to think that he could get a much more prominent position, people like Bill Cash nearly destroyed everything.
Given all these huge advantages, if their campaign had been of equal effectiveness to Vote Leave then, all else remaining equal, Cameron would almost certainly (>95% likely) have won.
Why did all these forces not add up to overwhelming and devastating firepower? If you want to understand the combination of things that gives us largely dysfunctional government and therefore undermined the IN campaign – a mix of selecting and promoting the wrong people, wrong education and training, bad incentives, anti-adaptive institutions and so on – then read this in which I explain in detail why Whitehall does not and cannot work properly.
The approximate truth
The closest approximation to the truth that we can get is that Leave won because of a combination of 1) three big, powerful forces with global impact: the immigration crisis, the financial crisis, and the euro crisis which created conditions in which the referendum could be competitive; 2) Vote Leave implemented some unrecognised simplicities in its operations that focused attention more effectively than the other side on a simple and psychologically compelling story, thus taking advantage of those three big forces; and 3) Cameron and Osborne operated with a flawed model of what constitutes effective political action and had bad judgement about key people (particularly his chief of staff and director of communications) therefore they made critical errors. Even if (1) and (2) had played out the same, I think that if that duo had made one of a few crucial decisions differently they would very likely have won.
When I started to research opinion in 2014-15 and compared it to my experience of the euro campaign (1999-2002), it was clear three forces had changed opinion on the EU.
1) The immigration crisis. 15 years of immigration and, recently, a few years of the migration crisis from the East and Africa, dramatically portrayed on TV and social media, had a big effect. In 2000, focus groups were already unhappy with immigration but did not regard it as a problem caused by the EU. By 2015, the EU was blamed substantially for the immigration/asylum crisis and this was entangled with years of news stories about 'European courts' limiting action against terrorists and criminals. Actually often these stories concerned the Strasbourg court of the ECHR (not the ECJ) though, ironically, the EU's adoption of its Charter of Fundamental Rights meant that many issues concerning the ECHR became relevant to the EU debate, something that almost nobody in SW1 realised and we tried and largely failed to explain (one of the very few who did understand this was Boris's wife, an accomplished lawyer, who I discussed this with in autumn 2015).
2) The 2008 financial crisis. This undermined confidence in Government, politicians, big business, banks, and almost any entity thought to be speaking for those with power and money. Contra many pundits, Miliband was right that the centre of gravity has swung against free markets. Even among the world of Thatcherite small businesses and entrepreneurs opinion is deeply hostile to the way in which banks and public company executive pay work. Over and over again outside London people would rant about how they had not/barely recovered from this recession 'while the politicians and bankers and businessmen in London all keep raking in the money and us mugs on PAYE are paying for the bailouts, now they're saying we've just got to put up with the EU being crap or else we'll be unemployed, I don't buy it, they've been wrong about everything else…' All those amazed at why so little attention was paid to 'the experts' did not, and still do not, appreciate that these 'experts' are seen by most people of all political views as having botched financial regulation, made a load of rubbish predictions, then forced everybody else outside London to pay for the mess while they got richer and dodged responsibility. They are right. This is exactly what happened.
Many Tory MPs and 'free market' pundits / think tankers are living in a fantasy world in which they want hostility to big business to end even though everybody can see that those who failed largely escaped responsibility and have even gone back to doing the same things. (I've argued since 2001 for big changes on executive pay to almost zero effect. SW1 is full of people who think they're 'defending markets' but are actually defending the opposite – corporate looting. In the 1930s Britain put people in jail because of what happened in the 1920s. We should have done the same after 2008.)
3) The euro crisis. Britain joined the EEC because it was a basket case in the 1970s and 'Europe' was seen as a modernising force that could help us recover and improve the economy and living standards. As the euro crisis hit, millions saw Greece in chaos, even flames, for month after month. This undermined confidence in the EU as a modern successful force – 'it's so bad even Germany's in trouble now because of the euro', 'not even Germany can afford to sort this out', people would say.
Together these three big forces undermined confidence in the EU project as a modern force for progress that brings prosperity and solves problems and pushed it into about 30-35% of the population (younger, richer, better educated) which increasingly saw the EU in terms of 'are you racist / supporter of Farage?' This feeling was central in 1975. It diminished gradually but was still partly there 1999-2002 when I was doing focus groups on the euro. (It is why I had so many arguments at the time with eurosceptics explaining to them that if we accepted Blair's framing of the euro debate as IN/OUT of the EU, we would lose. Our two slogans were therefore 'Europe yes, euro no' and 'Keep the pound, keep control'.)
Second, they undermined confidence in those in charge. There had been strong anti-Westminster feelings growing for over a decade. In 2004 with James Frayne and my uncle I set up the campaign to fight the referendum on the North East Regional Assembly as a training exercise for an EU referendum (then envisaged after Blair's 2005 victory). We came from behind and won 80-20 (not a misprint) despite having almost no money, no support, and the entire North East establishment against us because we exploited this feeling ('politicians talk, we pay' was our slogan). SW1 ignored the result. It did not appreciate the scale of this growing force even after the financial crash and expenses scandal. Normal electoral politics and the structural grip of established political parties fooled insiders about the extent of support for people like Cameron. Cameron won negatively – because he was not Brown or Miliband. There was very little positive feeling for him. They fought the referendum with him and Osborne at the front as if they were fighting Brown or Miliband and asking people to make a choice: this is not how most people saw it.
These three big forces and the failure of the parties to cope, combined with the daily resentment of paying taxes for the bill of the 2008 Crash, meant that in a vote like 2016 where people did not have to vote to stop Brown or Miliband 'stealing my money', millions who were unpersuaded by Cameron/Osborne felt free to vote positively for something ('take back control') and against a duo they disliked, distrusted, and saw as representative of politicians' failure over many years.
These three big forces had global impact and had much more effect on people who pay a normal amount of attention to politics than every speech, article, pamphlet and 'campaign' about the EU over 15 years, the sum total of which had almost no discernible effect.
Those who think I am exaggerating the relative lack of influence of conscious SW1 activity could consider another example – the Gove education reforms 2010-14 (which I was closely involved with). These reforms were one of the most prominent stories of the 2010-15 Government with thousands of stories and broadcast discussions. I researched public attitudes to these reforms after I resigned from government in January 2014 (contrary to widespread belief the Cameron operation spent very little time and resources before 2014 on researching public opinion, they were focused on the media rather than the public). Approximately nothing of our arguments – including the years of speeches by Blair too – had got through to the public.The entire SW1 media debate had approximately no impact on public opinion. People had some idea of some changes if they had kids in school but knew almost nothing of the arguments. Consider how much more motivated people were to learn about this than they were about the EU. (Part of the reason is that the language that Cameron and SW1 generally used was about 'choice, competition' and so on. I was almost totally unsuccessful in persuading people to talk about the issue in a different way which is one of the reasons I spent so little time on communication and almost all my time on management in the DfE. Gove knew the problem but also knew that there was no chance of getting Cameron to do things differently.)
This is relevant to the immigration argument in particular. Many pundits who described themselves as 'modernisers' wrote columns over the years arguing that immigration was an issue because Cameron was making foolish promises about it and the media therefore paid more attention to it. This is wrong. Cameron's foolish promises certainly made his situation worse but it is wrong to think that public interest in an issue is proportional to the attention paid by politicians and newspapers in SW1. The public only pays attention to a tiny subset of issues that politicians and the media bang on about. It is largely impossible to predict which things will catch fire and which will not, though process stories and 'scandals' almost always have zero effect and insiders repeatedly get this wrong. Long before there was any prominent media discussion of 'the Australian points system' you could hear it being discussed in focus group after focus group to an extent that was very surprising to me and was very surprising to every single person I discussed it with, including Farage (who adopted the policy because of focus groups, the causal chain was not – Farage talks >> focus groups respond).
Making these three forces even more powerful was the nature of the reaction from those in charge in the EU and Britain – a general failure not only to grip the problems but even to show that they understood what the problems were. There was clearly no sensible movement for reform of the EU. As it lurched from crisis to crisis, its only response was 'the EU needs more power' (this is, of course, the founding logic of the Monnet-Delors system). The British Government clearly had no sensible plan for dealing with the EU's crises and dysfunction. Worse, their responses were often obviously rubbish, such as the 'tens of thousands' immigration promise that people could see had no chance of being met yet politicians just kept repeating it. People naturally concluded – these guys in London don't grasp the seriousness of the problems, they haven't a clue what to do, and are treating us like idiots. Cameron's renegotiation did not change this view. The Government therefore entered the campaign in a very different state to Wilson in 1975.
These three forces meant that by summer 2015 only about a third of the electorate positively wanted to be inside the EU. Another third strongly wanted to leave and were not worried about the economy. Another fifth had roughly the view that – the EU is rubbish, I'd like to be outside, but I'm worried about the short-term effects on jobs and living standards so maybe I'll vote IN (see the ICM table above). Further, our research showed that the strong Leave third was significantly more enthusiastic about the referendum than the strong Remain third and the swing fifth, and therefore more likely to vote.
Vote Leave exploited these forces
I will go into this in much more detail and I will ignore all management/operational issues here.
Our story rested on five simple foundations that came from listening very hard to what people really knew, thought, and said:
1. 'Let's take back control'. The overall theme. When I researched opinion on the euro the best slogan we could come up with was 'keep control'. I therefore played with variations of this. A lot of people have given me a lot of credit for coming up with it but all I really did was listen. (NB. 'back' plays into a strong evolved instinct – we hate losing things, especially control.)
2. 'The official bill of EU membership is £350 million per week – let's spend our money on our priorities like the NHS instead.' (Sometimes we said 'we send the EU £350m' to provoke people into argument. This worked much better than I thought it would. There is no single definitive figure because there are different sets of official figures but the Treasury gross figure is slightly more than £350m of which we get back roughly half, though some of this is spent in absurd ways like subsidies for very rich landowners to do stupid things.)
Pundits and MPs kept saying 'why isn't Leave arguing about the economy and living standards'. They did not realise that for millions of people, £350m/NHS was about the economy and living standards – that's why it was so effective. It was clearly the most effective argument not only with the crucial swing fifth but with almost every demographic. Even with UKIP voters it was level-pegging with immigration. Would we have won without immigration? No. Would we have won without £350m/NHS? All our research and the close result strongly suggests No. Would we have won by spending our time talking about trade and the Single Market? No way (see below).
NB. Unlike most of those on our side the IN campaign realised the effectiveness of this, as Cooper, Coetze and others said after 23 June. E.g. 'The power of their £350 million a week can't be overstated.' Andrew Cooper, director of strategy for the IN campaign.
Some people now claim this was cynical and we never intended to spend more on the NHS. Wrong. Boris and Gove were agreed and determined to do exactly this. On the morning of 24 June they both came into HQ. In the tiny 'operations room' amid beer cans, champagne bottles, and general bedlam I said to Boris – on day one of being PM you should immediately announce the extra £100 million per week for the NHS [the specific pledge we'd made] is starting today and more will be coming – you should start off by being unusual, a political who actually delivers what they promise. 'Absolutely. ABSOLUTELY. We MUST do this, no question, we'll park our tanks EVERYWHERE' he said. Gove strongly agreed. If they had not blown up this would have happened. The opposite impression was created because many Tories who did not like us talking about the NHS reverted to type within seconds of victory and immediately distanced themselves from it and the winning campaign. Unlike Gove and Boris they did not learn from the campaign, they did not listen to the public. Until people trust that the NHS is a financial priority for Tories, they will have no moral authority to discuss management issues. This obvious fact is psychologically hard to absorb because of the strength of gang feelings in politics.
(There are already myths about some of these events. The press conference of 24 June is now written up as the two of them 'terrified of what they had done' but this is completely wrong. They were subdued partly because they were genuinely sad about Cameron and partly because they did not want to be seen as dancing on his grave. Some of the media created the psychologically compelling story that they were regretful / frightened about victory but this was not at all their mood in HQ on the morning of 24 June. Boris came in punching the air like Maradona after a great goal, hugging staff and clearly euphoric. It is completely wrong to portray him as regretful.)
3. 'Vote Leave to take back control of immigration policy. If we stay there will be more new countries like Turkey joining and you won't get a vote. Cameron says he wants to "pave the road" from Turkey to here. That's dangerous. If we leave we can have democratic control and a system like Australia's. It's safer to take back control.'
I was surprised at what a shock it was to IN when we hit them with Turkey. By the time this happened they were in an almost impossible position. I wanted them to announce a veto. It would not have been believed and would have had the opposite effect – people would have taken the danger of Turkey joining more seriously. If your life depended on winning for IN, the answer is clear: they should have said long before the campaign started as part of the renegotiation process that they would veto any accession.
4. 'The euro is a nightmare, the EU is failing, unemployment is a disaster, their debts and pensions are a disaster, if we stay YOU will be paying the bills. It's safer to take back control and have a new relationship based on free trade and friendly cooperation instead of the European Court being in charge of everything…' (This is not an official text, just a summary of the notion off the top of my head.)
5. Anti-Establishment. E.g. We aligned our campaign with those who were furious with executive pay / corporate looting (about 99% of the country). We aligned ourselves with the public who had been let down by the system.
Mandelson regarded this as 'sheer nerve, sheer chutzpah'. It was obvious. The hard thing was sticking to it despite the sensibilities of many of our own supporters. One of the most effective TV performances of the campaign was the day Boris hit the theme of corporate looting in a market square. No10 were rightly panicked and in response pushed out Heseltine a few hours later to make a very personal attack on Boris. This made sense tactically but was a strategic error. All such personal attacks helped persuade Boris to up the ante. This was vital with a month to go when the immigration figures came out. Rudd and others argue that Cameron should have attacked Boris and others more. Wrong. They should have played it Zen publicly and had a much better black ops team.
Cameron/Osborne mistakes
I'll go into this separately but just to give a few examples…
1. Cameron never had to offer the referendum in the first place. His sudden U-turn was a classic example of how his Downing Street operation lurched without serious thought in response to media pressure, not because of junior people but because of Cameron himself and his terrible choice of two main advisers (Llewellyn and Oliver). This happened many times and I wrote about all the damage it caused on other issues after I left government (HERE). This was the biggest example. It was a product of a deeper error – a combination of his failure of party management (misleading them about the best way to handle the party) and failure to understand how swing voters really think and therefore the dangers of a vote (see below).
2. If Cameron/Osborne had had a top notch person like David Plouffe running their campaign and they did as they were told then they would have won (>95% confidence), all else being equal. They were warned many times by their closest friends about Oliver and Llewellyn, including by Gove, but would not listen.
3. Their renegotiation was flawed from the start and badly undermined their central message. They compounded their errors in 2015 by accepting the pathetic deal in 2016. If they had walked away in February then Vote Leave would quickly have imploded and the flying monkeys would have taken over the campaign.
4. They made themselves too prominent in the campaign and were too crude. Lacking a feel for psychology they gradually undermined their own message. Oliver thought Obama's 'back of the queue' was brilliant. It was counterproductive. They thought ratcheting up the warnings to DEFCON 1 was effective. It was counterproductive.
5. They doubled down on 'tens of thousands'. They thought they would lose credibility if they didn't. The opposite was true. They should have dropped this in 2015 – for example, in an exclusive to the Independent on a Saturday in early August 2015 – and gone into the campaign without it. Every time they defended it they were helping us.
6. They suckered themselves into over-prioritising their coalition versus message. Blair's campaign against us in the North East did the same. When you do this you lose focus and clarity which is usually fatal. The error was perhaps most visible the day Cameron unveiled an absurd poster that effectively listed all the 'serious people' on their side and – creative genius! – a blank page for us. A total waste of valuable time. The fact of being the Government meant the broadcasters let them lead the news almost all the time but they often wasted it like this. (I would bet that that ad was never put in focus groups or if it was the results were ignored.)
7. One of my basic criticisms of Cameron/Osborne from the start was the way they steered by pundit. During the 2015 election Crosby partly corrected this and they partly learned the lesson. But left to their own devices in the referendum when under pressure they defaulted to their instincts at a crucial moment. The reaction to the dreadful murder was an example of how the media and SW1 can live effectively in a parallel universe. Somehow they convinced themselves that this event might undo over a decade of growing hostility for those in power. They therefore tried to push the theme that actually MPs are great, 'they are in it for good reasons' and so on. The media led themselves into a dead end and No10, defaulting to their instincts of steering by pundit, followed. As soon as I saw Osborne and Matt Hancock wasting their time tweeting broken multicoloured hearts and encouraging #weloveourMP, I knew they had screwed their own OODA loop. We knew from focus groups (conducted by the brilliant Henry de Zoete who also played a crucial role in coordinating the digital and data science teams) that opinion outside London was extremely different to that of MPs and those in charge of most news. We went straight back to what we knew were the winning messages leaving Hancock and co to tweet broken hearts.
BUT BUT… Roland Rudd and others have attacked them for their basic strategy of focus on the economy and argue there should have been 'a positive campaign for the EU'. WRONG. Cameron and Osborne were right about this big call. There was not enough time or money to change basic attitudes. As the campaign developed and there were signs of pressure from Rudd and others I crossed my fingers and hoped they would shift strategy. No10 were right to ignore him.
I suspect that in general big mistakes cause defeat much more often than excellent moves cause victory. There are some theoretical reasons to suspect this is true from recent statistical analysis of human and computer decisions in chess. Two results are particularly interesting. 1) The very best computers seem to make moves that preserve the widest possible choices in the future, just as the most effective person in politics for whom we have good sources, Bismarck, operated always on the principle of 'keep two irons in the fire'. (We tried to mimic this by adopting a message that we thought had the highest probability of winning in the largest number of plausible branching futures, hence £350m/NHS.) 2) Even great humans are distinguishable from great computers by their propensity to make clear tactical errors occasionally amid the fog of war. This is significant enough that it wipes out the advantage of going first – i.e. it being 'your move' is seen as a plus but in fact it is a minus for humans because of the probability of a significant error, while for computers this effect is absent. (See Human and Computer Preferences at Chess, 2014. It would be very interesting to know if these results are supported by the recent success of Deep Mind with computer GO.)
Summary of the false dichotomy
False: 'Leave won because of the campaign.' E.g. Without 15 years of out of control immigration, our message of 'take back control' would not have had enough traction. Campaigns can ride big waves but they almost never make them.
False: 'Leave won because of a big event [immigration, 2008 crash etc], the campaign was irrelevant.' E.g. If the campaign had not deployed £350 million and the NHS (which almost nobody on our side liked), we would not have neutralised/overwhelmed Project Fear.
True: 'Leave won because 1) three big forces created conditions in which the contest was competitive, AND 2) Vote Leave exploited the situation imperfectly but effectively, AND 3) Cameron/Osborne made big mistakes. If just one of these had been different, it is very likely IN would have won.'
Overall, the now-mocked conventional wisdom that 'the status quo almost always wins in referendums like this' obviously has a lot of truth to it and it only proved false this time because of a combination of events that was improbable.
A 'miracle' to get 48%? Beaten by lies? Corbyn the AWOL saviour?
Since losing many inside the IN campaign now talk dejectedly as if they could never have won and tell rationalising fairy tales. They are wrong. They almost did win. Some have latched onto the idea that they were overwhelmed by an epic, global force of 'right-wing populism'. Mandelson defends himself by saying 48% looks 'like a miracle' given the populist tide. Most have latched onto the idea that their 'complex truth' was overwhelmed by 'simple lies' and they are happy with their comforting 'post-truth' sobriquet – a delusion that leaves them very vulnerable to being shocked again. Many have even argued that they lost because they could not persuade Corbyn to make more speeches.
These stories are psychologically preferable to the idea that their own errors caused defeat (just as it is for some of those in Hilary's campaign) but should not be taken seriously.
The least plausible claim is that Corbyn sabotaged what was otherwise a winning campaign. This is argued mainly by the same people (including Mandelson) who in a party context also argue that Corbyn is a joke who nobody takes seriously. The idea that more speeches by Corbyn would have persuaded vital swing voters has no good evidence. These people wanted to 'take back control'. Corbyn's message was – there should be not just more immigration but no limits on it. There are not many branching histories in which this is a winner.
This 'epic global force' of 'populism' was thought by the same people before 23 June to be puny in comparison with the force of the combined Establishment hammering a message of economic fear in support of the status quo. Having underestimated certain trends in public opinion the same people are now exaggerating them (see below).
This is connected to 'complexity'. Month after month they argued (including to us in private discussions) that they would win largely because they had the advantage of the status quo – an advantage proved in votes around the world over many years. They were right. That was a big advantage. It is much simpler to argue for the status quo than for a very complex change – that is exactly why most 'change' referendums lose, just as they briefed the media. Now they say 'The EU is very complex, it requires a lot of information to explain it' (Craig Oliver). Their claim that actually they had the 'complex' argument to make against our 'simple lies' is laughable for exactly the reasons they gave themselves before they came unstuck.
Connected to this idea is that the great rationalists Cameron and Osborne – they of Project Fear and their comic 'emergency budget' and in 2015 the pictures of Salmond picking pockets designed successfully to persuade the English that the Scots would steal their money – were undone by a great surge of 'emotion'. Osborne is taking this delusion so far he is writing a book titled ludicrously 'The age of unreason'. When you lose and you blame it on millions of people being overtaken by 'unreason' – after previously winning by exploiting nationalist hostility – it's a sure sign that you are the one not reasoning straight and able to face your errors. For the likes of Osborne it is 'irrational' to reject the views of people like him. For most of us, people like Osborne are not experts to be trusted – they are charlatans not to be taken seriously.
Many of those who blame defeat on 'lies', including Cameron, Osborne, and Clegg themselves told flat-out lies. One example will do. Cameron and Osborne claimed repeatedly on TV, almost always unchallenged, that their new deal meant 'after six months if you haven't got a job you have to leave'. This is not an argument over the fairness of using a gross/net figure, like '£350 million', or even a properly bogus figure like the Treasury's £4,000 per household figure. It is a different category of claim – a flat out 100% lie. (For more details see HERE.) How much time did Today, Newsnight, and the Guardian spend explaining to people that the PM and Chancellor were lying through their teeth? Approximately none. Why? Because very few of those complaining about lies really are cross about 'lies' – they are cross they lost and they are not so interested in discussing a lie that undermines the pro-EU campaign's attempt to neutralise fear of immigration.
Further, many of the same people spent the entire campaign saying 'Vote Leave has admitted a Leave vote means leaving the Single Market, this is what will happen make no mistake…' and now say 'the Single Market was not an issue, Vote Leave never had a policy on it and there is no mandate for leaving it'. Cameron, Osborne, Mandelson, Campbell and Clegg spent much of the last 20 years lying through their teeth to further their own interests and prestige. Now they whine about 'lies'. They deserved worse than they got – and reasonable Remain-ers deserved better leadership.
Fools and knaves
Many of those who worked on the IN side are now wrongly attacked as fools by pundits who would have praised them as geniuses had they won, while many on the OUT campaign are wrongly praised.
Example… 'If Remain wins Cameron ought to be hailed as the genius strategist of western democratic politics' (Rentoul). Pundits who wrongly hailed Cameron as a genius after the 2015 election now wrongly describe him as a bumbling oaf. He was neither – he was the best of a bad bunch picked pseudo-randomly in a broken system and out of his depth. 600,000 votes either way does not make one set of people geniuses and another set of people morons. Geniuses in politics are rarer than in maths and physics and nobody involved in the referendum on either side is remotely close to one. Some of those who worked on the IN side were much more able than many on the winning side. It does not make sense to label people on the IN side idiots because of errors made by Cameron, Osborne, Llewellyn, and Oliver.
Example: many have said to me 'you were so clever to hold back on immigration until the start of purdah'. Wrong. It is true that we did not do much on immigration before the 10 week official campaign. That is because, as I wrote in 2014, we did not need to. It was far more important to plant other seeds and recruit support that would have been put off if we had focused early on immigration. Immigration was a baseball bat that just needed picking up at the right time and in the right way. The right time was before purdah and we set in motion during January-April a series of things like the free referendum address with the right message but we could not persuade many prominent people to do what was needed until after 26 May. The right way was via the NHS (unifying) – not 'we want our country back' of Farage (divisive). The timing was not 'a brilliant move' by me, it was a combination of good luck and seizing a tactical chance to persuade people of something I'd failed to do for weeks, but such things get rewritten as such if you win.
It is also foolish to see the conflict in terms of who is 'nicer' and 'nastier'. I don't think the people on our side are nicer. There are lovely and loathsome people, liars and charlatans on both sides.
Many OUT-ers talk as if we were destined to win. Wrong. The IRA used to say 'you have to get lucky every time but we only have to get lucky once'. For Leave to win a string of events had to happen many of which were independently improbable or 50-50 and therefore the combination was very improbable. The result was certainly not an inevitable outcome of 'the great British public simply voting for democracy' as some romantics delude themselves.
Oblonsky and the frogs before the thunderstorm: fashion, delusions of the educated, and the Single Market
'I feel that, in some ways, this was a conflict between good forces in society and bad forces. I feel that the bad forces on 23 June won a very significant victory.' Matthew Parris.
Matt Ridley: Matthew, you're not saying that 17 million people are, deep down, racists?
Matthew Parris: Yes. (Spectator, December 2016)
Why is almost all political analysis and discussion so depressing and fruitless? I think much has to do with the delusions of better educated people. It is easier to spread memes in SW1, N1, and among Guardian readers than in Easington Colliery.
Generally the better educated are more prone to irrational political opinions and political hysteria than the worse educated far from power. Why? In the field of political opinion they are more driven by fashion, a gang mentality, and the desire to pose about moral and political questions all of which exacerbate cognitive biases, encourage groupthink, and reduce accuracy. Those on average incomes are less likely to express political views to send signals; political views are much less important for signalling to one's immediate in-group when you are on 20k a year. The former tend to see such questions in more general and abstract terms, and are more insulated from immediate worries about money. The latter tend to see such questions in more concrete and specific terms and ask 'how does this affect me?'. The former live amid the emotional waves that ripple around powerful and tightly linked self-reinforcing networks. These waves rarely permeate the barrier around insiders and touch others.
These factors are deepened by the fact that almost all of those whose job it is to explain politics and campaigns have never been responsible for a complex organisation in general or a campaign in particular, so they are unsuited to understand how politics ripples out from decisions at the centre through dysfunctional bureaucracies to the ground. They almost always exaggerate the extent to which important decisions have been considered carefully by people who know what they are talking about. (The worse educated are actually often helped by their lack of education towards the truth.) They constantly discuss complex systems as though errors can be eradicated instead of asking how quickly errors are adapted to and learned from. This perspective biases them in favour of existing centralised systems that fail continually and against innovations with decentralised systems. They understand little about the challenges faced by small businesses and the lower middle classes.
The more closely involved people are in the media and politics the more they are driven by fashion and the feeling, rarely acknowledged and almost always rationalised, that 'this is my gang'. Look at all those in SW1 who tweet attacks on Dacre to each other then retweet the praise from their friends, then look at those who attack them. Look at Robert Peston tweeting pictures of the London Eye and Habermas quotes on election night and his opponents ranting about 'elites'. Both sides are just like football team fans defending their in-group and attacking their out-group enemies. The more they think of themselves as original the more likely they are to be conformist – and conformist within very narrow parameters. We all fool ourselves but the more educated are particularly overconfident that they are not fooling themselves. They back their gang then fool themselves that they have reached their views by sensible, intelligent, reasoning.
This makes them particularly vulnerable to 'influence operations'. It also makes them vulnerable to repeated errors about what the sort of people who ignore politics other than for a few weeks before voting time are thinking. It creates something of a paradox: it is almost impossible to get a good feel of public opinion, or of 'the winning strategy', by listening to those whose job it is to speculate about it. However often this happens, the lesson is never learned. It is very hard to see how it could change as it is so entangled with our evolved nature.
There is a wonderful passage in Anna Karenina that sums this up, much better than any 'political scientist' has done:
'Oblonsky never chose his tendencies and opinions any more than he chose the style of his hat or coat. He always wore those which happened to be in fashion. Moving in a certain circle where a desire for some form of mental activity was part of maturity, he was obliged to hold views in the same way he was obliged to wear a hat. If he had a reason for preferring Liberalism to the Conservatism of many in his set, it was not that he considered the liberal outlook more rational but because it corresponded better with his mode of life… The Liberal Party said that marriage was an obsolete tradition which ought to be reformed, and indeed family life gave Oblonsky very little pleasure, forcing him to tell lies and dissemble, which was quite contrary to his nature. The Liberal Party said, or rather assumed, that religion was only a curb on the illiterate, and indeed Oblonsky could not stand through even the shortest church service without aching feet, or understand the point of all that dreadful high-flown talk about the other world when life in this world was really rather pleasant… Liberalism had become a habit with Oblonsky and he enjoyed his newspaper, as he did his after-dinner cigar, for the slight haze it produced in his brain.'
Towards the end of the novel, there is a discussion about the then big issue of Turkish atrocities and the rise of pan-Slavism. The old prince replies to the intellectuals who are talking rubbish about 'the national feeling' that they think is 'sweeping the country':
'Yes, all the papers say the same thing. That's true. So much the same that they are just like frogs before a storm! You can't hear anything for their croaking.'
Many will reply, 'Oblonsky is a dilettante, not a serious character, you can't compare him with people like Robert Peston'. Oblonsky isn't a dummy, he's brighter than many of the posh duffers in his club. And also consider Anna's husband, Karenin – a terrifying reminder that the 'serious characters' in politics are really no better than Oblonsky regarding fashion. In politics, just about all of us are some combination of Oblonsky and Karenin. If you think you aren't, you're probably fooling yourself. If you're on TV a lot, you're almost definitely fooling yourself.
There are many examples of how real Oblonskys, who control practically all important cultural institutions, think. They believed things about Stalin's regime so outlandish that it is hard to appreciate now. They were more in favour of Britain joining the euro, not because they understood 'the complexities' better but because they were suckered into thinking about it as a moral test – are you on the side of the 'baddies' or the goodies'? As the BBC Europe editor said to me back then, in similar terms to Matthew Parris about the 2016 referendum, 'the thing is Dominic, we like foreigners and cappuccinos and we hate racists'. Polls show that better educated people are less likely to have accurate views about the science of evolution and genetics (their desire to send moral signals suckers them into believing fairy tales).
The conformity of the educated is in some ways a good thing – most obviously, a basic consensus about things like not killing one's domestic opponents that is extremely unusual historically. But it has many bad effects too. There is a collective lack of imagination which makes the system very susceptible to disastrous shocks. They share a narrow set of ideas about how the world works which mistakes their own view as the only possible sensible approach. They are aways writing about how 'shocking' things are to them – things that never were as low probability events as they imagine. They can't imagine something like Stalin deliberately creating a famine or deliberately murdering millions. They tell themselves that Hitler will be 'more sensible in power' and 'engagement' is the right path. Western liberals (like Clinton and many pro-euro campaigners) and conservatives (like Bush) talked of relations with Putin as if he is a normal western politician rather than an ex-KGB mafia overlord with views very far from western liberals. They tell each other 'I can't imagine President Trump, it just can't happen'. Many conservatives are now telling themselves that they should not take Trump too literally but that too is a failure of imagination – his character is clear to those unblinded by gang mentality and he will govern in character.
The referendum was a great example of this. Large numbers of people better educated than average – the sort of people who work as producers at the BBC – talked about their vote like this:
'Farage is racist, he hates gay people and made that comment about foreigners with HIV, he wants to turn the clock back and pull the drawbridge up, I'm not like that, my friends aren't like that, I am on the other side to people like that, I am tolerant and modern, I will vote IN.'
All over the country sentiments almost identical to this were expressed in large numbers. The idea that millions of graduates voted because they 'studied the issues' is laughable to anybody who spent time measuring opinion honestly. Almost none of these people know more about what a Customs Union is than a bricky in Darlington. They did not vote on the basis of thinking hard about the dynamics of EMU or about how Brussels will cope with issues like gene drives. Millions thought – there's two gangs and I know which one I'm in. Another subset of the better educated feared the short-term economic disruption of a Leave vote would cost them money. They also did not vote on the basis of deep consideration of the issues.
The modern day Oblonsky reads an op-ed about how 'the CBI warns of the dangers of leaving the Single Market' and 'the dangers of racist extremists' and, having no idea of what 'the Single Market' is, jabbers away at their dinner party about how concerned they are about leaving 'the Single Market', and a warm haze of knowing one is on the 'good' side of the argument envelops the brain.
When it comes to the central issues of the nature of the EU's trading relationships and what a UK-EU relationship might look like outside the EU, we are dealing with a particularly strong example of this phenomenon. Not only do the Oblonskys not know what they are talking about, neither do almost any of the supposed experts and specialists.
Lots of people said to me 'when are you going to set out the details of the UK-EU trade relationship if you win?' What would have been the point of that?! Approximately nobody knows anything about the important details of how the EU works including the MPs who have spent years talking about it and the journalists who cover it – indeed, often those who talk about it most are the most ignorant (and most overconfident). This is still true six months after the vote – imagine how much more true it was in the six months before the vote.
I am not aware of a single MP or political journalist who understands the Single Market – its history, its nature, its dynamics, its legal system, the complex interactions between law, economics, business, history and so on. Cameron, Osborne and Clegg certainly don't. Neither does Bill Cash. Neither does any head of the CBI. Neither do Jon Snow, Robert Peston, Evan Davis or John Humphreys so they do a rubbish job of exposing politicians' ignorance.
The number of people who do is tiny. In our campaign there were two – Oliver Lewis and Richard Howell – who understood a large fraction of it and the common misconceptions. They constantly had to explain to MPs, MEPs, and journalists why their ideas were misunderstandings. Maybe there is a business/economics journalist somewhere who really understands it. There are certainly some exceptional lawyers who understand narrow aspects extremely well, though few of these also understand the political and business dimensions. I have spoken to many very successful business people and never met one on either side who understands the Single Market in depth. In the entire campaign I am not aware of a single programme on TV that even tried to delve into these issues seriously (Newsnight was particularly bad, combining smugness and vapidity such as dropping Evan Davis by helicopter on an offshore platform to babble about 'sovereignty' trying to make the Leave side look like a bunch of weirdo cranks). British elites handed over power to the Monnet-Delors project with barely one-in-a-thousand understanding in detail why, what it involved, and its likely evolution (and that one-in-a-thousand almost all concluded that the public could not be trusted to know the truth – I'll explore another time the ideas of this tiny group).
Further, it was clear that Cameron/Osborne intended to run a campaign based on hysterical warnings and bogus arguments/figures while ignoring the big questions about how the EU works and its trajectory. No10 tried to turn the whole complex issue into a question about whether the economy would grow a little bit slower over the next few years – a trivial issue relative to the significance of the overall question. They are not a duo who have ever engaged the public on a serious matter in a serious way. Their brains don't work like that. They formed early habits of looking at everything through a very narrow prism of SW1 conventional political wisdom. Given this, the way the media works, how outnumbered we were among the influential broadcast media, and the way in which the media (inevitably to some extent) takes its lead from No10, why would I have tried to run a campaign based on educating normal people to a far higher level than the professionals and 'experts' who were fighting and covering the campaign? It would have been impossible to get even two sensible MPs to explain the same complex argument about such things on TV without cocking it up – it was hard enough to get people to say 'let's spend our money on our priorities' without days of arguing. (If the vote had happened in 2017 and we'd had all that time to build sensibly more could have been done.)
We did try to get the media to focus on deeper questions of how the EU is run, its problems, its evolution and so on. We knew from the research that the more coverage of the EU, its powers, its record, its plans and so on the better for us. We had little to fear from serious policy discussion and much to gain. But we largely failed. (A big speech from Gove was turned by the Financial Times – yes, the FT that bemoaned the 'low quality debate' – into a story about whether he had 'gaffed' by mentioning Albania, though in plastering 'Albania' all over the place the FT accidentally helped us.) No10 calls up the BBC and says, 'we've got a business letter tomorrow with dozens of household companies warning of Armageddon.' If we published something worthy on the Eurozone's debt and demographic nightmare, the structural problems of the Eurozone and implications of the Five Presidents' Report, how far did this get? 'Sounds boring. Who's fronting it? Got any new names? Any chance of Boris putting the boot into Dave and George?', is the first question from the BBC TV producer who has no interest in 'the arguments'.
It was not in our power to change basics of how the media works. We therefore twisted them to our advantage to hack the system.
Hack the medium, hack the message: 'the alternative government' launches Sunday 29 May
The media is obsessed with process and the snakes and ladders of careers. Many hacks said to me words to the effect: 'I don't care about the issues, I care about whether Cameron will still be PM at the end of the year.' We could not match No10 in the golden currency of 'names'. But we could give the media an even more valuable currency – a leadership story. When Boris and Gove decided to go for it after 29 May immigration numbers, we launched the story of 'the alternative government'.
The media were understandably obsessed with this story so we served it up to them in such a way that they also had to cover our message. For 10 days, we dominated the news with a set of stories on the Australian points system, VAT on fuel, Turkey, the NHS and so on all based on 'it's safer to take back control'. Broadcasters lapped it up – even ITV News which barely pretended to be impartial was useful.
What did the public hear? They heard that prominent Conservatives, particularly Boris and Gove, did not trust Cameron's promises or warnings and that there was an alternative path – we could 'take back control', have 'an Australian style immigration system', and 'spend our money on our priorities like the NHS'. In an environment in which the central arguments concerning trade and the economy were incomprehensible to the 'experts' themselves and the history and dynamics of the EU either unknown to or suppressed by broadcasters, people chose between two simple stories. Vote Leave's was more psychologically compelling, given the three powerful forces at work and No10's errors.
(NB. Whoever leaked the Hilary email story was probably doing something similar. This played into the media obsession with scandal and process such that they spent a ridiculous amount of time on it despite probably 80% of them wanting Hilary to win. It shows how powerfully the media is in the grip of dynamics they rarely reflect on themselves. Putin's communication maestro, Surkov, uses these sorts of tricks all the time. Cf. Peter Pomerantsev's great book, a must read for any MP before they pontificate on Putin's mafia government.)
The political media and how to improve it
High prestige pundits and editors yield great power over the stories told (and have far more power over politicians like Cameron, unfortunately, than they realise) but the field is not based on real expertise. Fields dominated by real expertise are distinguished by two features: 1) there is enough informational structure in the environment such that reliable predictions are possible despite complexity and 2) there is effective feedback so learning is possible.
Neither condition applies generally to politics or the political media. In the most rigorous studies done, it has been shown that in general political experts are little better than the proverbial dart throwing chimp and that those most confident in their big picture views and are most often on TV – people like Robert Peston, Jon Snow, and Evan Davis – are the least accurate political 'experts' (cf. HERE).
We know that cognitive diversity is vital for political accuracy yet almost all political institutions and the media – including the dominant people at Newsnight, the Economist, the FT, and Parliament – are actually remarkably homogenous, as discussed above, and they herd around very similar ideas about how the world works. Scientists and entrepreneurs in particular are almost totally excluded from political influence.
There is no structure to hold them to account either internally or externally so, like anyone when not forced to be rigorous, they fool themselves. It is normal to write month after month that the IN campaign cannot lose because of XYZ then just as confidently and authoritatively explain why IN lost without any intermediate step of identifying and explaining errors.
Despite the rise of social media most people get most of their news from TV. TV coverage of politics rarely illuminates much because there is no clear way to decide who is right about anything. The format makes it almost impossible for any useful discussion to happen. Interviewers, politicians, and pundits talk past each other with no clarity about assumptions. Questions are vague, often meaningless, posed by interviewers who rarely have more than a thin bluffer's understanding of any policy issue and the same is usually true of those answering; the more famous the interviewer, the less likely it is they know anything about, say, education policy and like David Cameron they are bluffing. (As soon as a story is deemed 'political' it is taken out of the hands of specialists (who are very rarely actually specialists anyway) and given to 'political' hacks who have no idea of the policy.) Most of those professionally involved are much more interested in the 'horse race' political dimension than the policy. They obsess on process and scandal but most people have no interest in the process or 'scandals' because they assume 'they're all dodgy in some way'. Nobody tries to make predictions that can be checked and the shows don't take what is said seriously enough to catalogue it. Simplistic stories compete so political analysis is dominated by endless false dichotomies.
Those making the shows do not understand how people learn so the dead format recycles grim clichés like Evan Davis saying '… economy down the plug hole', while filming an actual plug hole, or Nick Robinson saying '… will the economy take off' standing in front of a plane actually taking off (both of these have happened). Every night the News contains reports that are a mix of incomprehensible, facile, and boring to millions while also usually at best simplistic and often just wrong when it comes to policy / issues. The possibilities of the medium are largely ignored.
Insiders think of the masses as being irrational in paying so little attention to political debate. I think they are rational. If you want to understand politics you should read serious things and invest time and effort in researching public opinion. You should particularly make an effort to invert your point of view and consider opinions very different to your own. Time spent watching/listening to shows like Newsnight and Today is not just wasted – it is actively distorting reality and making you less informed. I often meet people who are cleverer than those in politics and successful but they have deluded views about politics because they pay too much attention to political analysis. Overall, unless you are professionally involved in politics you will be better off if you stop >95% active reading of political analysis. You will miss occasional worthwhile things but the effort of sorting them is not worth it. If something is genuinely very good / unusual and you have avoided isolating yourself in an echo chamber that insulates you from opinions very different to your own then someone reliable will send it to you. Even if you are professionally involved in politics I would do roughly the same. Extreme focus on important things you can control will repay far far more than time spent reading speculation about things you can't control.
I read very little punditry during the campaign – just enough to preserve a sense of the gaps between the 'croaking frogs' and the real world. If I'd had less infighting to deal with I'd have read even less as I could have been less concerned about tracking certain things. In my entire time in the DfE (three years) I never listened to Today once (I listened to a handful of interviews on the web). I focused on managing priorities and saying 'No, stop, that's a waste of time' every day.
This situation is particularly ironic because the media industry is in a panic about the internet, falling ad revenues and profits, the collapse of print journalism and so on.
There is a better way.
Example 1. Shows should require precise quantitative predictions about well-formed questions as Superforecasters do. Newspapers should do the same when interviewing people. The next step is using this process to push people towards admitting conditional errors like 'if I am proved wrong about X by date Y then I will admit I was wrong to claim Z'. If political shows pushed their guests to do this and kept track of the predictions it could have a big positive effect. (Next time they come on you can flash up their record on a screen so the public can see how often they are right.) It is vital to change incentives so people are encouraged to admit errors and learn instead of fooling themselves constantly. For those who refuse it would be easy to develop a protocol that categorises their vague comments and puts numbers on them. This will push them to 'correct the record'.
Example 2. Rip up the format for political shows and base broadcasts on a) an empirical assessment of what people actually know and b) the science of how people really learn and how best to communicate. Instead of the tedious low-information interviews, imagine what could be done if one had a mix of artists, scientists, and policy specialists trying really hard to use the possibilities of film to explain things, then used cutting edge data science to test how effective they were as part of a learning cycle driving higher quality. A news broadcast now contains much less information content and much higher noise than reading. The only way to improve this is experimenting with formats in a scientific way. Doing this would force those making the news to think more about policy and the audience would be much more engaged. People are interested in policy and 'how X will affect me, my family, and my community'. It would also obviously require a lot of changes in the media but this is coming anyway because existing business models are blowing up.
Example 3. Pay for this partly by firing most of your political commentators like Dan Hodges. Broadcasters, fire 90% of your political correspondents. They are a waste of money. Hire a much smaller number of much better people with radically different skills and backgrounds and a different focus. (By doing #1 you will soon see who is more/less accurate so you'll have a good benchmark.)
Together these changes could improve the quality by a factor of x10 or more. The same principle of focusing on precise quantitative predictions about well-formed questions could also be used to improve policy making and management of bureaucracies by developing clusters of well-formed questions that 'surround' a vaguer big question that is not so susceptible to measurement.
For example, break down 'will Britain leaving the EU be a success?' into dozens of simpler more precise questions that can be quantified and which together give a useful part of an overall answer. This process could be put on a prediction platform for little money and dramatically improve the quality of decisions. The Superforecasters new consultancy could do this pretty simply with little help and not much money.
As usual in systems that are failing, the youngest people understand the problems and possibilities best while the most senior / prestigious figures can't think of anything to do other than get overpaid for what they've always done. If you run a big media company, you should replace the expensive old schoolers like Jon Snow with younger, cheaper, and brighter new schoolers with an extreme focus on the public, not SW1.
An example of a simple, powerful media story that is wrong and contributed to forecasting errors on Brexit – 'the centre ground'
One of the most misleading stories in politics is the story of 'the centre ground'. In this story people's views are distributed on an X-axis with 'extreme left' at one end, 'extreme right' at the other end, and 'the centre ground' in the middle. People in 'the centre' are 'moderate'. 'Extremists' are always 'lurching' while 'sensible moderates' are urged to 'occupy the centre'.
This story is one of the dominant features of political discussion and the basis for endless interviews, columns, and attempts at political 'strategy'. The story is deeply flawed and where it is not trivially true it is deeply misleading.
Swing voters who decide elections – both those who swing between Conservative/Labour and those who swing between IN/OUT – do not think like this. They support much tougher policies on violent crime than most Tory MPs AND much higher taxes on the rich than Blair, Brown, and Miliband. They support much tougher anti-terrorism laws than most Tory MPs AND they support much tougher action on white collar criminals and executive pay than Blair, Brown, and Miliband.
One of the key delusions that 'the centre ground' caused in SW1 concerned immigration. Most people convinced themselves that 'swing voters' must have a 'moderate' and 'centre ground' view between Farage and Corbyn. Wrong. About 80% of the country including almost all swing voters agreed with UKIP that immigration was out of control and something like an Australian points system was a good idea. This was true across party lines.
This was brought home to me very starkly one day. I was conducting focus groups of Conservative voters. I talked with them about immigration for 20 minutes (all focus groups now start with immigration and tend to revert to it within two minutes unless you stop them). We then moved onto the economy. After two minutes of listening I was puzzled and said – who did you vote for? Labour they all said. An admin error by the company meant that I had been talking to core Labour voters, not core Tory voters. On the subject of immigration, these working class / lower middle class people were practically indistinguishable from all the Tories and UKIP people I had been talking to.
The media tried to categorise Vote Leave as 'right wing' while Tory MPs and Farage's gang were screaming at me about our championing the NHS and our attacks on the indefensible pay of FTSE CEOs. SW1 did not understand our appeal but the crucial voters did because they do not think as the 'experts' think they think. We tried to speak to a majority in the country. Cameron and Osborne have never won even 40%. They approached it as they did previous battles but this greatly limited their appeal. Most UKIP and Tory voters (rather than MPs/insiders) agreed with us on the NHS and executive pay while also agreeing with us on the need to take back control of immigration policy from a system that has obviously failed. Our campaign was neither Left nor Right in the eyes of the crucial audience.
The media made a similar mistake with Trump. Trump did lots of things wrong and the post facto re-branding of his campaign as 'brilliant' is very silly. BUT he had a national message the core of which appealed to a big majority and which defied categorisation as Left/Right. Again the media do not realise this – they label it, like Vote Leave, as 'populist right' (abetted by some charlatan academics). But the reason why it is successful is exactly because it is not a simple right-wing message.
It doesn't occur to SW1 and the media that outside London their general outlook is seen as extreme. Have an immigration policy that guarantees free movement rights even for murderers, so we cannot deport them or keep them locked up after they are released? Extreme. Have open doors to the EU and don't build the infrastructure needed? Extreme. Take violent thugs who kick women down stairs on CCTV, there is no doubt about their identity, and either don't send them to jail or they're out in a few months? Extreme. Have a set of policies that stops you dealing with the likes of 'the guy with the hook' for over a decade while still giving benefits to his family? Extreme. Ignore warnings about the dangers of financial derivatives, including from the most successful investor in the history of the world, and just keep pocketing the taxes from the banks and spending your time on trivia rather than possible disasters? Extreme. Make us – living on average wages without all your lucky advantages – pay for your bailouts while you keep getting raises and bonuses? Extreme and stupid – and contemptible.
These views are held across educational lines, across party lines, and across class lines. Cameron, Blair, and Evan Davis agree about lots of these things and tell people constantly why they are wrong to think differently but to millions they are the extremists.
(This is not a post facto rationalisation. I wrote about the centre ground and the EU in 2014 HERE.)
Why I got involved and my role
Winning the referendum against Cameron was not the way I wanted things to happen. I thought the chances of winning a referendum against a PM on the other side, with all the possibilities for him to mobilise the system behind IN, were low. Many prominent Eurosceptics (not all) lobbied for it out of a combination of self-promotion and not knowing how to solve the real problem – what should the UK-EU relationship be? The referendum was very useful for many Out-ers: it provided a much simpler political focus than figuring out a complex positive agenda, removed the need for difficult thinking and action, and gave people a chance to pose on the side of 'democracy'. I thought it foolish to push for a referendum while simultaneously not building a serious movement to win it. (I had tried to start building such a movement in 2004 after the euro battle was clearly won but could not persuade crucial people so decided to drop the issue for a while.) Romantic long shots are rarely wise in politics particularly if there is a better path.
I thought it wiser and safer to wait for Cameron to go then try to capture the Tory leadership and change the UK-EU relationship from Downing Street with someone who actually wanted to solve the problems (Cameron's best friends would not claim that he wanted to spend his time trying to solve these deep problems, he wanted not to think about the EU and got into an existential battle he never wanted). If you are going to have a referendum, then have it when controlling the institutions and when you can set the agenda. A British PM could invite the EU to evolve such as to include a) those in the euro, Single Market and 'free movement', and b) those outside some or all of those three but with free trade and friendly cooperation between all. The chances are low that there would have been support for fundamental change but then a divorce could have happened after a serious clarifying debate which would have occurred ~2018-25, including the Eurozone countries figuring out what they would do. This would have been a much better way to proceed than the charade of Cameron's 'renegotiation'. Either Europe would have embraced a new and more open architecture (unlikely) or the Government would have won a Leave referendum with 60-70% and prompted a lot of clarifying thought across Europe.
I also thought it foolish of Cameron to cave into the pressure and promise a referendum in 2013. So did Gove and Osborne both of whom told Cameron not to do it. He mistakenly thought it would take the wind out of UKIP's sails and did not understand why it would actually boost UKIP and Farage. (This was not hard to foresee and I suspect part of the problem was that Cameron did not appreciate that him promising a referendum would be thought by most as just a typical pre-election lie.) The idea that there was an irresistible force for a referendum is pushed by Farage's and Cameron's supporters. They are both wrong. The country supported one but without any passion outside the small fraction who had long been passionate about it. Most Tory MPs did not want it. Most Tory donors thought the timing was wrong and wanted a focus on stopping Miliband who they feared. Those MPs who did want it could mostly have been bought off or distracted in other ways – a mix of some policy, gongs, bribes, and so on in the usual fashion. Putting a date on the vote was particularly mistaken – it would have been far better to leave it open-ended 'in the next Parliament'.
Once the election happened there was a sudden panic among OUT-ers. UKIP was an organisational disaster. There was no national campaign prepared. There were many tiny groups who often hated each other more than they wanted to win and were conditioned to expect failure and defeat. There was an abundance of people who thought that the campaign was quite simple – put me on TV, they thought, and the nation will appreciate my natural leadership. There was practically nothing of what was actually needed. Many quickly flipped into panic mode assuming the vote was unwinnable.
Having opposed the push for a referendum, I was faced with an uncomfortable choice in May 2015. Either keep out of politics, refuse to help, and then feel miserable about the tragicomic campaign, or re-engage with people I did not want to work with, feel miserable about the tragicomic campaign, and in almost every way make my life worse. In many ways irrationally, I chose the latter. My thinking was something like this: the chance of changing the whole political system (more profoundly than in a normal election) comes along very rarely, the chaos of the eurosceptics and the complacency of Cameron creates a very slim bridge to seize control and do it, a small chance of very high impact is worth the gamble. About a month or so later my wife was pregnant. If the timing had been slightly different I might well have stayed retired.
I thought that Leaving would improve the probability of 1) Britain contributing positively to the world and 2) minimising dangers. I thought it would:
minimise Britain's exposure to the problems caused by the EU;
improve the probability that others in Europe would change course before more big crises hit, e.g. by limiting free movement which is the biggest threat to continued free trade;
require and therefore hopefully spark big changes in the fundamental wiring of UK government including an extremely strong intelligent focus on making Britain the best place in the world for science and education;
improve the probability of building new institutions for international cooperation to minimise the probability of disasters.
The foundation problem with the EU was best summarised by the brilliant physicist David Deutsch, the man who extended Alan Turing's 1936 paper on computation into the realm of quantum mechanics. Deutsch said:
'The EU is incompatible with Britain's more advanced political culture. I'm voting Leave… [E]rror correction is the basic issue, and I can't foresee the EU improving much in this respect… [P]reserving the institutions of error correction is more important than any policy… Whether errors can be corrected without violence is not a "concern" but a condition for successfully addressing concerns.'
Healthy and effective systems like our immune system and the English common law allow constant and rapid error-correction. Unhealthy and ineffective systems like the EU and modern Whitehall departments block error-correction. They are extremely centralised and hierarchical therefore information processing is blocked and problems are not solved. In politics this often leads to disasters when more and more resources are devoted to reinforcing failure. NB. This most fundamental question played effectively no role in the debate.
This fundamental problem generates its other problems. It arises because of how Monnet and Delors created its institutions deliberately in opposition to the Anglo-American system they bitterly opposed. The Foreign Office romantic delusion of 'influence' was peddled by every PM since Thatcher. Every one left office having demonstrated how empty the hope is. True influence comes from demonstrating success – not sitting in meetings for forty years in an institution that is programmed on principles that guarantee worse error-correction than the evolved institutions of the Anglo-American system.
I will go into the problems of the EU another time. I will just make one important point here.
I thought very strongly that 1) a return to 1930s protectionism would be disastrous, 2) the fastest route to this is continuing with no democratic control over immigration or human rights policies for terrorists and other serious criminals, therefore 3) the best practical policy is to reduce (for a while) unskilled immigration and increase high skills immigration particularly those with very hard skills in maths, physics and computer science, 4) this requires getting out of the EU, 5) hopefully it will prod the rest of Europe to limit immigration and therefore limit the extremist forces that otherwise will try to rip down free trade.
One of our campaign's biggest failures was to get even SW1 to think seriously about this, never mind millions of voters. Instead the false idea spread and is still dominant that if you are on the side of free trade, think controlled immigration generally a positive force, and want more international cooperation rather than a return to competing nation states then you must support the EU. I think this error is caused by the moral signalling and gang mentality described above.
What was my role?
My role mainly involved:
a) trying to suppress/divert/overcome internal coalition warfare to a level where about ten crucial people were protected enough to do their jobs,
b) building the team,
c) management,
d) taking a small number of important decisions about policy, message, money, and the machine,
e) providing clear focus and priorities, including the vital job that nobody likes of saying 'no' to hundreds of people (thus making (a) harder), and
f) dealing with big problems.
The media tends to suggest my role was mainly talking to them. This is wrong. The same happened with my role in the DfE. In both projects my main role was management. Serious management means extreme focus and this requires saying No an awful lot. Contrary to the media story, I dislike confrontation and rows like most people but I am very strongly motivated by doing things in a certain way and am not motivated by people in SW1 liking me. This is often confused with having a personality that likes fighting with people. One of the basic reasons so much in politics is mismanaged is that so often those responsible are more interested in social relations than in results and unlike in other more successful fields the incentives are not structured to control this instinct.
Many have written that I got involved with this because of 'hate' or 'loathing' for Cameron. Wrong. I do not hate Cameron. I do not respect him, which is different. I thought that he was in politics for bad reasons – essentially because he was someone who wanted 'To Be', not someone who wanted 'To Do' (see the Colonel Boyd speech) and his priority was himself and a small gang, not the public. I also thought Cameron was mostly (not all) bad at the job, despite having some of the necessary temperamental characteristics, and was flattered by having Brown then Miliband as opponents. I didn't object to him blocking me from Government in 2010. He was entitled not to hire someone who did not take him seriously and ignored the orders of his Chief of Staff.
I spent a few years of my life (1999-2002) trying to stop Blair on the euro before anyone had heard of Cameron. In 2004 I co-founded the campaign that won the referendum on the North East Regional Assembly 80-20 as a training exercise for a possible future EU referendum. My motivation was the issue itself – not personal antipathy for Cameron or anybody else. I've never been a party person. I'm not Tory, libertarian, 'populist' or anything else. I follow projects I think are worthwhile.
Farage's motley crew claim that I did this campaign in order to lose it deliberately then get a job in No 10 with Cameron. It is pointless to discuss this theory though the fact that they understood so little about the political environment, and struggled to use Google, was an important fact.
I am not clever, I have a hopeless memory, and have almost no proper 'circle of competence'. I made lots of mistakes in the campaign. I have had success in building and managing teams. This success has not relied on a single original insight of any kind. It comes from applying what Charlie Munger calls unrecognised simplicities of effective action that one can see implemented by successful people/organisations.
Effective because they work reliably, simple enough that even I could implement them, and 'unrecognised' because they are hiding in plain sight but are rarely stolen and used. I found 10-15 highly motivated people who knew what they were doing and largely left them to get on with it while stopping people who did not know what they were doing interfering with them, we worked out a psychologically compelling simple story, and we applied some simple management principles that I will write about another time. It is hard to overstate the relative importance in campaigns of message over resources. Our success is an extreme example given the huge imbalance in forces on either side. In many ways Trump's victory has little resemblance to what we did but in this respect he is another example.
We also got lucky.
I will post a number of blogs of the referendum to try to answer some basic questions including:
What were the main political, operational, financial/budgetary, and data/digital lessons from the campaign?
What worked and did not work?
How confident can we be about these judgements?
There is a natural set of categories and I will post links to blogs below:
Some basic numbers that summarise important elements.
Strategy, message, polls.
Policy.
Data and digital.
On data science, digital marketing, canvassing software made available for download. (NB. There has been some confusion about this blog. The VICS system is a web-based canvassing tool, the first proper one that works in the UK – it was one component of our overall data science approach and should not be equated with it. It is not a data science tool – it provided data to the data science team.)
The ground campaign.
The media.
Internal politics and the infighting.
Dynamics that affect 'what next'.
The rules: how could they be improved to make future votes serve the public better?
Please leave comments and corrections below. I am happy to approve hostile comments if they have substance and will moderate comments to avoid putting sensible people off reading them.
Posted in Decisions, Information Markets, International Relations & War, Politics, The Sphinx of Psychology & Persuasion	| Tagged BBC, Bismarck, Blair, Boris, Cameron, Clegg, Colonel Boyd, Craig Dre, David Deutsch, ECHR, EU, EU referendum, experts, Foreign Office, Gisela Stuart, Gove, Heywood, Llewellyn, Media, polls, predictions, pundits, Single Market, Tetlock, UKIP, Vote Leave | 101 Comments
October 29, 2016 by dominiccummings
On the referendum #20: the campaign, physics and data science – Vote Leave's 'Voter Intention Collection System' (VICS) now available for all
'If you don't get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest. You're giving a huge advantage to everybody else. One of the advantages of a fellow like Buffett … is that he automatically thinks in terms of decision trees and the elementary math of permutations and combinations… It's not that hard to learn. What is hard is to get so you use it routinely almost everyday of your life. The Fermat/Pascal system is dramatically consonant with the way that the world works. And it's fundamental truth. So you simply have to have the technique…
'One of the things that influenced me greatly was studying physics… If I were running the world, people who are qualified to do physics would not be allowed to elect out of taking it. I think that even people who aren't [expecting to] go near physics and engineering learn a thinking system in physics that is not learned so well anywhere else… The tradition of always looking for the answer in the most fundamental way available – that is a great tradition.' Charlie Munger, Warren Buffet's partner.
During the ten week official campaign the implied probability from Betfair odds of IN winning ranged between 60-83% (rarely below 66%) and the probability of OUT winning ranged between 17-40% (rarely above 33%). One of the reasons why so few in London saw the result coming was that the use by campaigns of data is hard to track even if you know what to look for and few in politics or the media know what to look for yet. Almost all of Vote Leave's digital communication and data science was invisible even if you read every single news story or column ever produced in the campaign or any of the books so far published (written pre-Shipman's book).
Today we have made a software product available for download – Vote Leave's 'Voter Intention Collection System' (VICS) – click HERE. It was named after Victoria Woodcock, Operations Director, known as Vics, who was the most indispensable person in the campaign. If she'd gone under a bus, Remain would have won. When comparing many things in life the difference between average and best is say 30% but some people are 50 times more effective than others. She is one of them. She had 'meetings in her head' as people said of Steve Wozniak. If she had been Cameron's chief of staff instead of Llewellyn and Paul Stephenson had been director of communications instead of Oliver and he'd listened to them, then other things being equal Cameron would still be on the No10 sofa with a glass of red and a James Bond flick. They were the operational/management and communications foundation of the campaign. Over and over again, those two – along with others, often very junior – saved us from the consequences of my mistakes and ignorance.
Among the many brilliant things Vics did was manage the creation of VICS. When we started the campaign I had many meetings on the subject of canvassing software. Amazingly there was essentially no web-based canvassing software system for the UK that allowed live use and live monitoring. There have been many attempts by political parties and others to build such systems. All failed, expensively and often disastrously.
Unfortunately, early on (summer 2015) Richard Murphy was hired to manage the ground campaign. He wanted to use an old rubbish system that assumed the internet did not exist. This was one of the factors behind his departure and he decided to throw in his lot with Farage et al. He then inflicted this rubbish system on Grassroots Out which is one of the reasons why it was an organisational/management disaster and let down its volunteers. After Vote Leave won the official designation, many GO activists defected, against official instructions from Farage, and plugged into VICS. Once Murphy was replaced by Stephen Parkinson (now in No10) and Nick Varley, the ground campaign took off.
We created new software. This was a gamble but the whole campaign was a huge gamble and we had to take many calculated risks. One of our central ideas was that the campaign had to do things in the field of data that have never been done before. This included a) integrating data from social media, online advertising, websites, apps, canvassing, direct mail, polls, online fundraising, activist feedback, and some new things we tried such as a new way to do polling (about which I will write another time) and b) having experts in physics and machine learning do proper data science in the way only they can – i.e. far beyond the normal skills applied in political campaigns. We were the first campaign in the UK to put almost all our money into digital communication then have it partly controlled by people whose normal work was subjects like quantum information (combined with political input from Paul Stephenson and Henry de Zoete, and digital specialists AIQ). We could only do this properly if we had proper canvassing software. We built it partly in-house and partly using an external engineer who we sat in our office for months.
Many bigshot traditional advertising characters told us we were making a huge error. They were wrong. It is one of the reasons we won. We outperformed the IN campaign on data despite them starting with vast mounts of data while we started with almost zero, they had support from political parties while we did not, they had early access to the electoral roll while we did not, and they had the Crosby/Messina data and models from the 2015 election while we had to build everything from scratch without even the money to buy standard commercial databases (we found ways to scrape equivalents off the web saving hundreds of thousands of pounds).
If you want to make big improvements in communication, my advice is – hire physicists, not communications people from normal companies and never believe what advertising companies tell you about 'data' unless you can independently verify it. Physics, mathematics, and computer science are domains in which there are real experts, unlike macro-economic forecasting which satisfies neither of the necessary conditions – 1) enough structure in the information to enable good predictions, 2) conditions for good fast feedback and learning. Physicists and mathematicians regularly invade other fields but other fields do not invade theirs so we can see which fields are hardest for very talented people. It is no surprise that they can successfully invade politics and devise things that rout those who wrongly think they know what they are doing. Vote Leave paid very close attention to real experts. (The theoretical physicist Steve Hsu has a great blog HERE which often has stuff on this theme, e.g. HERE.)
More important than technology is the mindset – the hard discipline of obeying Richard Feynman's advice: 'The most important thing is not to fool yourself and you are the easiest person to fool.' They were a hard floor on 'fooling yourself' and I empowered them to challenge everybody including me. They saved me from many bad decisions even though they had zero experience in politics and they forced me to change how I made important decisions like what got what money. We either operated scientifically or knew we were not, which is itself very useful knowledge. (One of the things they did was review the entire literature to see what reliable studies have been done on 'what works' in politics and what numbers are reliable.) Charlie Munger is one half of the most successful investment partnership in world history. He advises people – hire physicists. It works and the real prize is not the technology but a culture of making decisions in a rational way and systematically avoiding normal ways of fooling yourself as much as possible. This is very far from normal politics.
(One of the many ways in which Whitehall and Downing Street should be revolutionised is to integrate physicist-dominated data science in decision-making. There are really vast improvements possible in Government that could save hundreds of billions and avoid many disasters. Leaving the EU also requires the destruction of the normal Whitehall/Downing Street system and the development of new methods. A dysfunctional broken system is hardly likely to achieve the most complex UK government project since beating Nazi Germany, and this realisation is spreading – a subject I will return to.)
In 2015 they said to me: 'If the polls average 50-50 at the end you will win because of differential turnout and even if the average is slightly behind you could easily win because all the pollsters live in London and hang out with people who will vote IN and can't imagine you winning so they might easily tweak their polls in a way they think is making them more accurate but is actually fooling themselves and everybody else.' This is what happened. Almost all the pollsters tweaked their polls and according to Curtice all the tweaks made them less accurate. Good physicists are trained to look for such errors. (I do not mean to imply that on 23 June I was sure we would win. I was not. Nor was I as pessimistic as most on our side. I will write about this later.)
VICS allows data to be input centrally (the electoral roll, which in the UK is a nightmare to gather from all the LAs) and then managed at a local level, whether that be at street level, constituency or wider areas. Security levels can be set centrally to ensure that no-one can access the whole database. During the campaign we used VICS to upload data models which predicted where we thought Leave voters were likely to be so that we could focus our canvassing efforts, which was important given limited time and resources on the ground. The model produced star ratings so that local teams could target the streets more likely to contain Leave voters.
Data flowed in on the ground and was then analysed by the data science team and integrated with all the other data streaming in. Data models helped us target the ground campaign resources and in turn data from the ground campaign helped test and refine the models in a learning cycle – i.e. VICS was not only useful to the ground campaign but also helped improve the models used for other things. (This was the point of our £50 million prize for predicting the results of the European football championships, which gathered data from people who usually ignore politics – I'm still frustrated we couldn't persuade someone to insure a £350 million prize which is what I wanted to do.) In the official 10 week campaign we served about one billion targeted digital adverts, mostly via Facebook and strongly weighted to the period around postal voting and the last 10 days of the campaign. We ran many different versions of ads, tested them, dropped the less effective and reinforced the most effective in a constant iterative process. We combined this feedback with polls (conventional and unconventional) and focus groups to get an overall sense of what was getting through. The models honed by VICS also were used to produce dozens of different versions of the referendum address (46 million leaflets) and we tweaked the language and look according to the most reliable experiments done in the world (e.g. hence our very plain unbranded 'The Facts' leaflet which the other side tested, found very effective, and tried to copy). I will blog more about this.
These canvassing events represented 80-90% of our ground effort in the last few months, hence some of the reports by political scientists derived from Events pages on the campaign websites, which did not include canvassing sessions, are completely misleading about what actually happened (this includes M Goodwin who is badly confused and confusing, and kept telling the media duff information after he was told it was duff). There was also a big disinformation campaign by Farage's gang, including Bone and Pursglove, who told the media 'Vote Leave has no interest in the ground campaign'. This was the opposite of the truth. By the last 10 weeks we had over 12,000 people doing things every week (we had many more volunteers than this but the 12,000 were regularly active). When Farage came to see me for the last time (as always fixated only on his role in the debates and not the actual campaign which he was sure was lost) he said that he had 7,000 activists who actually did anything. He was stunned when I said that we had over 12,000. I think Farage et al believe their own spin on this subject and were deluded not lying. (Obviously there was a lot of overlap between these two figures.) These volunteers delivered about 70 million leaflets out of a total ~125 million that were delivered one way or another.
While there were some fantastic MPs who made huge efforts on the ground – e.g. Anne Marie Trevelyan – it was interesting how many MPs, nominally very committed to Leave, did nothing useful in their areas nor had any interest in ground campaigning and data. Many were far more interested in trying to get on TV and yapping to hacks than in gathering useful data, including prominent MPs on our Board and Campaign Committee, some of whom contributed ZERO useful data in the entire campaign. Some spent much of the campaign having boozy lunches with Farage gossiping about what would happen after we lost. Because so many of them proved untrustworthy and leaked everything I kept the data science team far from prying eyes – when in the office, if asked what they did they replied 'oh I'm just a junior web guy'. It would have been better if we could have shared more but this was impossible given some of the characters.
VICS is the first of its kind in the UK and provided new opportunities. It is, of course, far from ideal. It was developed very quickly, we had to cut many corners, and it could be improved on. But it worked. Many on the ground, victims of previous such attempts, assumed it would blow up under the pressure of GOTV. It did not. It worked smoothly right through peak demand. This was also because we solved the hardware problem by giving it to Rackspace which did a great job – they have a system that allows automatic scaling depending on the demand so you don't have to worry about big surges overwhelming the system.
There were many things we could have done much better. Our biggest obstacle was not the IN campaign and its vast resources but the appalling infighting on our own side driven by all the normal human motivations described in Thucydides – fear, interest, the pursuit of glory and so on. Without this obstacle we would have done far more on digital/data. Having seen what is offered by London's best communications companies, vast improvements in performance are clearly possible if you hire the right people. A basic problem for people in politics is that approximately none have the hard skills necessary to distinguish great people from charlatans. It was therefore great good fortune that I was friends with our team before the campaign started.
During the campaign many thousands of people donated to Vote Leave. They paid for VICS. Given we spent a lot of money developing it and there is nothing equivalent available on the market and Vote Leave is no more (barring a very improbable event), we thought that we would make VICS available for anybody to use and improve though strictly on the basis that nobody can claim any intellectual property rights over it. It is being made available in the spirit of the open source movement and use of it should be openly acknowledged. Thanks again to the thousands of people who made millions of sacrifices – because of you we won everywhere except London, Scotland and Northern Ireland against the whole Government machine supported by almost every organisation with power and money.
I will write more about the campaign once the first wave of books is published.
PS. Do not believe the rubbish peddled by Farage and the leave.EU team about social media. E.g. a) They boasted publicly that they paid hundreds of thousands of pounds for over half a million Facebook 'Likes' without realising that b) Facebook's algorithms no longer optimised news feeds for Likes (it is optimised for paid advertising). Leave.EU wasted hundreds of thousands just as many big companies spent millions building armies of Likes that were rendered largely irrelevant by Facebook's algorithmic changes. This is just one of their blunders. Vote Leave put our money into targeted paid adverts, not buying Likes to spin stories to gullible hacks, MPs, and donors. Media organisations should have someone on the political staff who is a specialist in data or have a route to talk to their organisation's own data science teams to help spot snake oil merchants.
PPS. If you are young, smart, and interested in politics, think very hard before studying politics / 'political science' / PPE at university. You will be far better off if you study maths or physics. It will be easy to move into politics later if you want to and you will have more general skills with much wider application and greater market value. PPE does not give such useful skills – indeed, it actually causes huge problems as it encourages people like Cameron and Ed Balls to 'fool themselves' and spread bad ideas with lots of confidence and bluffing. You can always read history books later but you won't always be able to learn maths. If you have these general skills, then you will be much more effective than the PPE-ers you will compete against. In a few years, this will be more obvious as data science will be much more visible. A new interdisciplinary degree is urgently needed to replace PPE for those who want to go into politics. It should include the basics of modelling and involve practical exposure to people who are brilliant at managing large complex organisations.
PPPS. One of the projects that the Gove team did in the DfE was funding the development of a 'Maths for Presidents' course, in the same spirit as the great Berkeley course 'Physics for Presidents', based on ideas of Fields Medallist Tim Gowers. The statistics of polling would be a good subject for this course. This course could have a big cultural effect over 20 years if it is supported wisely.
Posted in Artificial Intelligence / Artificial Life / Robotics / BCIs / Computer Science, Decisions, International Relations & War, Physics, Politics, The Sphinx of Psychology & Persuasion, Tools	| Tagged Cameron, Craig Dre, data science, EU, EU referendum, experts, Gove, Gowers, Llewellyn, machine learning, Media, Paul Stephenson, polls, PPE, predictions, pundits, Thucydides, VICS, Victoria Woodcock, Vote Leave | 27 Comments
July 3, 2016 by dominiccummings
On the referendum #19: Final message from Vote Leave HQ to our supporters
Below is the final message from the Vote Leave HQ team to our supporters.
I will be blogging about the campaign at some point over the next few weeks / months.
Dear XXX
WE TOOK BACK CONTROL!
Last week you changed the course of history. Vote Leave took on almost every force with power and money and we won. Britain chose to Vote Leave.
This victory would not have happened without your amazing help and generosity. Thousands of you donated. Thousands of you volunteered. Thousands of you spoke to friends and family on our behalf to spread the message. THANK YOU!
In just ten months we built from scratch an unprecedented national movement that took our campaign to every corner of the country. We got to places that 'politics as usual' ignored. People who have been ignored, and have never been involved in politics before, suddenly spoke out and took action.
In 2008, the worst financial crisis since 1929 hit the world. The people who paid the bills were mainly those on P.A.Y.E. They are still paying. They are also paying the bills for the EU's and the euro's dysfunction. Meanwhile many with power and money who were responsible for the mistakes and were completely wrong in their predictions dodged their fair share of the bills and got rich out of the EU system. We spoke for those on P.A.Y.E.
We did new things. Nobody in the UK has ever successfully built a web-based electoral database. Companies have spent millions and failed. We did it in a few months and succeeded. The combination of this database, our digital communication effort and our ground campaign broke new ground for political campaigns. This database product is worth a lot of money. We will shortly put the code online so that everyone can use it for free in the future (keep an eye on Github if interested). Hopefully it will help other campaigns give the public a powerful voice as we have. We've shown political parties how they can change and stop ignoring large parts of the country.
Why is this important? The British political system is broken in many ways and needs big changes – the EU is not our only problem. Our campaign was never controlled by any party though there were great people from all parties who helped us. All the parties have very deep problems. The way they are structured incentivises MPs to focus on themselves and their party – not the public interest.
It is important that the Conservative leadership candidates accept that the vote must be respected. Both the leading IN candidate (Theresa May) and the leading OUT candidate (Michael Gove) have made clear that if they win they will respect the vote and deliver a new UK-EU deal. This could mean, among other things, democratic control of immigration policy. This could marginalise extremists and allow a fair, sensible, and humane new policy. It could mean new trade deals and new jobs. It could mean more money for health, education, and science.
But we cannot be sure it will happen. In particular, while there are many wonderful civil servants there are also many who regard our victory as a disaster. They will try to stop or minimise changes. Not all the candidates in the Conservative leadership campaign have shown an ability to deliver big changes in the face of civil service opposition. Many in Labour are in complete denial about the real state of opinion and the real problems of the EU. Few MPs have the skills needed to manage normal government departments – never mind the EU negotiation and complex problems that implementing the referendum result require. Many MPs are desperate to ignore any lessons from the referendum and go back to politics as usual. The situation is very worrying.
Westminster cannot be relied upon. Taking back control to Britain is just the first step. The next step should be major political changes in Britain so that the broken Westminster and Whitehall system has to focus on the public interest in a way it does not now. If we increase the power of MPs and officials without changing how they behave, we will not solve our problems. We need organisations like Vote Leave to operate permanently to give a voice to those who otherwise won't be heard.
This campaign did not win because of support in Westminster – it won because of support in the country that has forced Westminster to listen. But three MPs in particular worked closely together and helped us win: Michael Gove (Conservative), Boris Johnson (Conservative), and Gisela Stuart (Labour) who was also a wonderful Chair. We want to thank all three of them too. They put their careers and reputations on the line. THANK YOU Boris, Gisela, and Michael. Thank you too to other MPs of all parties who helped, such as Anne-Marie Trevelyan and Graham Stringer.
It's been a privilege to have your support throughout this campaign. Your dedication brought victory.
On behalf of the team here at Vote Leave, and on behalf of the public, THANK YOU – and goodbye.
The Vote Leave HQ
P.S. If you want to keep in touch with events after we have won, then follow the private blog of our Campaign Director, Dominic Cummings – CLICK HERE. If we ever want to send up a 'bat signal' that Westminster is cheating the vote and we need to form a new movement, you will see the bat signal there…
P.P.S. The website will remain online for many years. We are not using your data for any other purpose. All personal data will be permanently destroyed as we promised at the start. If you want to contribute to our 'lessons learned' investigation, then please take this survey – CLICK HERE
Posted in International Relations & War, The Sphinx of Psychology & Persuasion, UK Politics	| Tagged Boris, Cameron, EU referendum, Gove, Single Market | 13 Comments
On the referendum #8: Unprofessional pundit – Adam Boulton
I thought a few weeks ago I would keep track of some pundits writing on the referendum (cf. the errors of Steve Richards here).
A few texts yesterday told me to read Adam Boulton's Sunday Times column:
'In anticipation of a Greek no, Britain's "no" champions are urging Cameron to play hardball. They are cheered by the appointment of Dominic Cummings to kick-start the campaign. He is advocating a Syriza-style approach, suggesting that if Britain votes no it will still be possible to renegotiate and hold a second referendum to stay in. Cameron must crush this argument fast if he is not to be undermined among Tory activists. A "yes" vote in Greece followed by firm discipline for Athens would do that. In the meantime, he could point out that Cummings's strategic brilliance at the education department led to him losing his job and almost cost the career of his master, Michael Gove.'
A few other hacks have called to say that a No10 spad is claiming I was fired from the DfE, while another one claims I was 'secretly' fired but allowed to claim I resigned. (Great message discipline Dre.)
If Boulton had googled or called, he would have realised he'd been lied to.
Facts? In September 2013 I told Gove I was resigning from the DfE but would stay for a few months to help the transition to a new team. In October 2013, it was reported in a few places (first by Tim Shipman) that I'd resigned. I stayed until 31 January 2014 to help replacements figure out how the DfE worked. Boulton would have seen this immediately if he had googled instead of parroted a briefing.
Boulton's Syriza analogy is also silly. My point about a second referendum had nothing to do with Syriza and was written before the Greek referendum had even been announced.
Boulton has been surly towards me since we were on a platform together discussing the media in 2003. He got cross about criticisms of media professionalism. He's obviously still struggling with basic fact checking.
I can use Google. Up pops straight away a Boulton prediction on the euro from 2001: 'Britons will come round to the idea [joining the euro] once they've handled euro money', a line that was straight out of the Blairite spin doctors' briefing notes at the time. Blairite pundits like Boulton – like their sources Mandelson, Clarke et al – need to be reminded of their duff predictions on the euro when they wheel out all the usual stale conventional wisdom in the coming referendum.
Adam – next time clowns in No10 tell you stuff about the referendum, check to see if it's true before you repeat it…
Ps. AB responds on Twitter: 'So there's no reason to be so pompous, tendentious and offensive. We just see things from different perspectives.' Odd that he sees correcting factual errors as 'offensive'. It's struck me many times over the years how thin-skinned some hacks can be given how they're always criticising others. Tetlock's seminal study on 'political experts' famously showed that the more a pundit is on TV, the more likely they are to be wrong AND not to admit they're wrong.
Posted in UK Politics	| Tagged Boulton, EU referendum, Gove | 2 Comments
May 11, 2015 by dominiccummings
On the Referendum #1: Gove and the Human Rights Act
Within a day of Gove being made Justice Secretary, there is already hysterical and misleading reporting of what might happen to human rights.
The Tory manifesto said: 'The next Conservative Government will scrap the Human Rights Act, and introduce a British Bill of Rights. This will break the formal link between British courts and the European Court of Human Rights, and make our own Supreme Court the ultimate arbiter of human rights matters in the UK.'
It is being reported with dismay by some and excitement by others that Gove will now do this. The dreadful organisation 'Liberty' is issuing hysterical warnings. Tory hacks are rubbing their hands.
Both are very premature.
Fraser Nelson writes in his Spec blog that 'The new Tory majority in the Commons can simply pass a vote stating that the UK Supreme Court is senior to Strasbourg.'
This is wrong except in a very limited sense.
NB. I have not spoken to MG about any of this. But I would be very surprised if before the day is out he is not told the following in his new department.
This is a very basic summary of the relevant legal situation…
1. Until we joined the EEC, all primary legislation had the same force. The common law had long ago developed a doctrine of 'implied repeal' meaning they would give effect to a later statute over a former. Constitutional Acts such as the Act of Union had no greater legal entrenchment than anything else. Parliament could change anything with normal legislation.
2. After we joined the EEC, the common law evolved a new idea – it dropped 'implied repeal' when it touched on the 1972 European Communities Act. This was a classic sensible common law approach. Parliament had said it wanted to join the EEC. The courts therefore said that they would interpret all other laws from the perspective that Parliament wanted to maintain that position. Continuing with implied repeal would have led to legal chaos.
3. Instead, the courts said that if Parliament wanted to amend or repeal the 1972 ECA it would have to do so explicitly. This made perfect sense. Parliament remained sovereign. It could repeal any previous Act regardless of international legal commitments. (Though of course it cannot change the factual existence of those international commitments with all they imply.) But it had to do so explicitly so the courts could be clear about Parliament's intentions and avoid legal chaos. These things were argued in the courts during the 1980s in cases such as Factortame and the current legal position is set out in the judgement on the 'Metric Martyr' case.
4. Blair brought in the Human Rights Act to give effect to the European Convention on Human Rights. The courts extended the common law adjustment of 'implied repeal' to the HRA. There are now two Acts that have a 'superior' legal position that require explicit amending or repeal by Parliament. They have this position not because of what Parliament has said but because of what the common law says. The common law is the ultimate arbiter of Parliamentary sovereignty – an idea that has evolved since the 16th century.
5. It is therefore true that Parliament can repeal the HRA if it does so explicitly. There have been some attempts by radical lawyers to dispute this position, including in the Metric Martyr case, but English judges have so far not taken what would be a politically dynamite position of saying the courts should refuse to accept explicit primary legislation. If they were to do so, it would lead to one of the biggest constitutional conflicts in centuries. It cannot be ruled out. (Similarly attempts to curtail the scope of judicial review could also lead to a mega clash between Parliament and the courts since the courts have hitherto set the scope of JR and Parliament has not interfered. The original development of JR in the 16th century was a great blessing in the development of the rule of law and liberty in Britain and was fundamental to the superior constitutional development of Britain viz the rest of Europe. It now needs major reform but this issue is also extremely fraught, complex, and entangled with the ECA1972, HRA, and how the permanent civil service uses all this to enmesh ministers in management chaos to stop things they don't like.)
6. However, the important question is – does the Government simply have another – 'our own' – human rights Act to replace the current HRA that also gives effect to the ECHR, or does it also withdraw from the ECHR itself? (I.e. does it withdraw from its international legal obligations as well as repeal domestic legislation?) If it does the former (i.e. replace the HRA with its own new version), then the current situation is simply tweaked. The courts may or may not make some small changes to how they interpret the ECHR but the fundamentals would be completely unaffected. Provided we are still committed under international law to the Strasbourg court, then we will continue to suffer from the often abysmal judgements made there. The Supreme Court will not be 'supreme'. The situation could resemble the situation before the HRA when people went straight to Strasbourg to make human rights claims because they could not go via the English courts. This situation was worse than the current situation because the much more sensible English courts had no say on the matter and we were wholly reliant on Strasbourg (where sometimes judges controlled by Putin sit).
7. There is a further complication. The EU has its own Charter of Fundamental Rights. This is the Charter that Blair promised would have no more legal force 'than The Sun or The Beano'. Of course, as usual with the promises of British governments (Labour and Tory) on such issues over the years it was either deeply incompetent or dishonest. Who knows which but the Charter is there and of course it does have legal force. Further, Strasbourg judgements are used by the European Court of Justice (ECJ) in its own judgements so Strasbourg has a separate route to having legal effect in the UK and in English courts.
This means that even if the Government were to a) repeal the HRA and b) leave the ECHR altogether cutting direct ties in international law to Strasbourg, it would still, by virtue of its continuing membership of the EU, be subject in various ways to judgements of the Strasbourg court. The Supreme Court would not be 'supreme'. [I have edited Para 7 slightly, cf. Ps.3 below.]
There are therefore two connected very big questions that MPs and hacks need to ask.
A) Will the Government leave the ECHR so that not only will we have our own Human Rights Act but British citizens will not be able to go to Strasbourg any more than US or Chinese citizens can?
B) Will the Government roll the ECHR/Strasbourg supremacy issue into its renegotiation of EU membership in order that the manifesto promise is kept and the Supreme Court is made 'supreme'? If not, the Luxembourg court (ECJ) will continue to impose the views of the Strasbourg court (ECHR) even if No10 takes the radical option on question A (which it probably will not), and the English courts will enforce such ECJ judgements absent explicit amending or repeal of the ECA.
I would bet the odds of both happening are less than 5%. Even if I am wrong and No10 attempts both, it is very hard to see how our membership of the EU would work such that we alone are not bound by Brussels and Luxembourg interpretations of the Strasbourg court. On this issue as on so many others to come, there is no serious half-way house that renegotiation can bring. Mandarins like Hannay and Kerr – so wrong about geopolitics and post-war history in my opinion – are, obviously, right when they point such things out.
This should make clear that Gove does not have the power to solve these problems unless No10 decides on a truly radical approach to the EU renegotiation. The answers can only come from No10 if they come at all. Liberty can calm down. Excited Tory pundits should keep their enthusiasm for radicalism in check. David Cameron has successfully played on the ignorance of MPs and the media about these issues for a decade to encourage a feeling of radicalism in some quarters while the lawyers read the actual words and know the truth.
If you want to understand the history of legal thinking over the issues of Parliamentary sovereignty and the EU/HRA, I strongly suggest you read the judgement in the Metric Martyr case. For MPs and hacks who need to understand Government proposals, you need to shell out some cash on top notch public lawyers who specialise in this area and you need to focus on the detail. The gap between the alpha lawyers and the rest on these issues is huge and worth the extra cash. These issues are much more intellectually demanding than public service reform and specialist knowledge about them is much rarer. Also, >95% of those who have the required specialist knowledge have either an ideological or financial interest (or both) in the status quo.
I was campaign director of the campaign that opposed Britain joining the euro 1998-2002. After that, decisions by a few people meant that the momentum and structure built by that campaign was – in my opinion disastrously – destroyed. A decade in which people should have been figuring out the answers to questions like those sketched above was squandered. Too many people focused on clamouring for a referendum instead of figuring out the extreme complexities of the issues. This was all the more odd given how many Eurosceptics complain that even winning a referendum in Europe has just led to another referendum. The fact that a referendum on the EU would not only be very hard to win but would also not even guarantee victory anyway has been almost entirely ignored. Why wouldn't Whitehall and Brussels respond to an unlikely OUT victory by saying – 'Ok, well now we'll negotiate a new OUT deal and, of course, the people must have their say on that, mustn't they…'?
Those who want to reverse (what I see as) the historic error of Macmillan et al deciding post-Suez that Britain had to join the EEC now have to do something that is alien to modern SW1 – build a non-party machine capable of top notch policy thinking (integrating many different forms of expertise) and communications (far beyond the level displayed by anybody in the election) that can also suppress the destructive dynamics of eurosceptic internal squabbling. None of the parties has a coherent picture of Britain's future – their manifestos are asinine, without answers to any of the big questions of economics, technology, or geopolitics. Whitehall has no alternative to our trajectory of decline and self-delusion. (Click HERE for a long-term view of this problem.) But the challenge of winning a referendum and actually leaving the EU on good terms is even harder than fixing these problems – in one sense, it almost presupposes their partial solution. Having mostly squandered a decade, 'the silent artillery of time' is on the side of the status quo…
Ps. Please leave corrections etc below, particularly to relevant legal links. I will blog more on these issues and link to some of the best stuff. My last blog on the EU battle and a Times op-ed I wrote is HERE (also NB. – the model of swing voter psychology applied during the last election is wrong, as I explained in this blog, and contributed to failures of prediction).
Ps. 2 [added later]. I should also have pointed out that there is another huge complication – devolution and a solution to the Scottish problem. Whatever the new Government does – whether a federal UK or not – will also affect and be affected by the HRA and broader EU issues.
Ps. 3 [added later]. The EU accession to the ECHR is complicated by this December 2014 judgement (thanks to the lawyers who speedily pointed this out as I hadn't noticed it). The EU tried to sign up to the ECHR in its own right but this was, ironically, ruled illegal by the ECJ itself, defending its own position. However, I do not think it affects the main point. The ECJ will still apply Strasbourg judgements in its own decisions as it wishes, as it has for decades. Even leaving the ECHR entirely would not make the Supreme Court 'supreme' over Strasbourg, and the ECHR would continue to dominate English law on various issues via EU law and the ECJ, and English courts would enforce this absent amending or repealing the 1972 ECA. Exactly how it would work is very complicated and surely unknowable in advance. The only way for the Supreme Court to be 'supreme' viz the EU and ECHR is by a) repealing the HRA, b) withdrawing from the ECHR international treaty, and c) repealing or amending the 1972 ECA to prevent the ECJ (Luxembourg) being superior to the English courts.]
Ps. 4 [added later]. Many seem to assume that Gove will behave in a similar way in the MoJ to the DfE. I think this is mistaken. The situations are very different. E.g.1 In the DfE we had prepared for the job over years including working out in secret with lawyers long before the election what the 2010 Academies Act should do. MG has had no similar preparation for the MoJ. E.g. 2 The issues are intellectually much harder. The complexities of the HRA are hard even for very clever lawyers. By comparison school reform is controversial and hard in various practical ways but does not present the same profound intellectual difficulties and subtleties. E.g. 3. MG made mistakes in how he communicated. He has doubtless learned from his experience. E.g. 4. In the DfE, very little needed legislation therefore we could largely ignore No10 and just do things. In the MoJ, the situation is different as I explain above. E.g. 5. For all these reasons and more, it would make sense for MG to go at a much more careful pace. This does not mean he is giving up / gone soft or any other lobby clichés – it just reflects that No10 is in charge of the most important thing and it is extremely unlikely No10 will even try to solve the core problems. The MoJ needs a very different approach and our approach in the DfE would be a poor guide.
Ps. 5 (added 27 May) The news today about this issue being delayed should be no surprise given the above. It strengthens my view that there is approximately zero chance of the core issues with the HRA being dealt with while Cameron is PM. Dre has made the Government look stupid by briefing around MG's appointment that the Human Rights Act would be dealt with within '100 days'. Now that Dre is 'coordinating domestic policy', it is official that policy is a subset of crap spin in the No10 organogram and, free of Crosby's discipline, Cameron is back to his familiar role as the nation's UberPundit. For ten years the lobby has swallowed his spin on human rights. One advantage of today's media car crash on this is that they may finally realise that Cameron has never had any intention of solving this problem. Self-described eurosceptics who believed him have no excuse for continuing self-delusion.
Ps. 7 Someone emails to say 'why approximately zero?' Because if, for example, a bomb goes off in London then the whole conventional wisdom will spin on its axis, people who gave self-important interviews about their determination to 'protect civil liberties' will give new self-important interviews saying 'of course there must be sensible modifications', polls will show >80% support for ditching the supremacy of Strasbourg etc. Precisely because Cameron has no principles, when he feels a gun is put to his head he can change his mind very fast. His party has been slow to understand this but something like a bomb would turn the debate upside down in hours. It is obviously impossible to quantify the probability of such an event (cf. the 2008 JASON study which I'll dig out). Obviously changing such profound things in such circumstances is likely to lead to many errors particularly when a Prime Minister has no other model of behaviour than steering by the wind of the pundits.
Ps. 8 The Telegraph splash today (1 June 2015) says that Cameron has already ruled out leaving the jurisdiction of the Strasbourg court. No surprise. The No10 line that 'Gove hasn't made up his mind yet' doesn't make sense. Obviously only the prime minister can decide whether to withdraw from an international treaty, as removing the jurisdiction of the Strasbourg court requires. Gove's job on the HRA is to punt it into the long grass then deliver a fudge that leaves Strasbourg in charge. The sensible thing for him to do is give this doomed project to a junior minister and focus on other priorities.
Posted in International Relations & War, UK Politics	| Tagged Cameron, ECHR, EU referendum, Gove, Human Rights Act | 35 Comments
February 5, 2015 by dominiccummings
Bureaucratic cancer and the sabotage of A Level reform
'Bureaucracy is cancerous in head and limbs; only its belly is sound and the laws it excretes are the most straightforward shit in the world… With this bureaucracy including the judges on the bench we can have press laws written by angels and they cannot lift us from the swamp. With bad laws and good civil servants one can still govern, with bad civil servants the best laws cannot help.' Otto von Bismarck, 1850.
'I had the agreement in principle of my colleagues; I had the agreement in principle of the entire Landtag; and yet, although minister-president, I found myself absolutely unable to bring the matter one step further along. Agreement does not help me at all when passive resistance – from what direction in this complicated machine is impossible to learn – is conducted with such success that I am scarcely in a position after two to three years to answer even the most basic questions.' Otto von Bismarck, 1878.
If the most effective political operator of the modern world frequently complained about the difficulty of enforcing policy against a hostile bureaucracy, we should not be surprised if similar problems recur over and over again.
Here is an interesting example of how education policy is made and how Whitehall works.
In 2012, we announced that the DfE would step back from controlling A Levels and give universities control. (Allegra Stratton ran the original story on Newsnight.) The main mechanism was ALCAB. It was a nightmare to set up partly because although subject experts very much wanted to be involved the administrators who control universities wanted to stay out of the controversy and said to us in the DfE 'we don't want to have to say publicly that A Level papers are bad'.
We forced ALCAB to be created. MG and I spent a lot of time in awful meetings forcing it through. Its main role was supposed to be an annual review of specific A Level papers so that professors XYZ could say 'hopeless question in the Edexcel physics paper, it gets the definition of entropy wrong again, it fails to test XXX' etc.
The DfE has closed this committee down. It emerged via this Times Higher Education story.
I pointed a few hacks to it. They have called the DfE press office and spads. Both of those entities were given a line from officials saying 'ALCAB's work is done, no story here'. (Cf. Forsyth's blog here.)
This is a lie. The main role was an annual review process. This should have been conducted this year and 2016 in preparation for new A Levels in 2017. It was envisaged as a permanent role. Interestingly, the letters completely elide this main role out of existence and present ALCAB as having only a temporary role.
Now this annual review won't happen.
This is almost a Jedi-level operation from DfE officials. The DfE hated giving away control, obviously, and hated ALCAB. The very point of the process – a sword of Damocles in the form of eminent professors saying 'crap questions' each year – was supposed to force the DfE, exam boards, and Ofqual to raise their game. You can imagine how popular this was. Now the situation will revert to the status quo – the DfE firmly in charge and those pesky professors who point out things like – specific papers do not test the maths skills in the specifications – are happily excluded, with no 'unhelpful' public scrutiny of standards.
I very much doubt that poor Nicky Morgan Nicky Morgan [*see end] realises what she has done. It was probably a letter buried deep in her box weeks ago that she had no reason to suspect meant she was being used to subvert reform and entrench Whitehall's power. It is impossible for a new minister to spot all such things – you don't know what you don't know. We can also safely bet that No10 has not the faintest idea about what ALCAB is or what the annual review process was supposed to do.
This is how Whitehall closes down threats to its power. Although it is systemically incompetent viz policy and implementation, its real focus is on its own power, jobs, and money. To these, it pays careful attention and deploys its real skills.
It is possible that the hard struggle to improve A Levels and remove politicians' and Whitehall's grip of them is now substantially lost, without the MPs having a clue as to why and the details lost in a miasma of untraceable decisions and discussions.
Nicky Morgan and her spads should ask Rose (head of private office) and Wormald (Perm Sec) not just 'how did this happen?', but also 'why were we and the press office given lies to tell the media?' They would also be well advised to make clear that a repetition of this fancy footwork will mean someone fired. But of course this will have little effect. The officials are lining up their holidays and their own plans for the future, safe in the happy knowledge that whoever 'wins' the election, they will remain in charge. The MPs of all parties are largely content for this situation to continue. In the focus groups, swing voters will continue to say 'they're all the same' with much more accuracy than they realise, but few in Westminster are really listening and even fewer know what is to be done…
I will blog a few reflections on No10's 'schools week' tomorrow. NB. notice how, just as I wrote in The Hollow Men, this No10 'schools week' is like all the others – two days of rubbish gimmicks, a self-inflicted cockup ('real terms cuts to the budget'), followed by silence such that by Friday the 8 people who knew it was 'schools week' have themselves forgotten? Plus ca change…
Ps. If you want details on the devaluation of exams since 1988, and therefore why the annual review process was so important, read THIS.
UPDATE. Some have asked 'how much confidence did you have in ALCAB doing a good job?' Answer? Initially not much. They are all under huge pressure to say everything is fine. Initially for example, despite physics departments across the country complaining about the removal of calculus from Physics A Level (complaints that practically none of them will repeat publicly because of fear of their VC office), it did not look like ALCAB would be much use and they rejected calls from various professors I know on this subject. There is massive political pressure to focus exclusively on the numbers taking an A Level rather than the quality of the A Level.
But my hope was that by creating something that would be seen as the 'voice of the university subject experts', they would have to listen and adapt in order to maintain credibility and avoid embarrassing challenges. There are more and more enraged academics fed up of VC offices lying to the media and misrepresenting academics' opinions. I thought that creating something would push the debate in increasingly sensible directions where the emphasis would be on the skills needed on arrival at university. Now, everything to do with A Levels is dominated by political not educational concerns about the numbers doing them and 'access'. This has helped corrupt the exam system. If we had professors of physics, French, music etc every year publicly humiliating exam boards for errors, this would soon improve things from a low base and make it much harder for MPs and Whitehall to keep corrupting public exams.
[* I wrote 'poor Nicky Morgan' with the feeling – poor her, I know what it's like to be pottering around in the DfE dealing with all sorts of problems before the horror of Question Time then someone walks in with a new bigger problem… But a few people email to say it sounds patronising which was not deliberate, hence deletion…]
Posted in Curriculum & Exams, Education	| Tagged Gove, Ofqual, Spads, Wormald | 8 Comments
Standards In English Schools Part I: The introduction of the National Curriculum and GCSEs
The Introduction to this series of blogs, HERE, sets out the background and goals.
There are many different senses in which people discuss 'standards'. Sometimes they mean an overall judgement on the performance of the system as judged by an international test like PISA. Sometimes they mean judgements based on performance in official exams such as KS2 SATs (at 11) or GCSEs. Sometimes they mean the number of schools above or below a DfE 'floor target'. Sometimes they mean the number of schools and/or pupils in Ofsted-defined categories. Sometimes people talk about 'the quality of teachers'. Sometimes they mean 'the standards required of pupils when they take certain exams'. Today, the media is asking 'have Academies raised standards?' because of the Select Committee Report (which, after a brief flick through, seems to have ignored most of the most interesting academic studies done on a randomised/pseudo-randomised basis).
This blog in the series is concerned mainly with the questions of – what has happened to the standards required of pupils when they take GCSEs and A Levels as a result of changes since the mid-1980s, and how do universities and learned societies judge the preparation of pupils for further studies. Have the exams got easier? Do universities and learned societies think pupils are well-prepared for further studies?
I will give a very short potted history of the introduction of GCSEs and the National Curriculum before examining the evidence of their effects. If you are not interested in the history, please skip to the Section B on Evidence. If you just want to see my Conclusions, scroll to the end for a short section.
I stress that my goal is not to argue for a return to the pre-1988 system of O Levels and A Levels. While it had some advantages over the existing system, it also had profound problems. I think that an unknown fraction of the cohort could experience far larger improvements in learning than we see now if they were introduced to different materials in different ways, rather than either contemporary exams or their predecessors, but I will come to this argument, and why I have this belief, in a later blog.
I have used the word 'Department' to represent the DES of the 1980s, the DfE of post-2010, and its different manifestations in between.
This is just a rough first stab at collecting things I've shoved in boxes, emails etc over the past few years. Please leave corrections and additions in Comments.
A. A very potted history
Joseph introduces GCSEs – 'a right old mess'
The debate over the whole of education policy, and particularly the curriculum and exams, changed a lot after Callaghan's Ruskin speech in 1976 and the Department's Yellow Book. Before then, the main argument was simply about providing school places and the furore over selection. After 1976 the emphasis shifted to 'standards' and there was growing momentum behind a National Curriculum (NC) of some sort and reforms to the exam system.
Between 1979-85, the Department chivvied LAs on the curriculum but had little power and nothing significant changed. Joseph was too much of a free marketeer to support a NC so its proponents could not make progress.
Joseph was persuaded to replace O Levels with GCSEs. He thought that the outcome would be higher standards for all but he later complained that he had been hoodwinked by the bureaucratic process involving The Schools Examination Committee (SEC). He later complained:
'I should have fought against flabbiness in general more than I did… I thought I did, but how do you reach into such a producer-oriented world? … "Stretching" was my favourite word; I judged that if you leant on that much else would follow. That's what my officials encouraged me to imagine I was achieving… I said I'd only agree to unify the two examinations provided we established differentiation [which he defined as 'you're stretching the academic and you're stretching the non-academic in appropriate ways'], and now I find that unconsciously I have allowed teacher assessment, to a greater extent than I assumed. My fault … my fault… it's the job of ministers to see deeply… and therefore it's flabby… You don't find me defending either myself or the Conservative Party, but I reckon that we've all together made a right old mess of it. And it's hurt most those who are most vulnerable.' (Interview with Ball.)
I have not come across any other ministers or officials from this period so open about their errors.
The O Level survived under a different name as an international exam provided by Cambridge Assessment. It is still used abroad including in Singapore which regularly comes in the top three in all international tests. Cambridge Assessment also offers an 'international GCSE' that is, they say, tougher than the 'old' GCSE (i.e. the one in use now before it changes in 2015) but not as tough as the O Level. This international GCSE was used in some private schools pre-2010 along with 'international GCSEs' from other exam boards. From 2010, state schools could use iGCSEs. In 2014, the DfE announced that it would stop this again. I blogged on this decision HERE.
Entangled interests – Baker and the National Curriculum
In 1986, Thatcher replaced Joseph with Baker hoping, she admitted, that he would make up 'in presentational flair what ever he lacked in attention to detail'. He did not. Nigel Lawson wrote of Baker that 'not even his greatest friends would describe him as a profound thinker or a man with mastery of detail'. Baker's own PPS said that at the morning meeting 'the main issue was media handling'. Jenny Bacon, the official responsible for The National Curriculum 5-16 (1987), said that Baker liked memos 'in "ball points" … some snappy things with headings. It wasn't glorious continuous prose…[Ulrich, a powerful DES official] was appalled but Baker said "That's just the kind of brief I want".'
Between 1976 and 1986, concern had grown in Whitehall about the large number of awful schools and widespread bad teaching. Various intellectual arguments, ideology, political interests (personal and party), and bureaucratic interests aligned to create a National Curriculum. Thatcherites thought it would undermine what they thought of as the 'loony left', then much in the news. Baker thought it would bring him glory. The Department and HMI rightly thought it would increase their power. After foolishly announcing CTCs at Party Conference, thus poisoning their brand with politics from the start, Baker announced he would create a NC and a testing system at 7, 11, and 14.
The different centres of power disagreed on what form the NC would take. HMI lobbied against subjects and wanted a NC based on 'areas of expertise', not traditional subjects. Thatcher wanted a very limited core curriculum based on English, maths, and science. The Department wanted a NC that stretched across the whole curriculum. Baker agreed with the Department and dismissed Thatcher's limited option as 'Gradgrind'.
In order to con Thatcher into agreeing his scheme, Baker worked with officials to invent a fake distinction between 'core' and 'foundation' subjects. As Baker's Permanent Secretary Hancock said, 'We devised the notion of the core and the foundation subjects but if you examine the Act you will see that there is no difference between the two. This was a totally cynical and deliberate manoeuvre on Kenneth Baker's part.'
The 1988 Act established two quangos to be what Baker called 'the twin guardians of the curriculum' – The National Curriculum Council (NCC), focused on the NC, and The Schools Examinations and Assessment Council (SEAC), focused on tests. Once the Act was passed, Baker's junior minister Rumbold said that 'Ken went out to lunch.' Like many ministers, he did not understand the importance of the policy detail and the intricate issues of implementation. He allowed officials to control appointments to the two vital committees and various curriculum working groups. Even Baker's own spad later said that Baker was conned into appointing 'the very ones responsible for the failures we have been trying to put right'. Baker forlornly later admitted that 'I thought you could produce a curriculum without bloodshed. Then people marched over mathematics. Great armies were assembled', and he 'never envisaged it would be as complex as it turned out to be'. Bacon, the official responsible for the NC, said that Baker 'wasn't interested in the nitty gritty'. Nicholas Tate (who was at the NCC and later headed the QCA) said that Baker was 'affable but remote. He didn't trouble his mind with attainment targets. He was resting on his laurels.' Hancock, his Permanent Secretary, said that 'after 1987 he became increasingly arrogant and impatient'. In 1989, Baker was moved to Party Chairman leaving behind chaos for his successor.
According to his colleagues, Baker was obsessed with the media, he did not try to understand (and did not have the training to understand) the policy issues in detail, and he confused the showmanship necessary to get a bill passed with serious management – he described himself as 'a doer' but the 'doing' in his mind consisted of legislation and spin. He did not even understand that there were strong disputes among teachers, subject bodies, and educationalists about the content of the NC – never mind what to do about these disputes. (Having watched the UTC programme from the DfE, the same traits were much in evidence thirty years later.)
Baker's legacy 1989 – 1997: Shambles
Baker's memoirs do not mention the report of The Task Group on Assessment (TGAT), chaired by Professor Paul Black, commissioned by Baker in 1987 to report on how the NC could be assessed. The plan was very complicated with ten levels of attainment having to be defined for each subject. Thatcher hated it and criticised Baker for accepting it. Meanwhile the Higginson Report had recommended replacing A Levels with some sort of IB type system. Bacon said that 'the political trade-off was Higginson got ditched … and we got TGAT. In retrospect it may have been the wrong trade off.'
MacGregor could not get a grip of the complexity. He did not even hire a specialist policy adviser because, he said, 'I didn't feel I needed one.' He blamed Baker for the chaos who, he said, 'hadn't spent enough time thinking about who was appointed to the bodies. He left it to officials and didn't think through what he wanted the bodies to do. For the first year I was unable to replace anybody.' The chairman of NCC described how they used 'magic words to appease the right' and get through what they wanted. The officials who controlled SEAC stopped the simplification that Thatcher wanted using the 'legal advice' card, claiming that the 1988 Act required testing of all attainment targets. (I had to deal with the same argument 25 years later.) MacGregor was trapped. He had an unworkable system and was under contradictory pressure from Thatcher to simplify everything and from Baker to maintain what he had promised.
Clarke bluffed and bullied his way through 18 months without solving the problems. His Permanent Secretary described the trick of getting Clarke to do what officials wanted: 'The trick was to never box him into a corner… Show him where there was a door but never look at that door, and never let on you noticed when he walked through.' Like MacGregor, Clarke blamed Baker for the shambles: '[Baker] had set up all these bloody specialist committees to guide the curriculum, he'd set up quango staff who as far as I could see had come out of the Inner London Education Authority the lot of them.' Clarke solved none of the main problems with the tests, antagonised everybody, and replaced HMI with Ofsted.
After his surprise win, Major told the Tory Conference in 1992, 'Yes it will mean another colossal row with the education establishment. I look forward to that.' Patten soon imploded, the unions went for the jugular over the introduction of SATs, and by the end of 1993 Number Ten had backtracked on their bellicose spin and was in full retreat with a review by Dearing (published 1994). Suddenly, the legal advice that had supposedly prevented any simplification was rethought and officials told Dearing that the legal advice did allow simplification after all: 'our advice is that the primary legislation allows a significant measure of flexibility'. (In my experience, one of the constants of Whitehall is that legal advice tends to shift according to what powerful officials want.) Dearing produced a classic Whitehall fudge that got everybody out of the immediate crisis but did not even try to deal with the fundamental problems, thus pushing the problems into the future.
The historian Robert Skidelsky, helping SEAC, told Patten 'these tests will not run' and he should change course but Patten shouted 'That is defeatist talk.' Skidelsky decided to work out a radically simpler model than the TGAT system with a small group in SEAC: 'We pushed the model through committee and through the Council and sent it off to John Patten. We never received a reply. Six months after I resigned Emily Blatch approached me and said she had been looking for my paper on Assessment but no one seems to know where it is.'
Patten was finished. Gillian Shephard was put in to be friendly to the unions and quiet the chaos. Soon she and Major had also fallen out and the cycle of briefing and counter-briefing against Number Ten returned with permanent policy chaos. One of her senior officials, Clive Saville, concluded that 'There was a great intellectual superficiality about Gillian Shephard and she was as intellectually dishonest as Shirley Williams. She was someone who wanted to be liked but wasn't up to the job.'
A few thoughts on the process
The Government had introduced a new NC and test system and replaced O Levels with GCSEs. (They also introduced new vocational qualifications (NVQs) described by Professor Alan Smithers as a 'disaster of epic proportions … utterly lightweight'.) The process was a disastrous bungle from start to finish.
Thatcher deserves considerable blame. She allowed Baker to go ahead with fundamental reforms without any agreed aims or a detailed roadmap. She knew, as did Lawson, that Baker could not cope with details yet appointed him on the basis of 'presentational flair' (media obsession is often confused with 'presentational flair').
The best book I have read by someone who has worked in Number Ten and seen why the Whitehall architecture is dysfunctional is John Hoskyns' Just In Time. Extremely unusually for someone in a senior position in No10, Hoskyns both had an intellectual understanding of complex systems and was a successful manager. Inevitably, he was appalled at how the most important decisions were made and left Number Ten after failing to persuade Thatcher to tear up the civil service system. Since then, everybody in Number Ten has been struggling with the same issues. (If she had taken his advice history might have been extremely different – e.g. no ERM debacle.) His conclusion on Thatcher was:
'The conclusion that I am coming to is that the way in which [Thatcher] herself operates, the way her fire is at present consumed, the lack of a methodical mode of working and the similar lack of orderly discussion and communication on key issues, means that our chance of implementing a carefully worked out strategy – both policy and communications – is very low indeed… Difficult problems are only solved – if they can be solved at all – by people who desperately want to solve them… I am convinced that the people and the organisation are wrong.' (Emphasis added.)
Arguably the person who knowingly appoints someone like Baker is more to blame for the failings of Baker than Baker is himself. Major and the string of ministers that followed Baker were doomed. They were not unusually bad – they were representative examples of those at the apex of the political process. They did not know how to go about deciding aims, means, and operations. They were obsessed with media management and therefore continually botched the policy and implementation. They could not control their officials. They could not agree a plan and blamed each other. If they were the sort of people who could have got out of the mess, then they were the sort of people who would not have got into the mess in the first place.
Officials over-complicated everything and, like ministers, did not engage seriously with the core issue – what should pupils of different abilities be doing and how can we establish a process where we can collect reliable information. The process was dominated by the same attitude on all sides – how to impose a mentality already fixed.
It was also clearly affected by another element that has contemporary relevance – the constant churn of people. Just between summer 1989 and the end of 1992, there was: a new Permanent Secretary in May 1989, a new SoS in July 1989 (MacGregor), another new SoS in November 1990 (Clarke), a new PM and No10 team (Major), new heads for the NCC and SEAC in July 1991, then another new SoS in spring 1992 (Patten) and another new Permanent Secretary. Everybody blamed problems on predecessors and nobody could establish a consistent path.
Even its own Permanent Secretaries later attacked the DES. James Hamilton (1976-1983) was put into DES in June 1976 from the Cabinet Office to help with the Ruskin agenda and found a place where 'when something was proposed someone would inevitably say, "Oh we tried that back in whenever and it didn't work"…'. Geoffrey Holland (1992-3) admitted that, 'It [DES] simply had no idea of how to get anything off the ground. It was lacking in any understanding or experience of actually making things happen.'
A central irony of the story shows how dysfunctional the system was. Thatcher never wanted a big NC and a complicated testing system but she got one. As some of her ideological opponents in the bureaucracy tried to simplify things when it was clear Baker's original structure was a disaster, ministers were often fighting with them to preserve a complex system that could not work and which Thatcher had never wanted. This sums up the basic problem – a very disruptive process was embarked upon without the main players agreeing what the goal was.
Although the think tanks were much more influential in this period than they are now, Ferdinand Mount, head of Thatcher's Policy Unit, made a telling point about their limitations: 'Enthusiasts for reform at the IEA and the CPS were prodigal with committees and pamphlets but were much less helpful when it came to providing practical options for action. This made it difficult for the Policy Unit's ideas to overcome the objections put forward by senior officials'. Thirty years later this remains true. Think tanks put out reports but they rarely provide a detailed roadmap that could help people navigate such reforms through the bureaucracy and few people in think tanks really understand how Whitehall works. This greatly limits their real influence. This is connected to a wider point. Few of those who comment prominently on education (or other) policy understand how Whitehall works, hence there is a huge gap between discussions of ideal policy and what is actually possible within a certain timeframe in the existing system, and commentators think that all sorts of things that happen do so because of ministers' wishes, confusing public debate further.
I won't go into the post-1997 story. There are various books that tell this whole story in detail. The National Curriculum remained but was altered; the test system remained but gradually narrowed from the original vision; there were some attempts at another major transformation (such as Tomlinson's attempt to end A Levels, thwarted by Blair) but none took off; money poured into the school system and its accompanying bureaucracy at an unprecedented rate but, other than a large growth in the number and salaries of everybody, it remained unclear what if any progress was being made.
This bureaucracy spent a great deal of taxpayers' money promoting concepts such as 'learning styles' and 'multiple intelligences' that have no proper scientific basis but which nevertheless were successfully blended with old ideas from Vygotsky and Piaget to dominate a great deal of teacher training. A lot of people in the education world got paid an awful lot of money (Hargreaves, Waters et al) but what happened to standards?
(The quotes above are taken mainly from Daniel Callaghan's Conservative Party Education Policies 1976-1997.)
B. The cascading effects of GCSEs and the National Curriculum
Below I consider 1) the data on grade inflation in GCSEs and A Levels, 2) various studies from learned societies and others that throw light on the issue, 3) knock-on effects in universities.
1. Data on grade inflation in GCSEs and A Levels
We do not have an official benchmark against which to compare GCSE results. The picture is therefore necessarily hazy. As Coe has written, 'we are limited by the fact that in England there has been no systematic, rigorous collection of high-quality data on attainment that could answer the question about systemic changes in standards.' This is one of the reasons why in 2013 we, supported by Coe and others, pushed through (against considerable opposition including academics at the Institute of Education) a new 'national reference test' in English and maths at age 16, which I will return to in a later blog.
However, we can compare the improvement in GCSE results with a) results from international tests and b) consistent domestic tests uncontrolled by Whitehall.
The first two graphs below show the results of this comparison.
Chart 1: Comparison of English performance in international surveys versus GCSE scores 1995-2012 (Coe)
Chart 2: GCSE grades achieved by candidates with same maths & vocab scores each year 1996-2012 (Coe)
Professor Coe writes of Chart 1:
'When GCSE was introduced in 1987 [I think he must mean 1988 as that was the first year of GCSEs or else he means 'the year before GCSEs were first taken'], 26.4% of the cohort achieved five grade Cs or better. By 2012 the proportion had risen to 81.1%. This increase is equivalent to a standardised effect size of 1.63, 3 or 163 points on the PISA scale… If we limit the period to 1995 – 2011 [as in Chart 1 above] the rise (from 44% to 80% 5A*-C) is equivalent to 99 points on the PISA scale [as superimposed on Chart 1]… [T]he two sets of data [international and GCSEs] tell stories that are not remotely compatible. Even half the improvement that is entailed in the rise in GCSE performance would have lifted England from being an average performing OECD country to being comfortably the best in the world. To have doubled that rise in 16 years is just not believable…
'The question, therefore, is not whether there has been grade inflation, but how much…' [Emphasis added.] (Professor Robert Coe, 'Improving education: a triumph of hope over experience', 18 June 2013, p. vi.)
Chart 2 plots the improving GCSE grades achieved by pupils scoring the same each year in a test of maths and vocabulary: pupils scoring the same on YELLIS get higher and higher GCSE grades as time passes. Coe concludes that although 'it is not straightforward to interpret the rise in grades … as grade inflation', the YELLIS data 'does suggest that whatever improved grades may indicate, they do not correspond with improved performance in a fixed test of maths and vocabulary' (Coe, ibid).
This YELLIS comparison suggests that in 2012 pupils received a grade higher in maths, history, and French GCSE, and almost a grade higher in English, than students of the same ability in 1996.
It is important to note that neither of Coe's charts or measurements include the effects of either a) the initial switch from O Level to GCSE or b) what changed with GCSEs from 1988 – 1995.
The next two charts show this earlier part of the story (both come from Education: Historical statistics, House of Commons, November 2012). NB. they have different end dates.
Chart 3: Proportion getting 5 O Levels / GCSEs at grade C or higher 1953/4 – 2008/9
Chart 4: Proportion getting 1+ or 3+ passes at A Level 1953/4 – 1998/9
Chart 3 shows that the period 1988-95 saw an even sharper increase in GCSE scores than post-1995 so a GCSE/YELLIS style comparison that included the years 1988-1995 would make the picture even more dramatic.
Chart 4 shows a dramatic increase in A Level passes after the introduction of GCSEs. One interpretation of this graph, supported by the 1997-2010 Government and teaching unions, is that this increase reflected large real improvements in school standards.
There is GCSE data that those who believe this argument could cite. In 1988, 8% of GCSEs were awarded an 'A' in GCSE. In 2011, 23% of GCSEs were awarded an 'A' or 'A*' in GCSE. The DfE published data in 2013 which showed that the number of pupils with ten or more A* grades trebled 2002-12. This implies a very large increase in the numbers of those excelling at GCSE, which is consistent with a picture of a positive knock-on effect on improving A Level results.
However, we have already seen that the claims for GCSEs are 'not believable' in Coe's words. It also seems prima facie very unlikely that a sudden large improvement in A Level results from 1990 could be the result of immediate improvements in learning driven by GCSEs. There is also evidence for A Levels similar to the GCSE/YELLIS comparison.
Chart 5: A level grades of candidates having the same TDA score (1988-2006)
Chart 5 plots A Level grades in different subjects against the international TDA test. As with GCSEs, this shows that pupils scoring the same in a non-government test got increasingly higher grades in A Levels. The change in maths is particularly dramatic from an 'Unclassified' mark in 1988 to a B/C in 2006.
What we know about GCSEs combined with this information makes it very hard to believe that the sudden dramatic increase in A Level performance since 1990 is because of real improvements and suggests another interpretation: these dramatic increases in A Level results reflected (mostly or entirely) A Levels being made significantly easier probably in order to compensate for GCSEs being much easier.
However, the data above can only tell part of the story. Logically, it is hard or impossible to distinguish between possible causes just from these sorts of comparisons. For example, perhaps someone might claim that A Level questions remained as challenging as before but grade boundaries moved – i.e. the exam papers were the same but the marking was easier. I think this is prima facie unlikely but the point is that logically the data above cannot distinguish between various possible dynamics.
Below is a collection of studies, reports, and comments from experts that I have accumulated over the past few years that throws light on which interpretation is more reasonable. Please add others in Comments.
(NB. David Spiegelhalter, a Professor of Statistics at Cambridge, has written about problems with PISA's use of statistics. These arguments are technical. To a non-specialist like me, he seems to make important points that PISA must answer to retain credibility and the fact that it has not (as of the last time I spoke to DS in summer 2014) is a blot on its copybook. However, I do not think they materially affect the discussion above. Other international tests conducted on different bases all tell roughly the same story. I will ask DS if he thinks his arguments do undermine the story above and post his reply if any.)
2. Studies 2007 – now
NB1. Most of these studies are comparing changes over the past decade or so, not the period since the introduction of the NC and GCSEs in the 1980s.
NB2. I will reserve detailed discussion of the AS/A2/decoupling argument for a later blog as it fits better in the 'post-2010 reforms' section.
Learned societies. The Royal Society's 2011 study of Science GCSEs: 'the question types used provided insufficient opportunity for more able candidates … to demonstrate the extent of their scientific knowledge, understanding and skills. The question types restricted the range of responses that candidates could provide. There was little or no scope for them to demonstrate various aspects of the Assessment Objectives and grade descriptions… [T]he use of mathematics in science was examined in a very limited way.' SCORE also published (2012) evidence on science GCSEs which reported 'a wide variation in the amount of mathematics assessed across awarding organisations and confirmed that the use of mathematics within the context of science was examined in a very limited way. SCORE organisations felt that this was unacceptable.'
The 2012 SCORE report and Nuffield Report showed serious problems with the mathematical content of A Levels. SCORE was very critical:
'For biology, chemistry and physics, it was felt there were underpinning areas of mathematics missing from the requirements and that their exclusion meant students were not adequately prepared for progression in that subject. For example, for physics many of the respondents highlighted the absence of calculus, differentiation and integration, in chemistry the absence of calculus and in biology, converting between different units… For biology, chemistry and physics, the analysis showed that the mathematical requirements that were assessed concentrated on a small number of areas (e.g. numerical manipulation) while many other areas were assessed in a limited way, or not at all… Survey respondents were asked to identify content areas from the mathematical requirements that should feature highly in assessments. In most cases, the biology, chemistry and physics respondents identified mathematical content areas that were hardly or not at all assessed by the awarding organisations.
'[T]he inclusion of more in-depth problem solving would allow students to apply their knowledge and understanding in unstructured problems and would increase their fluency in mathematics within a science context.'
'The current mathematical assessments in science A-levels do not accurately reflect the mathematical requirements of the sciences. The findings show that a large number of mathematical requirements listed in the biology, chemistry and physics specifications are assessed in a limited way or not at all within these papers. The mathematical requirements that are assessed are covered repeatedly and often at a lower level of difficulty than required for progression into higher education and employment. It has also highlighted a disparity between awarding organisations in their assessment of the use of mathematics within biology, chemistry and physics A-level. This is unacceptable and the examination system, regardless of the number of awarding organisations, must ensure the assessments provide an authentic representation of the subject and equip all students with the necessary skills to progress in the sciences.
'This is likely to have an impact on the way that the subjects are taught and therefore on students' ability to progress effectively to STEM higher education and employment.' SCORE, 2012. Emphasis added.
The 2011 Institute of Physics report showed strong criticism from university academics of the state of physics and engineering undergraduates' mathematical knowledge. Four-fifth of academics said that university courses had changed to deal with a lack of mathematical fluency and 92% said that a lack of mathematical fluency was a major obstacle.
'The responses focused around mathematical content having to be diluted, or introduced more slowly, which subsequently impacts on both the depth of understanding of students, and the amount of material/topics that can be covered throughout the course…
'Academics perceived a lack of crossover between mathematics and physics at A-level, which was felt to not only leave students unprepared for the amount of mathematics in physics, but also led to them not applying their mathematical knowledge to their learning of physics and engineering.' IOP, 2011.
The 2011 Centre for Bioscience criticised Biology and Chemistry A Levels and preparation of pupils for bioscience degrees: 'very many lack even the basics… [M]any students do not begin to attempt quantitative problems and this applies equally to those with A level maths as it does to those with C at GCSE. A lack of mathematics content in A level Biology means that students do not expect to encounter maths at undergraduate level. There needs to be a more significant mathematical component in A level biology and chemistry.' The Royal Society of Chemistry report, The five decade challenge (2008), said there had been 'catastrophic slippage in school science standards' and that Government claims about improving GCSE scores were 'an illusion'. (The Department said of the RSC report, 'Standards in science have improved year on year thanks to 10 years of sustained investment and improvement in teaching and the education system – this is something we should celebrate, not criticise. Times have changed.')
Ofqual, 2012. Ofqual's Standards Review in 2012 found grade inflation in both GCSE and A-levels between 2001-03 and 2008-10: 'Many of these reviews raise concerns about the maintenance of standards… In the GCSEs we reviewed (biology, chemistry and mathematics) we found that changes to the structure of the assessments, rather than changes to the content, reduced the demand of some qualifications.'
On A-levels, 'In general we found that changes to the way the content was assessed had an impact on demand, in many cases reducing it. In two of the reviews (biology and chemistry) the specifications were the same for both years. We found that the demand in 2008 was lower than in 2003, usually because the structure of the assessments had changed. Often there were more short answer, structured questions' (Ofqual, Standards Reviews – A Summary, 1 May 2012, found here).
Chief Executive of Ofqual, Glenys Stacey, has said: 'If you look at the history, we have seen persistent grade inflation for these key qualifications for at least a decade… The grade inflation we have seen is virtually impossible to justify and it has done more than anything, in my view, to undermine confidence in the value of those qualifications' (Sunday Telegraph, 28 April 2012).
The OECD's International Survey of Adult Skills (October 2013). This assessed numeracy, literacy and computing skills of 16-24-year-olds. The tests were done over 2011/2012. England was 22nd out of 24 for literacy, 21st out of 24 for numeracy, and is 16th out of 20 for 'problem solving in a technology-rich environment'.
PISA 2012. The normal school PISA tests taken in 2012 (reported 2013) showed no significant change between 2009-12. England was 21st for science, 23rd for reading, and 26th for mathematics. A 2011 OECD report concluded: 'Official test scores and grades in England show systematically and significantly better performance than international and independent tests… [Official results] show significant increases in quality over time, while the measures based on cognitive tests not used for grading show declines or minimal improvements' (OECD Economic Surveys: United Kingdom, 16 March 2011, p. 88-89). This interesting chart shows that in the PISA maths test the children of English professionals perform the same as children of Singapore cleaners (Do parents' occupations have an impact on student performance?, PISA 2014).
Chart 6: Comparing pupil maths scores by parent occupation, UK (left) and Singapore (right) maths skills (PISA 2012)
TIMMS/PIRLS. The TIMMS/PIRLS tests (taken summer 2011, reported December 2012) told a similar story to PISA. England's score in reading at age 10 increased since 2006 by a statistically significant amount. England's score in science at age 10 decreased since 2007 by a statistically significant amount. England's scores in science at age 14 and mathematics at ages 10 and 14 showed no statistically significant changes since 2007. (According to experts, the PISA maths test relies more on language comprehension than TIMMS which is supposedly why Finland scores higher in the former than the latter.)
National Numeracy (February 2012). Research showed that in 2011 only a fifth of the adult population had mathematical skills equivalent to a 'C' in GCSE, down a few percent from the last survey in 2003. About half of 16-65 year olds have at best the mathematical skills of an 11 year-old. A fifth of adults will struggle with understanding price labels on food and half 'may not be able to check the pay and deductions on a wage slip.'
King's College, 2009. A major study by academics from King's College London and Durham University found that basic skills in maths have declined since the 1970s. In 2008, less than a fifth of 14 year-olds could write 11/10 as a decimal. In the early 1980s, only 22 per cent of pupils obtained a GCE O-level grade C or above in maths. In 2008, over 55 per cent gained a GCSE grade C or above in the subject (King's College London/University of Durham, 'Secondary students' understanding of mathematics 30 years on', 5 September 2009, found here).
Chart 7: Performance on ICCAMS / CSMS Maths tests showing declines over time
Shayer et al (2007) found that performance in a test of basic scientific concepts fell significantly between 1976 and 2003. '[A]lthough both boys and girls have shown great drops in performance, the relative drop is greater for boys… It makes it difficult to believe in the validity of the year on year improvements reported nationally on Key Stage 3 NCTs in science and mathematics: if children are entering secondary from primary school less and less equipped with the necessary mental conditions for processing science and mathematics concepts it seems unlikely that the next 2.5 years KS3 teaching will have improved so much as more than to compensate for what students of today lack in comparison with 1976.'
Chart 8: Performance on tests of scientific concepts, 1976 – 2003 (Shayer)
Tymms (2007) reviewed assessment evidence in mathematics from children at the end of primary school between 1978 and 2004 and in reading between 1948 and 2004. The conclusion was that standards in both subjects 'have remained fairly constant'.
Warner (2013) on physics. Professor Mark Warner (Cambridge University) produced a fascinating report (2013) on problems with GCSE and A Level Physics and compared the papers to old O Levels, A Levels, 'S' Level papers, Oxbridge entry exams, international exams and so on. After reading it, there is no room for doubt. The standards demanded in GCSEs and A Levels have fallen very significantly.
'[In modern papers] small steps are spelt out so that not more than one thing needs to be addressed before the candidate is set firmly on the right path again. Nearly all effort is spent injecting numbers into formulae that at most require GCSE-level rearrangements… All diagrams are provided… 1986 O-level … [is] certainly more difficult than the AS sample… 1988 A-level … [is] harder than most Cambridge entrance questions currently… 1983 Common Entrance [is] remarkably demanding for this age group, approaching the challenge of current AS… There is a staggering difference in the demands put on candidates… Exams [from the 1980s] much lower down the school system are in effect more difficult than exams given now in the penultimate years [i.e. AS].'
For example, the mechanics problems in GCSE Physics are substantially shallower than those in 1980s O Level, which examined concepts now in A Level. The removal of calculus from A Level physics badly undermined it. Calculus is tested in A Level Maths' Mechanics I paper and Mechanics II and III test deeper material than Physics A Level. This is one of the reasons why Cambridge Physics department stopped requiring Physics A Level for entry and made clear that Further Maths A Level is acceptable instead (many say it is better preparation for university than physics A Level is).
Warner also makes the point that making Physics GCSE and A Level much easier did not even increase the number taking physics degrees, which has declined sharply since the mid-1980s. He concludes: 'one could again aim for a school system to get a sizable fraction of pupils to manage exams of these [older] standards. Children are not intrinsically unable to attack such problems.' (NB. The version of this report on the web is not the full version – I would urge those interested to email Professor Warner.)
Gowers (2012) on maths. Tim Gowers, Cambridge professor and Fields Medallist, described some problems with Maths A Level and concluded:
'The general point here is of course that A-levels have got easier [emphasis added] and schools have a natural tendency to teach to the test. If just one of those were true, it would be far less of a problem. I would have nothing against an easy A-level if people who were clever enough were given a much deeper understanding than the exam strictly required (though as I've argued above, for many people teaching to the test is misguided even on its own terms, since they will do a lot better on the exam if they have not been confined to what's on the test), and I would not be too against teaching to the test if the test was hard enough…
'[S]ome exams, such as GCSE maths, are very very easy for some people, such as anybody who ends up reading mathematics at Cambridge (but not just those people by any means). I therefore think that the way to teach people in top sets at schools is not to work towards those exams but just to teach them maths at the pace they can manage.'
Durham University analysis gives data to quantify this conclusion. Pupils who would have received a U (unclassified) in Maths A-Level in 1988 received a B/C in 2006 – see above for Chart 5 showing this (CEM Centre Durham University, Changes in standards at GCSE and A-Level: Evidence from ALIS and YELLIS, April 2007). Further Maths A Level is supposedly the toughest A Level and probably it is but a) it is not the same as its 1980s ancestor and b) it now introduces pupils to material such as matrices that used to be taught in good prep schools.
I spent a lot of time 2007-14 talking to maths dons, including heads of departments, across England. The reason I quote Gowers is that I never heard anybody dispute his conclusion but he was almost the only one who would say it publicly. I heard essentially the same litany about A Level maths from everybody I spoke to: although there were differences of emphasis, nobody disputed these basic propositions. 1) The questions became much more structured so pupils are led up a scaffolding with less requirement for independent problem-solving. 2) The emphasis moved to memorising some basic techniques the choice of which is clearly signalled in the question. 3) The modular system a) encouraged a 'memorise, regurgitate, forget' mentality and b) undermined learning about how different topics connect across maths, both of which are bad preparation for further studies. (There are also some advantages to a modular system that I will return to.) 4) Many undergraduates, including even those in the top 5% at such prestigious universities as Imperial, therefore now struggle in their first year as they are not well-prepared by A Level for the sort of problems they are given in undergraduate study. (The maths department at Imperial became so sick of A Level's failings that they recently sought and got approval to buy Oxford's entrance exam for use in their admission system.)
I will not go into arguments about vocational qualifications here but note the conclusion of Alison Wolf whose 2011 report on this was not disputed by any of the three main parties:
'The staple offer for between a quarter and a third of the post- 16 cohort is a diet of low-level vocational qualifications, most of which have little to no labour market value.'
3. Knock-on effects in universities
Serious lack of maths skills
There are many serious problems with maths skills. Part of the reason is that many universities do not even demand A Level maths. The result? As of about 2010-12, about 20% of Engineering undergraduates, about 40% of Chemistry and Economics undergraduates, and about 60-70% of Biology and Computer Science undergraduates did not have A Level Maths. Less than 10% of undergraduate bioscience degree courses demand A Level Maths therefore 'problems with basic numeracy are evident and this reflects the fact that many students have grades less than A at GCSE Maths. These students are unlikely to be able to carry out many of the basic mathematical approaches, for example unable to manipulate scientific notation with negative powers so commonly used in biology' (2011 Biosciences report). (I think that history undergraduates should be able to manipulate scientific notation with negative powers – this is one of the many things that should be standard for reasonably able people.)
The Royal Society estimated (Mathematical Needs, 2012) that about 300,000 per year need a post-GCSE Maths course but only ~100,000 do one. (This may change thanks to Core Maths starting in 2015, see later blog.) This House of Lords report (2012) on Higher Education in STEM subjects concluded: 'We are concerned that … the level at which the subject [maths] is taught does not meet the requirements needed to study STEM subjects at undergraduate level… [W]e urge HEIs to introduce more demanding maths requirement for admissions into STEM courses as the lack, or low level, of maths requirements at entry acts as a disincentive for pupils to study maths and high level maths at A level.' House of Lords Select Committee on Science and Technology, Higher Education in STEM subjects, 2012.
Further, though this subject is beyond the scope of this blog, it is also important that the maths PhD pipeline 'which was already badly malfunctioning has been seriously damaged by EPSRC decisions', including withdrawal of funding from non-statistics subjects which drew the ire of UK Fields Medallists, cf. Submission by the Council for the Mathematical Sciences to the House of Lords, 2011. The weaknesses in biology also feed into the bioscience pipeline: only six percent of bioscience academics think their graduates are well prepared for a masters in the fast-growing field of Computational Biology (p.8 of report).
Closing of language departments, decline of language skills
I have not found official stats for this but according to research done for the Guardian (with FOIs):
'The number of universities offering degrees in the worst affected subject, German, has halved over the past 15 years. There are 40% fewer institutions where it is possible to study French on its own or with another language, while Italian is down 23% and Spanish is down 22%.'
As Katrin Kohl, professor of German at Jesus College (Oxford) has said, 'The UK has in recent years been systematically squandering its already poor linguistic resources.' Dawn Marley, senior lecturer in French at the University of Surrey, summarised problems across languages:
'We regularly see high-achieving A-level students who have only a minimal knowledge of the country or countries where the language of study is spoken, or who have limited understanding of how the language works. Students often have little knowledge of key elements in a country's history – such as the French Revolution, or the fact that France is a republic. They also continue to struggle with grammatical accuracy, and use English structures when writing in the language they are studying… The proposals for the revival of A-level are directly in line with what most, if not all, academics in language departments would see as essential.' (Emphasis added.)
The same picture applies to classical languages. Already by 1994 the Oxford Classics department was removing texts such as Thucydides as compulsory elements in 'Greats' because they were deemed 'too hard'. These changes continued and have made Classics a very different subject than it was before 1990. At Oxford, they introduced whole new courses (Mods B then Mods C) that do not require any prior study of the ancient languages themselves. The first year of Greats now involves remedial language courses.
I quote at length from a paper by John Davie, a Lecturer in Classics at Trinity College, Oxford, as his comments summarise the views of other senior classicists in Oxbridge and elsewhere who have been reluctant to speak out (In Pursuit of Excellence, Davie, 2013). Inevitably, the problems described are damaging the pipeline for masters, PhDs, and future scholarship.
'Classics as an academic subject has lost much of its intellectual force in recent years. This is true not only of schools but also, inevitably, of universities, which are increasingly required to adapt to the lowering of standards…
'In modernist courses…, there is (deliberately) no systematic learning of grammar or syntax, and emphasis is laid on fast reading of a dramatic continuous story in made-up Latin which gives scope for looking at aspects of ancient life. The principle of osmosis underlying this approach, whereby children will learn linguistic forms by constant exposure to them, aroused scepticism among many teachers and has been thoroughly discredited by experts in linguistics. Grammar and syntax learned in this piecemeal fashion give pupils no sense of structure and, crucially, deny them practice in logical analysis, a fundamental skill provided by Classics…
'[W]e have, in GCSE, an exam that insults the intelligence… Recent changes to this exam have by general consent among teachers made the papers even easier.
'In the AS exam currently taken at the end of the first year of A-level … students study two small passages of literature, which represent barely a third of an original text. They are asked questions so straightforward as to verge on the banal and the emphasis is on following a prescribed technique of answering, as at GCSE. Imagination and independent thought are simply squeezed out of this process as teachers practise exam-answering technique in accordance with the narrow criteria imposed on examiners.
'The level of difficulty [in AS] is not substantially higher than that of GCSE, and yet this is the exam whose grades and marks are consulted by the universities when they are trying to determine the ability of candidates… Having learned the translation of these bite-sized chunks of literature with little awareness of their context or the wider picture (as at GCSE, it is increasingly the case that pupils are incapable of working out the Latin/Greek text for themselves, and so lean heavily on a supplied translation), they approach the university interview with little or no ability to think "outside the box". Dons at Oxford and Cambridge regularly encounter a lack of independent thought and a tendency to fall back on generalisations that betray insufficient background reading or even basic curiosity about the subject. This need not be the case and is clearly the product of setting the bar too low for these young people at school…
'At A2 … students read less than a third of a literary text they would formerly have read in its entirety.
'There is the added problem that young teachers entering the profession are themselves products of the modernist approach and so not wholly in command of the classical languages themselves. As a result they welcome the fact that they are not required by the present system to give their pupils a thorough grounding in the language, embracing the less rigorous approach of modern course-books with some relief.
'In the majority of British universities Classics in its traditional form has either disappeared altogether or has been replaced by a course which presents the literature, history and philosophy mainly (or entirely) in translation, i.e. less a degree course in Classics than in Classical Civilisation.
'This situation has been forced upon university departments of Classics by the impoverished language skills of young people coming up from schools… It is not only the classical languages but English itself which has suffered in this way in the last few decades. Every university teacher of the classical languages knows that he cannot assume familiarity with the grammar and syntax of English itself, and that he will have to teach from scratch such concepts as an indirect object, punctuation or how a participle differs from a gerund…
'Even at Oxford cuts have been made to the number of texts students are required to read and, in those texts that remain, not as many lines are prescribed for reading in the original Latin or Greek.
'In the last ten years of teaching for Mods [at Oxford] I have been struck by how the first-year students who come my way at the start of the summer term appear to know less about the classical languages each year, an experience I know to be shared by dons at other colleges…
'GCSE should be replaced by a modern version of the O-level that stretches pupils… This would make the present AS exam completely unsuitable, and either a more challenging set of papers should be devised, if the universities wish to continue with pre A-level interviewing, or there should be a return to an unexamined year of wide reading before the specialisation of the last year.
'Although the present exam, A2, has more to recommend it than AS, it also would no longer be fit for purpose and would need strengthening. As part of both final years there should be regular practice in the writing of essays, a skill that has been largely lost in recent years because of the exam system and is (rightly) much missed by dons.'
This combination of problems explains why we funded a project with Professor Pelling, Regius Professor of Greek at Oxford, to fund teacher training and language enrichment courses for schools.
I will not go into other humanities subjects. I read Ancient & Modern History and have thoughts about it but I do not know of any good evidence similar to the reports quoted above by the likes of the Royal Society. I have spoken to many university teachers. Some, such as Professor Richard Evans (Cambridge) told me they think the standard of those who arrive as undergraduates is roughly the same as twenty years ago. Others at Oxbridge and elsewhere told me they think that essay writing skills have deteriorated because of changes to A Level (disputed by Evans and others) and that language skills among historians have deteriorated (undisputed by anyone I spoke to).
For example, the Cambridge Professor of Mediterranean History, David Abulafia, has contradicted Evans and, like classicists, pointed out the spread of remedial classes at Cambridge:
'It's a pity, then, that the director of admissions at Cambridge has proclaimed that the old system [pre-Gove reforms] is good and that AS-levels – a disaster in so many ways – are a good thing because somehow they promote access. I don't know for whom he is speaking, but not for me as a professor in the same university…
'[Gove] was quite right about the abolition of the time-wasting, badly devised and all too often incompetently marked AS Levels; these dreary exams have increasingly been used as the key to admissions to Cambridge, to the detriment of intellectually lively, quirky, candidates full of fizz and sparkle who actually have something to say for themselves…
'Bogus educational theories have done so much to damage education in this country… The effects are visible even in a great university such as Cambridge, with a steady decline in standards of literacy, and with, in consequence, the provision in one college after another of 'skills teaching', so that students who no longer arrive knowing how to structure an essay or even read a book can receive appropriate 'training'… Even students from top ranked schools seem to find it very difficult … to write essays coherently… In the sort of exams I am thinking of, essay writing comes much more to the fore and examiners would be making more subjective judgements about scripts. In an ideal world there would be double marking of scripts.' Emphasis added.
Judging essay skills is a more nebulous task than judging the quality of mechanics questions. Also, there is less agreement among historians about the sort of things they want to see in school exams compared to mathematicians and physicists who largely (in my experience, I stress, which is limited) agree about the sorts of problems they want undergraduates to be able to solve and the skills they want them to have.
I will quote a Professor of English at Exeter University, Colin MacCabe, whose view of the decline of essay skills is representative of many comments I have heard, but I cannot say confidently that this view represents a consensus, despite his claim:
'Nobody who teaches A-level or has anything to do with teaching first-year university students has any doubt that A Levels have been dumbed down… The writing of the essay has been the key intellectual form in undergraduate education for more than a century; excelling at A-level meant excelling in this form. All that went by the board when … David Blunkett, brought in AS-levels… A-levels … became two years of continuous assessment with students often taking their first module within three months of entering the sixth form. This huge increase in testing went together with a drastic change in assessment. Candidates were not now marked in relation to an overall view of their ability to mount and develop arguments, but in relation to their ability to demonstrate achievement against tightly defined assessment objectives… A-levels, once a test of general intellectual ability in relation to a particular subject, are now a tightly supervised procession through a series of targets. Assessment doesn't come at the end of the course – it is the course… In English, students read many fewer books… Students now arrive at university without the knowledge or skills considered automatic in our day… One of the results of the changes at A-level is that the undergraduate degree is itself a much more targeted affair. Students lack of a general education mean that special subjects, dissertations etc are added to general courses which are themselves much more limited in their approach… One result of this is a grade inflation much more dramatic even than A-levels… [T]here is little place within a modern English university for students to develop the kind of intellectual independence and judgment, which has historically been the aim of the undergraduate degree.' Observer, 22 August, 2004. (Emphasis added.)
If anybody knows of studies on history and other humanities please link in Comments below.
Oxbridge entrance
As political arguments increasingly focused on 'participation' and 'access', Oxford and Cambridge largely abandoned their own entrance exams in the 1990s. There were some oddities. Cambridge University dropped their maths test and were so worried by the results that they immediately asked for and were given special dispensation to reintroduce it and they have used one since (now known as the STEP paper, used by a few other universities). Other Cambridge departments who wanted to do the same were refused permission and some of them (including the physics department) now use interviews to test material they would like to test in a written exam. Oxford changed its mind and gradually reintroduced admission tests in some subjects. (E.g. It does not use STEP in maths but uses its own test which has more 'applied' maths.) Cambridge now uses AS Levels. Oxford does not (but does not like to explain why).
A Levels are largely useless for distinguishing between candidates in the top 2% of ability (i.e. two standard deviations above average). Oxbridge entry now involves a complex and incoherent set of procedures. Some departments use interviews to test skills that are i) either wholly or entirely untested by A Levels and ii) are not explicitly set out anywhere. For example, if you go to an interview for physics at Cambridge, they will ask you questions like 'how many photons hit your eye per second from Alpha Centauri?' – i.e. questions that you cannot cram for but from which much information can be gained by tutors watching how students grapple with the problem.
The fact that the real skills they want to test are asked about in interviews rather than in public exams is, in my opinion, not only bad for 'standards' but is also unfair. Rich schools with long connections to Oxbridge colleges have teachers who understand these interviews and know how to prepare pupils for them. They still teach the material tested in old exams and other materials such as Russian textbooks created decades ago. A comprehensive in east Durham that has never sent anybody to Oxbridge is very unlikely to have the same sort of expertise and is much more likely to operate on the very mistaken assumption that getting a pupil to three As is sufficient preparation for Oxbridge selection. Testing skills in open exams that everybody can see would be fairer.
I will return to this issue in a later blog but it is important to consider the oddities of this situation. Decades ago, open public standardised tests were seen as a way to overcome prejudice. For example, Ivy League universities like Harvard infamously biased their admissions system against Jews because a fair open process based on intellectual abilities, and ignoring things like lacrosse skills, would have put more Jews into Harvard than Harvard wanted. Similar bias is widespread now in order to keep the number of East Asians low. It is no coincidence that Caltech's admissions policy is unusually based on academic ability and it has a far higher proportion of East Asians than the likes of Harvard.
Similar problems apply to Oxbridge. A consequence of making exams easier and removing Oxbridge admissions tests was to make the process more opaque and therefore biased against poorer families. The fascinating journey made by the intellectual Left on the issue of standardised tests is described in Steven Pinker's recent influential essay on university admissions. I agree with him that a big part of the reason for the 'madness' is that the intelligentsia 'has lost the ability to think straight about objective tests'. Half a century ago, the Left fought for standardised tests to overcome prejudice, now many on the Left oppose tests and argue for criteria that give the well-connected middle classes unfair advantages.
This combination of problems is one of the reasons why the Cambridge pure maths department and physics department worked with me to develop projects to redo 16-18 curricula, teacher training, and testing systems. Cambridge is even experimenting with a 'correspondence Free School' idea proposed by the mathematician Alexander Borovik (who attended one of the famous Russian maths schools). Powerful forces tried to stop these projects happening because they are, obviously, implicit condemnations of the existing system – condemnations that many would prefer had never seen the light of day. Similar projects in other departments at other universities were kiboshed for the same reason, as were other proposals for specialist maths schools as per the King's project (which also would never have happened but for the determination of Alison Wolf and a handful of heroic officials in the DfE). I will return to this too.
C. Conclusions
Here are some tentative conclusions.
The political and bureaucratic process for the introduction of the GCSE and National Curriculum was a shambles. Those involved did not go through basic processes to agree aims. Implementation was awful. All elements of the system failed children. There are important lessons for those who want to reform the current system.
Given the weight of evidence above, it is hard to avoid the conclusion that GCSEs were made easier than O Levels and became easier still over time. This means that at least the top fifth are aimed aged 14 at lower standards than they would have been aimed at previously (not that O Levels were at all optimal). Many of them spend two years with low grade material and repeating boring drills, in order that the school can maximise its league table position, instead of delving deeper into subjects. Inflation seems to have stopped in the last two years, perhaps temporarily, but by the use of an Ofqual system known as 'comparable outcomes' which is barely understood by anybody in the school system or DfE.
A Levels, at least in maths, sciences, and languages, were quickly made easier after 1988 and not just by enough to keep pass marks stable but by enough to lead to large increases. Even A Level students are aimed at mundane tasks like 'design a poster' that are suitable for small children – not near-adults. (As I type this I am looking at an Edexcel textbook for Further Maths A Level which for some reason, Edexcel has chosen to decorate with the picture of a child in a 'Robin' masked outfit.)
The old 'S' level papers, designed to stretch the best A Level students, were abandoned which contributed to a decline of standards aimed for among the top 5%.
University degrees in some subjects therefore also had to become easier (e.g. classics) or longer (natural sciences) in order to avoid increases in failure rates. This happened in some subjects even in elite universities. Remedial courses spread, even in elite universities, to teach/improve skills that were previously expected on arrival (including Classics at Oxford and History at Cambridge). Not all of the problems are because of failures in schools or easier exams. Some are because universities themselves for political reasons will not make certain requirements of applicants. Even if the exam system were fixed, this would remain a big problem. On the other hand, while publicly speaking out for AS Levels, admissions officers also, very quietly, have been gradually introducing new, non-Government/Ofqual regulated, tests for admissions purposes. On this, it is more useful to watch what universities do than what they say.
These problems have cascaded right through the system and now affect the pipeline into senior university research positions in maths, sciences, and languages. For example, the lack of maths skills among biologists is hampering the development of synthetic biology and computational biology. It is very common now to have (private) discussions with scientists deploring the decline in English research universities. Just in the past few weeks I have had emails from an English physicist now at Harvard and a prominent English neuroscientist giving me details of these developments and how we are falling further behind American universities. As they say, however, nobody wants to speak out.
It is much easier to see what has happened at the top end of the ability curve, where effects show up in universities, than it is for median pupils. The media also focuses on issues at the top end of the ability curve, A Levels, and the Russell Group.
Because politicians took control of the system and used results to justify their own policies, and because they control funding, debate over standards became thoroughly dishonest, starting with the Conservative government in the 1980s and continuing to now when academics are pressured not to speak out by administrators for fear of politicians' responses. When governments are in control of the metrics according to which they are judged, there is likely to be dishonesty. If people – including unions, teachers, and officials – claim they deserve more money on the basis of metrics that are controlled by a small group of people operating an opaque process and controlling the regulator themselves, there is likely to be dishonesty.
An important caveat. It is possible that simultaneously a) 1-8 is true and b) the school system has improved in various ways. What do I mean?
This is a coherent (not necessarily right) conclusion from the story told above…
GCSEs are significantly easier than O Levels. Nevertheless, the switch to GCSEs also involved many comprehensives and secondary moderns dropping the old idea that maybe only a fifth of the cohort are 'academic' – the idea from Plato's Republic of gold, silver, and bronze children, that influenced the 1944 Act. Instead, more schools began to focus more pupils on academic subjects. Even though the standards demanded were easier than in the pre-1988 exams, this new focus (combined with other things) at least led between 1988 and now to a) a reduction in the number of truly awful schools and b) more useful knowledge and skills at least for the bottom fifth of the cohort (in ability terms), and perhaps for more. Perhaps the education of median ability pupils stayed roughly the same (declining a bit in maths) hence the consistent picture in international tests, the King's results comparing maths in 1978/2008, Shayer's results and so on (above). Meanwhile the standards demanded by post-1988 A Levels clearly fell (at least in some vital subjects), as the changes in universities testify, and S Level papers vanished, so the top fifth of the cohort (and particularly the +2 standard deviation population, i.e. the top 2%) leave school in some subjects considerably worse educated than in the 1980s. (Given most scientific and technological breakthroughs come from among this top 2% this has a big knock-on effect.) Private schools felt incentivised to perform better than state schools on easier GCSEs and A Levels rather than pursue separate qualifications with all the accompanying problems. There remains no good scientific data on what children at different points on the ability curve are capable of achieving given excellent teaching so the discussion of 'standards' remains circular. Easier GCSEs and A Levels are consistent with some improvements for the bottom fifth, roughly stability for the median, significant decline for the top fifth, and fewer awful schools.
This is coherent. It fits the evidence sketched above.
But is it right?
In the next blog in this series I will consider issues of 'ability' and the circularity of the current debate on 'standards'.
If people accept the conclusions about GCSEs and A Levels (at least in maths, sciences, and languages, I stress again) how should this evidence be weighed against the very strong desire of many in the education system (and Parliament and Whitehall) to maintain a situation in which the vast majority of the cohort are aimed at GCSEs (or international equivalents that are not hugely different) and, for those deemed 'academic', A Levels?
Do the gains from this approach outweigh the losses for an unknown fraction of the 'more able'?
Is there a way to improve gains for all points on the ability distribution?
I have been told that there is no grade inflation in music exams. Is this true? If YES, is this partly because they are not regulated by the state? Are there other factors? Has A Level Music got easier? If not why not?
What sort of approaches should be experimented with instead of the standard approaches seen in O Levels, GCSEs, and A Levels?
What can be learned from non-Government regulated tests such as Force Concepts Tests (physics), university admissions tests, STEP, IQ tests and so on?
What are the best sources on 'S' Level papers and what happened with Oxbridge entrance exams?
What other evidence is there? Where are analyses similar to Warner's on physics for other subjects?
What evidence is there for university grade inflation which many tell me is now worse than GCSEs and A Levels?
Posted in Curriculum & Exams, Education, Learning & Teaching, UK Politics, Uncategorized	| Tagged Baker, Gove, Gowers, Hoskins, Joseph, Mark Warner, Ofqual, Ofsted, Teacher training, Thatcher | 20 Comments
Standards In English Schools Part 0: Introduction
'I think the educational and psychological studies I mentioned are examples of what I would like to call Cargo Cult Science. In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they've arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas – he's the controller – and they wait for the airplanes to land. They're doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn't work. No airplanes land. So I call these things Cargo Cult Science, because they follow all the apparent precepts and forms of scientific investigation, but they're missing something essential, because the planes don't land.' Richard Feynman's Caltech commencement address on Education and Cargo Cult Science (1974).
'Let's put behind us once and for all the old sterile debate about dumbing down. I want to end young people being told that the GCSE or A-level grades they are proud of aren't worth what they used to be.' Ed Balls to the Labour Party Conference, 2007.
'It is undeniable that the last Labour government dramatically improved school standards in secondary education.' Tristram Hunt, 26 January 2015.
'Despite the apparently plausible and widespread belief to the contrary, the evidence that levels of attainment in schools in England have systematically improved over the last 30 years is unconvincing. Much of what is claimed as school improvement is illusory… standards have not risen; teaching has not improved… The question, therefore, is not whether there has been grade inflation, but how much…' (Professor Robert Coe, 2013, here.)
This series of blogs will discuss: 1) what we know about standards in English schools including the effect of the introduction of the National Curriculum and GCSEs; 2) how 'ability' and 'standards' should be defined; 3) what can be learned from the 2010-15 reforms and what incentives now dominate the system; 4) what research and policy agenda is needed; 5) what materials are there for those interested in standards beyond those of the National Curriculum and state controlled exams.
The debate about 'standards in English schools' is obviously of great importance but it suffers from many fundamental problems. Ironically for a debate that often involves the word 'rigour', the debate is itself unrigorous.
The main concepts are not properly defined. Politicians, policy people, officials, and journalists speak and write daily using phrases such as 'we must drive up standards so that [X% of schools or pupils] hit the standard of [Y]' when Y has no objective definition. Most obviously there has been enormous debate about grades in GCSEs and A Levels but these grades themselves are arbitrarily created according to criteria that would not impress physical scientists. The 'standards' are circular. Exams are regulated by the DfE and Ofqual in order that there is a very high chance that at least X% 'pass', then people say 'more than X% should pass', or 'X% is too tough'. But the X% is just based in the first place on where the system happens to be which is historically contingent – it is not based on any scientific judgement about what children of different abilities (rigorously defined) are capable of doing given certain teaching.
In the recent debate over reforming GCSEs, when we tried to drop their use in the accountability system in 2012, Nick Clegg insisted, and Cameron agreed, that the entire reform process be based on the principles that i) about 95% of the cohort should do the same exams at 16 and ii) not many more would fail to pass than now (2012). Definitions of a 'pass' were therefore set in order to fit with an a priori desire for a certain percentage to pass – a political desire of one party rather than an educational judgement. (The other two parties have had the same approach over the past thirty years – my point is not that the LibDems are particularly bad.)
Despite having a circular process for defining standards, it has been a central feature of education debates for politicians to set targets for what proportion of pupils every school must get to 'pass' – targets that have high stakes for school management and teachers. One can understand the motivation, given the bad effects for individuals of being in really bad schools, but the process as a whole does not make sense. Further, Ofqual imposes a system ('comparable outcomes') which is intended to combat grade inflation but which also seems to operate deliberately against the goal of significant rises in the proportion passing GCSEs. Further, Ofsted's reports add noise, not signal, given, as Professor Coe has said, 'its judgements have little scientific credibility' (some argue this is too generous).
Similarly, people in the education world use the word 'ability' but they almost never define or have an objective measure for 'ability'. The work of scientists on this subject has been almost entirely ignored and has had practically no effect on policy in England. Many teacher training colleges promote 'cargo cult' science on the subject of 'ability' to thousands of teachers who are therefore confident in views that are the opposite of what the science says.
As far as I am aware, there is no serious research agenda in English schools attempting to a) discover what pupils of different ability, using objective measures, are capable of achieving given certain teaching and b) use this knowledge to shape the curriculum, tests, and objective measures of school performance in an iterative feedback loop that can improve its accuracy over time.
The main point of these blogs is to help make the case for such a research programme (see below). Since first becoming involved in education debates in 2007, I have had many discussions about this. I have said to many people, including in the Royal Society, the home of British science, that we need a scientific approach to the issue of standards and ability. I wrote about it in my essay that became public in 2013. I argued for it in the DfE, with subject associations, with those responsible for teacher training ('the most bankrupt institution I know', said Hattie), and with many people who talk about 'research' and 'evidence'.
Few have wanted to engage in this subject because it is so politically fraught. Even fewer have done so publicly and I have personal experience of severe pressure put on many academics by university administrators not to tell the truth. However, a very positive development in English education is the growth of support for thinking seriously about evidence. In the DfE, there was a long battle on this issue that ended suddenly when the new Permanent Secretary arrived and immediately agreed with the appointment of Ben Goldacre to do a review of the Department's handling of evidence, research, and data, which was published in 2013. (I have written many critical things about officials, such as HERE, so it is worth noting that Wormald, and other officials particularly younger ones, took this enlightened view.) There is no doubt that the culture inside the Department changed as a result though there is a very long way to go in this area and it is reasonable to be doubtful about any of the three parties' commitment to this approach and about civil service commitment. Tom Bennett's efforts with ResearchEd have been fantastic and are one of the most hopeful things I've seen since 2007. There is also now a discussion about a possible College of Teachers – an institution that will only be credible if it has high standards on the subject of cargo cult research. Unsurprisingly, therefore, more people are starting to ask: what do we know about standards? (E.g. Sam Freedman recently blogged on it.)
I therefore thought I would jot down in a series of blogs various bits of evidence, history, thoughts, discussions and so on that I have accumulated since 2007.
Five broad areas
This series of blogs will consider inter alia these questions grouped in five rough areas (which may change as I go along).
A. What is the evidence concerning 'standards in English schools'? What was the effect of the introduction of 1) GCSEs and 2) the National Curriculum with its connected testing regime? What were the cascading effects on A Levels and higher education? What do comparisons with international tests and other academic studies tell us? What do subject associations and organisations such as the Royal Society say? What do universities and subject experts say?
B How should 'ability' and 'standards' be defined? What undermines sensible discussion about this?
C. What was Gove's team trying to do 2010-14? How effective were reforms concerning the curriculum, exams, and accountability (including the role of Ofsted)? What lessons might be learned from the period 2010-15? What incentives dominate the system now?
D. What should come next? What can we reasonably infer from the period since 1985 about what is very unlikely to work? What should the parties not put in their manifestos? What are the main reasons why political and policy discussion of this subject has been so controversial? How does the transformation of the technological landscape since the mid-1980s change arguments? How could a focus on evidence and empiricism help improve the system?
E. What materials are there that can be used by schools that are focused more on education and learning than the official accountability system?
The goal of these blogs is not to 'defend the Gove reforms'. When I get onto them, I will try to explain as clearly as I can why we tried to do certain things and what went wrong. GCSE reform (along with the disaster of Ofsted) is arguably the biggest failure of our team and therefore particularly needs analysis. The goal is not to affect party manifestos – it is possible but unlikely that someone reading this may be able to nudge things off a party or bureaucratic agenda. It is reasonable to assume that whatever the parties promise their plans will crumble on contact with reality. My main hope is that people outside SW1 at the coalface of education take matters into their own hands and develop their own approaches to scientific experimentation with the curriculum, exams, and training.
In my opinion, the only real hope for large improvements in learning is if 1) a critical mass of people become convinced of the need for an empirical approach and the rejection of 'cargo cult science' that has dominated education, and 2) an empirical programme emerges that iteratively a) tests what children of different abilities can learn and b) uses this information to alter curricula, tests, and teacher training. We need experiments and Grand Prizes in education that have brought dramatic breakthroughs in other areas, such as DARPA's Grand Challenge that led to breakthroughs in basic science and then to driverless cars. Imagine what well-defined Grand Challenges could bring to English schools.
Improvements in education do not need to be justified as goals with reference to other things such as economic growth. Learning and education are fundamental aspects of being human. However, it is obvious that humans will have to grapple with profound challenges over the next thirty years. The population will grow by another few billion, mainly in cities and connected to the mobile internet and 'the internet of things'. Energy and other resource demands will put the global system under huge pressure. We face old security threats like nuclear weapons and new threats such as the use of genetic engineering techniques empowering garage bio-hackers, for good and evil. For example, the revolutionary genome 'cut and paste' engineering tool, CRISPR, may soon be used to 'de-extinct' species and eradicate diseases but the same techniques could be used destructively. Much progress in machine intelligence and robotics is being driven by research controlled by militaries and intelligence agencies but little research is done on the profound dangers.
If we are to cope with these things, we will need new technologies, new institutions, and new ideas. Improving our education system is therefore obviously central. I have proposed that it ought to become the central organising principle for the British state, as an answer to Dean Acheson's famous quip that Britain had failed to find a post-imperial role.
Hopefully the discussion of standards in English schools will be useful regardless of whether you agree with this broader argument or not.
Please leave comments, corrections, research reports, complaints etc below. I will add things people leave as I go along and at the end try to produce something short and rigorous…
Posted in Education, UK Politics	| Tagged Goldacre, Gove, Ofsted, Wormald | 9 Comments
International GCSEs and the DfE ban – was there a better path?
It is reported that the DfE has decided not to allow international GCSEs to be allowed to be used in league tables.
This is not a surprise though I think it is bad policy. I will explain some background, my involvement in discussions about this before I left in January 2014, and why I think it is a mistake.
None of the boards' iGCSEs counted in league tables pre-2010. We thought this was a mistake. Some of the best private schools used the Cambridge International iGCSE. Some great state schools told us in Opposition that they wanted to do the same. It seemed reasonable to have more diversity in the system and let state schools do what private schools were doing particularly given the huge problems with standard GCSEs and the difficulty with reforming them.
During 2013, as standard GCSEs were being reformed, the issue arose of what to do about iGCSEs.
The issue was complicated by differences among the boards. I have never heard anyone claim that the Cambridge International iGCSE is easier than the standard GCSE. However, there were persistent arguments that other boards' iGCSEs were not as hard as GCSEs and private schools and others were being conned.
There was a piece of research circulating (from people very well known in the education world who are taken very seriously) that plotted the Cambridge iGCSE against the standard GCSE and another board's iGCSE, with the former harder than GCSEs and the latter easier than GCSEs. The research was not published because the people concerned were frightened of legal action (itself a telling detail about the epidemic dishonesty in the English debate on exams and standards, a dishonesty that, I think, most who discuss education policy greatly underestimate).
In 2013, officials and Ofqual argued that the reformed GCSEs starting from 2015 should completely replace iGCSEs which should not be allowed in the league tables. MG and I were not keen on this idea. I spoke to people about their concerns. I suggested the following path through (the below is not a quote from my memo, which I will dig out, but includes the main ideas)…
The DfE should announce that anybody who wants their exams, including iGCSEs, to be included in league tables will have to produce clear overwhelming and independent evidence that they are significantly more challenging than standard GCSEs. If they produce such evidence, we will include them; if not, not. This means that we avoid banning things that are obviously better than GCSEs but we also cull those exam boards who are abusing the system. We will also learn from the evidence presented and that exercise will be a useful thing – even if none of the boards submit anything we will learn something valuable. We're trying to move all of these debates towards people discussing evidence rather than hunches and this is a very good candidate for this approach.
The responses boiled down to two things.
1) Nobody had an argument against the idea as policy.
2) Nobody wanted to do it. The bureaucratic arguments amounted to: a) 'It's messy.' b) 'We'll get sued by the exam boards who are always cheating things and will hire fancy lawyers.' c) 'Ofqual doesn't want to.' Why? 'It's messy.' d) Unstated but hanging over the discussion – 'it's a lot of hard work on a marginal issue and nobody is going to attack us for being elitist if we ban them'.
Given this response, MG and spads thought we should try it.
Some time in autumn 2013, I can't remember when, I was tipped off that 'David Laws hates your idea, he just wants to ban them, there's a meeting shortly'. I went to the meeting. I was not optimistic and assumed I would have to torpedo him with 'SoS agrees with me', given the state of relationships by then given Clegg's appalling behaviour. My assumption was wrong.
The issues were explained. I gave him my argument. He had not heard it. I said that the bureaucratic arguments were not relevant – particularly absurd fears about legal challenges (an argument deployed daily) – and the best thing educationally was to give it a whirl regardless of some complexity and irrelevant media noise. Laws listened and asked questions. He was reasonable. He asked officials if anybody had a policy argument against the idea. Nobody did, the main argument was 'Ofqual really wants us to ban them'. It was also clear though that the issue was not closed and officials knew I was soon leaving.
It is no surprise to see the news today. The bureaucracy now has its clarity, but is it a good decision? Were Nicky Morgan / her spads given an alternative (it would not surprise me if the option was never presented to them)? What will the DfE say when a state school says 'Eton does the Cambridge iGCSE in X because they think it's better – why can't we offer our pupils the same thing, as you promised before the 2010 election?'? Can anybody see a downside to trying the other path, given one could always have reverted to banning everything if it proved unworkable?
It is of course possible that detailed work was done after I left and this decision was taken for reasons that are not public but are sound. If so, I am sure the new evidence-based DfE will make the technical arguments public.
Please leave comments, corrections etc.
UPDATE 1. This story strengthens my view that one of the most important things for the improvement of education in English state schools is the development of new exams that are outside the regulatory structure of the DfE and Ofqual – exams that are aimed purely at encouraging deep skills in mathematical modelling, extended writing and so on. It is not a coincidence that perhaps the most challenging exam taken in English schools – maths STEP – a) is not created by the domestic exam boards (Cambridge Assessment, not OCR), b) has zero input from DfE, c) is not regulated by Ofqual, d) has a clear educational purpose of encouraging deep skills needed for a serious undergraduate degree, and e) the people who use it as a tool would be horrified at the idea of the DfE, Ofqual, or 'education policy people' proposing Whitehall should have anything to do with it. I will blog soon on how I think a new 'post-GCSE & A Level' system could evolve.
UPDATE 2. As some emails winging from the DfE say, not all officials wanted the ban and some agreed with the course suggested above. True. (I tend to assume readers of this blog will assume the DfE is not monolithic.) But it was also clear which way Whitehall's gravity was pulling.
UPDATE 3. An email arrives from inside the DfE – a senior official who was involved in these decisions… He points to this research on iGCSEs. The C/D borderline figures are the most interesting.
I want to stress – I am not saying that international GCSEs are 'the answer' to the problems with the exam system. I do not think they are. I think the problems are much more fundamental and require much deeper changes. My point is that it would have been much better policy to ask the boards for hard evidence about the exams in order that the policy world can examine the issues on the basis of data rather than hunches and just 'officials say they're easier' / 'well why does Eton do them then?' etc, which is the level of debate over the past decade. If the DfE made the decision on the basis largely of the evidence in the link above, then it should explain this publicly so people can judge whether their thought process was reasonable, otherwise inevitably many will assume the decision was made for bureaucratic – not educational – reasons.
Posted in Curriculum & Exams	| Tagged Gove, Ofqual | 13 Comments