An Unnecessary KIS ? The UK ’ s Key Information Set , was it Really needed and What was its Real Purpose ?

This paper provides a critical analysis of the United Kingdom’s higher education Key Information Set (KIS), which was implemented following the 2011 UK White Paper ‘Students at the Heart of the System’. It argues that one of the central tenets of the KIS – providing information that students within a free market can make an informed choice and, through this process of consumer choice improve the quality of teaching, is untenable because a central component of the KIS, the National Student Survey (NSS), is unreliable when used for comparing university courses. Further, it argues that the KIS reified a neoliberal perspective about the worth and value of higher education qualification, positioning it as a commodity of value only to the paying individual rather than being something of value to society as a whole. It will be of particular interest to academics and policy makers from outside of the United Kingdom, where governmental and regulatory agencies may be implementing similar policies.

suggest that it has become "in some quarters at least a generic term of deprecation describing almost any economic and political development deemed to be undesirable." While acknowledging that the term is contested knowledge, a more in-depth discussion of its meaning is beyond the parameters of this paper. See, for example, Springer et al. (2016), Saad-Filho and Johnston (2005), Harvey (2007), and Thorson and Lie (2010) for a comprehensive history and discussion of neoliberalism. For this paper neoliberalism is used as term that refers to various processes (including formal, legal, and informal) by government, quasi-governmental organizations and large business corporations to encourage and allow private sector organizations and a free-market economy mindset of both seeing and facilitating the market as being the best determinant of price, quality, and efficiency to both permeate the public accepted wisdom of common sense 'truth' and as an overarching philosophy for driving change in the deregulation of public-sector provision and practices to 'improve' quality, individual choice and 'empower' the consumer. The consumer here is a university student. The author sees neoliberalism as being both a dominant discourse and an embodied practice.

The United Kingdom's Key Information Set (KIS)
The KIS was introduced in June 2011 in the UK government's White Paper 'Students at the Heart of the System'. The White Paper had three main themes: financing students, improving the student experience (particularly the quality of teaching), and increasing social mobility. As Allen and Broadbent (2012. p. 183) identify As with all such publications, the White Paper is essentially a political document that serves, not just the purposes of funding system reform or improvement in the quality of the university sector, but also a particular agenda.
The particular agenda served was that of neoliberalism. There was a clear assumption in the White Paper that universities were not already delivering a high-quality student experience and that by further opening up higher education to market forces, there would be 'necessary' improvements. For example, it argued that "institutions must deliver a better student experience" (Department for Business Innovation and Skills 2011, p. 4). The White Paper's introduction of the KIS claimed that it would improve the quality of higher education provision through informed 'consumer-choice' by encouraging competition between universities (Davies, 2012). Student choice would be in future "drive competition" by putting "students in the driving seat" (Department for Business Innovation and Skills, 2011, p. 2), and renewing focus on "highquality teaching" (ibid. p. 2). The White Paper used a narrow and limited concept of 'competition' that, arguably, should not be applied to university degree programs. Holmwood and McGettigan (2011, p. 3) suggest that The White Paper's thinking is distorted and restricted because of the limited meaning it gives to its key concept, competition. It equates competition with the commercial price competition that exists between businesses selling the same simple and uniform product.
A university degree in one discipline is not simply that same as another degree in a different discipline from a different institution. Yet the White Paper clear underpinning message was that this was the case.
The White Paper articulated the government's agenda to respond to 'student demand' with the overall goal being "higher education that is more responsive to student choice, that provides a better student experience and that helps improve social mobility" and "a sector that is freed to respond in new ways to the needs of students" (ibid. p. 8 points 24 and p. 3 forewords). Despite there being no convincing evidence provided that the sector was unresponsive to student demand or that it needed to be freed up to respond in new ways. Through the mechanism of the KIS, the government intended to "radically improve and expand the information available to prospective students" (ibid. p. 6), yet there was no solid evidence offered to support the idea that students did not already have sufficient information available to them. The introduction of the KIS indicated an intention to provide prospective students with both a radically improved quality and amount of information about university courses to enable them to choose their 'best option.' The inherent assumption was that the existing mechanisms for providing information were inadequate, insufficient, and in need of radical improvement. This was despite their already being a plethora of information available to prospective undergraduates, including university websites, promotional materials such as institutional and departmental prospectuses, university open day events, the UCAS website, the Times newspaper University League Table, the Times Higher Education, and Guardian league tables, books such as 'The Good Universities Guide' and numerous school and local government careers service advisors. Over a decade before the publication of the White Paper Barr (2001) identified that university applicants already had a more than adequate amount of information in easyto-understand formats available to help them make their decision as to which higher education (HE) course and which higher education institution (HEI) to apply to. Why then did the UK government feel there was a need to "radically improve and expand the information available to prospective students" (Department for Business Innovation and Skills, 2011, p. 6)? Was there even a need to provide more information of any sort, never mind to 'radically improve' things? Was the KIS necessary?
The White Paper and KIS identified a potential student's 'best option' in deciding which university to attend, partly based on the price that universities would be allowed to charge for their degree programs. An inherent assumption being that price is a major determinant of quality, articulated as "Putting financial power into the hands of learners makes student choice meaningful" (ibid.p. 5). The discourse here served to exclude other concepts of 'best.' The rationale was that a prospective university applicant was able to make a meaningful choice based solely on cost or their perception of the 'bestcost,' 'best-value' and 'best-quality.' The question of in what way or how 'meaningful' to a prospective student is choice both about which university to study at and which course to study there based on some perceived combination of best-cost, value-formoney and quality are ignored. So, did the White Paper signify that by including price or value-for-money as an additional factor that learner choice was in some way now more meaningful? I would argue, as have others, (e.g., Hossler et al. 1999) that these choices, were already meaningful to prospective students regardless of cost or perceived value-formoney to prospective students.
When looking at the KIS proposal rapidly becomes apparent that to consider the KIS as policy, one has to consider the rest of the White Paper, and other relevant policies such as the 2005 National Student Survey (NSS) of student satisfaction, introduced as a way of helping potential students to make 'informed choice' about the 'best' university to study at (Ramsden et al. 2010) and the 2010 UK government Browne Review (Independent Review of Higher Education Funding and Student Finance). As Ball (2007) has argued, Education policy concepts need to be contextualized within the wider context of government action and a range of domestic and international economic and social policies. In a similar vein, Gale (1999, p.3) suggests that "discourses do not simply assign meanings to texts in isolation but weave them together to form contexts." To better understand the policy context of the White Paper, one needs also to consider the document published one year earlier, the 2010 Browne Review.
"The proposals in the Browne Review were premised on the idea that everything desirable about higher education could be achieved by market forces, including…[students] choice subject" (Allen and Broadbent, 2012, p. 180). It introduced a range of changes within UK higher education, including a raising of the cap on student fees, with its main themes being improving participation rates (widening participation), improving the quality of the higher education system and improving affordability for both students and the state (Morgan 2009). It saw the government seeking "to emphasize the personal benefits of higher education as a justification for the continuation and increase in tuition fees and introduced a new, higher, tuition fees cap of £9000" (ibid. p. 181-182). The Browne Review identified that consumer-choice though consumer spending should improve the quality of higher education provision by opening up universities to a consumer market. Lord Browne of Madingley, chair of the review body articulated this as In our proposals, we are relying on student choice to drive up quality. Students will control a much larger proportion of the investment in higher education. They will decide where the funding should go, and institutions will compete to get it (Browne, 2010, p. 29).
In doing so the White Paper and Browne Review served to structurally shift public perception about the benefits of higher education from being something that benefitted society as a whole, including employers, to a degree as being a commodity, the purpose of which was to benefit the individual graduate in terms of the potentially higher salary they would receive over their lifetime compared with a non-graduate. Students were now clearly conceptualized as being consumers of higher education.
Collini (2013, p.8, emphasis mine) argued that the central logic of the coalition's policy is clear enough, and it is emphasized in the data universities are now required to provide for applicants, the Key Information Set. The value of university education is the income it enables you to earn minus the cost of acquiring that education. Thus, it follows that applicants should, therefore, compare the salaries of graduates from different institutions, deduct the fees charged by those institutions, then make their choice based on value for money.
The personal benefits or value of a degree were articulated as being financial benefits to the individual. The rationale is that, because the benefits of university study accrue largely to the individual, then there was a prima-facie case that the individual should bear the cost of their education, rather than those taxpayers who do not participate in university education. The benefits of a degree are thus restricted to improvements in human capital. A shift in the 'burden' of the cost of university education from all taxpayers to only those who participate was a taken-for-granted assumption in the Browne Review (Davies, 2012). This also served to shift a considerable public cost from the public purse to the individual. One alternative perspective is that because the benefits (of being able to employ 'educated' and skilled workers) accrue to employers, including private sector employers as well as public sector ones such as in the National Health Service, local government, the police service and so on then they should bear the costs. Yet the neoliberal discourse and its logic that the individual beneficiaries of higher education should shoulder the financial costsblinded the general public to this other possibility. Another perspective, considered later, is that there are benefits to society as a whole, ergo, society should bear the costs of higher education.
The White Paper sought to position the public's perception of the value of a degree as being its worth to the individual throughout their working life, by making a case that this was 'fairer.' As the Department for Business Innovation and Skills argued, "graduates do, on average, earn more than non-graduates…so it is fairer…expecting graduates to pay" (2011, p. 17). This served to commodify a university qualification as being a learner's personal investment in their future worth in the job market. As Holmwood (2011, p. 5) argued, "the government now affirms education only in its contribution to the economy and as private investment in human capital." This shift in perspective from a universitygraduate being a public benefit for society, to that of a degree being a personal benefit for the individual graduate (Holmwood and McGettigan, 2011) reified the neoliberal perspective. This repositioning of a degree as a personal benefit ignored the wider benefits to society of an individual's participation in higher education. Benefits such as reduction in participation in criminal activity, increases in health, improved mental wellbeing and lower levels of mental health problems, reduction in obesity and reduction in use of health care services, higher levels of active citizenship, increased levels of voting, better parenting, and many more positive attributes that benefit society as a whole rather than the individual. The wider benefits of learning have been extensively researched (for example, Schuller et al. 2001, Feinstein, 2002, Hammond 2002, Cote, 2005, Brennan et al. 2013as well as there is considerable evidence about the wider benefits of lifelong learning (see, for example the series of report publications by the former Centre for Research on the Wider Benefits of Learning https://iris.ucl.ac.uk/iris/ browse/researchActivity/13847 and the Department for Business Innovation and Skills commissioned report by Dolan et al. 2012).

The Key Information Set (KIS) and the National Student Survey (NSS)
When introduced the Key Information Set (KIS) provided information about: measures of student satisfaction obtained from the National Student Survey (NSS) which, annually, captures student satisfaction ratings from courses within each UK university from all final year undergraduates (although courses with fewer than 10 students are excluded); the Destination of Leavers from Higher Education (DLHE), which surveys students six months after graduation; a Long DLHE survey carried out forty months after graduation; and HEI supplied data about course costs and accommodation costs. This information was presented as fifteen items; eight from the NSS, two from the DLHE, and five from individual HEI records. The heavy reliance on NSS results is, I argue, a very serious flaw in the KIS. 2008 a commissioned report to the UK's Higher Education Funding Council found that "The vast majority of institutions are not significantly different in their NSS scores from those we would expect given their profile of student, course and institutional characteristics" (Surridge, 2008, Executive summary point eleven). While later work by Cheng and Marsh on the NSS in 2010 concluded that differences in the NSS responses between universities are not large and that they "lack reliability and few differ significantly from the grand mean" (2010 p. 707), furthermore that, at the level of the individual institution, there are few universities that differ significantly from the mean across all universities. "This suggests the inappropriateness of these ratings for the construction of league tables" (ibid. p. 708), yet this is precisely what has happened in the UK. The NSS is specifically used to inform and construct university league tables. This was subsequently reinforced by the use of the NSS as part of the KIS data set and the UNISTATS website (the official website for comparing UK higher education course data).
One of the stated intentions behind university league tables is that they should be used by applicants to compare different HEIs overall, different courses offered by different HEIs, and different courses offered within the same university, thus, in theory, allowing applicants to identify the 'best' universities and courses to which they may then apply. Because of this, they serve to force competition amongst universities to secure a high position in one or more of the league tables. Cheng and Marsh's 2010 study of twenty institutions with above or below average student NSS ratings showed that withincourse variation, along with small sample sizes, meant that it was extremely difficult to distinguish between teaching quality at the course level. They found that inter-course ratings "lack reliability, and few differ significantly from the mean" (ibid. p. 707). Furthermore, there was not good agreement among students within the same university. And at a course level, there was an even smaller proportion of differences that are statistically significant because "the number of students within each course is too small to reliably differentiate between the courses (ibid. p. 706). Their research specifically identified that the NSS ratings lacked reliability and differences between universities could only explain a small amount of variance in NSS responses, with the recommendation the NSS ratings should only be used with "appropriate caution" for comparing universities. This caution applied to comparisons of ratings averaged across different universities and particularly to comparisons of different courses either within the same university or the same course across different universities. As they identified Any such comparisons should be qualified about interpretations of probable error based on appropriate multilevel models. These necessary cautions in the interpretation of NSS ratings also call into question their usefulness for their intended purposes (ibid. p. 709).
Yet with league tables use to compare institutions, this appropriate caution is largely ignored, as the NSS is seen and used an inter-institutional comparator, both by universities and applicants. Since their work there have been numerous research papers critiquing the NSS, highlighting flaws in it and suggesting more meaningful and nuanced ways of ascertaining student satisfaction (see, for example, Child, 2011, Bennet and Kane, 2014, Yorke et al. 2014, Bell and Brooks, 2016, Sharpe, 2019. Within a relatively short time of the introduction of the NSS, many universities appeared to be engaged in 'playing the league table game,' i.e., selectively choosing and using league table data to show themselves in a positive light. Almost all do very well in at least one aspect of the NSS, and this is then given high priority on their website. As Cremonini et al. (2008 p. 379) identify, "each university will highlight those rankings where it is rated best, without mention of where it is not rated so well." Do they maintain that if all universities claim to be 'good', then how can a prospective undergraduate make any clear quality distinction between them?
Since rankings alleged raison d'etre is to help consumers (prospective students) make informed decisions on their educational future by providing comparable information on the quality of HEIs, faults in the indicators effectively invalidate the exercise. Rankings then appear more like "beauty contests" than comparative quality assessments (Cremonini et al. 2008 p. 378). Davies et al. (2009) considered the implementation of KIS through the lens of Human Capital Theory and student participation in HE for marginalized students, arguing that the government's position of viewing the worth or value of a degree as being the benefits which accrue to the individual is unwarranted. Their work questions the decision-making ability of autonomous individuals (prospective students) as to both which university to study at and which course to study. Yet this assumption about the applicant's decision-making ability fundamentally underpins the White Paper and the KIS data. Davies later work (2012) makes a clear case that learners are faced with several marginal decisions once the initial decision to study at uni,versity has been taken. If the costs and benefits of university study are conceptualized solely in terms of the financial benefit to the individual, there are several factors that serve to obscure a costbenefit analysis; yet the NSS and KIS encourage potential applicants to do just this. Factors such as wide variation in the distribution of earnings between subjects studied, with tighter grouping around an average earning for some subject areas, such as medicine and education than for others, such as economics, (Chevalier, 2011) lead to wide inaccuracy in learner's predictions of the financial premium that they potentially may acquire from studying different subjects. This is regardless of any choice whether or not to study that same subject at a different university. Chevalier's research showed that even once high-earning medical graduates had been removed from the Long-DHE data, he studied, that there was still "a 25% gap in the mean earnings of graduates between the worst and better-paid subjects" (Chevalier, 2011(Chevalier, p. 1189. Earlier work by Work by Webbink and Hartog(2004) identified that students had broadly accurate expectations about future earnings, yet there was a considerable difference between students and a high variance level within-subjects. This variance within subjects reduced the chance of finding a significant statistical difference between subjects. This might be interpreted as meaning that students were unaware of different salaries between different professions; therefore, expecting potential students to be able to make accurate decisions as to which course to study and which university to study it based on predicted future earnings is highly problematic. A note of caution must be made here; however, as the research refers to predicted starting salaries, not actual salaries after graduation.
When seen within the broader overall picture of the commodification of a HE qualification so that its value is regarded as its worth in the job market, this point may also be contextualized by Bourdieu's work on the concepts of social capital.
Because the material and symbolic profits which the academic qualification guarantees also depends on its scarcity, the investments made (in time and effort) may turn out to be less profitable than was anticipated when they were made (there has been a de facto change in the conversion rate between economic capital and capital (Bourdieu, 2004, p. 21).
What this effectively means is that if a prospective student makes their choice of degree subject based on its costs versus its future profitability in the form of enhanced salary, then they may be unable to do so with any predictable accuracy; because changes in the job market take place over time. There may, for example, be a high demand for computer programming graduates when a 17-year old applies to HE, but by the time they graduate this demand has diminished and there may by then be a surplus of programmers -making their degree of far less commercial value and financial benefit than they had estimated. If the future financial-value of a degree to the individual graduate cannot be predicted with any accuracy, then how can prospective undergraduates be expected to make their choice of university course using this as a determinant? Burgess et al. (2018), in a ten-year study of determinants of student satisfaction, have suggested that the NSS should measure students' satisfaction levels in the years after they have graduated, by when they may have a greater appreciation of the value of their degree. Their study concluded that the NSS process was robust, had improved since its introduction and offered better discrimination between courses than previously, yet that"If the NSS has one limitation, it is that it fails to address student perception of value-for-money and, with the increasing influence of market forces in HE, this seems to be an important oversight" (Burgess et al. .2018, p. 12). I would argue that this limitation is not merely an oversight, but a serious flaw in the NSS, that undermines the rationale for the Key Information Set.
One of the White Paper's stated central challenges was to ensure that HE improved the quality of the student experience "institutions must deliver a better student experience" (Depart ment for Business Innovation and Skills, 2011 p. 4). Was this an aspect of higher education that the White Paper artificially 'problematized'? Holmwood and McGettigan(2011, p.4) identified that Universities in the UK are not currently differentiated in terms of teaching quality. The National Student Survey shows that there is a very high degree of satisfaction among students across all universities with their courses…Although there are various rank orders of Universities that are produced using the NSS…all independent studies and statistical evaluations of NSS, show that those rank orders are invalid, precisely because nearly all Universities are clustered within a few points of each other, and the differences among them are, for the most part, not statistically significant.
This reinforces the view that the NSS and the KIS are not 'fit for purpose.' Yet, if, according to the NSS data, the majority of students were, before the White Paper, already satisfied with the quality of their higher education experience, then why did this require a 'radical improvement.' And, if it did, then why did the radical improvement continue to use a flawed student satisfaction survey? Either the NSS process was functioning effectively and identifying that the majority of students were satisfied. Therefore no radical improvement was required, or it was not functioning effectively, and so did require a major overhaul. But following the 'radical improvement' that took place the same NSS process is used. Although it should be noted that some of the NSS questions have been modified since then, the process is essentially the same.

An (un)necessary KIS?
I now return to the title of this paper, 'An Unnecessary KIS, was it needed, and what was its real purpose?'. Did the Key Information Set serve to improve the quality of university teaching and improve student choice, or did it act to provide a ready-made neoliberal solution to a previously nonexistent problem? The NSS has shown that, generally, student satisfaction with their courses has increased over the last decade or so. Since the introduction of the KIS, there have undoubtedly subsequently been increases in students' perceptions of teaching quality, particularly student satisfaction, though this is to an extent largely attributable to universities focusing on improving their areas of the NSS where they did least well (e.g., Burgess et al. 2018), increased emphasis in engaging with students about the NSS and in taking steps to improve their score (which may not be the same as actually improving teaching quality). A simple Google search using the key words "how to improve your NSS score," for example, yields over 1.7 million results (accessed 27/11/2019) along with institutional toolkits and guidance for 'doing well' in the NSS. As previously discussed, 'playing the league table game' does take place, yet this would be expected within the dominant neoliberal agenda for higher education. Given that prospective students already had a considerable range of information to draw upon in deciding which university and which university course to study, was the purpose of the KIS to shift public perception about the value of a university qualification from being that of a public good (largely funded by the public purse) to that of being a private commodity (funded by the individual). This paper has argued that its real purpose was the latter and that the claim of providing information so that prospective students within a free market can make a genuinely informed choice based on the quality of teaching is untenable because the NSS is unreliable when used for comparing university courses.