Postdigital Research Measurement

Apologies for not getting back to you sooner. I'm afraid we are going to have to put [author’s Postdigital Science and Education article] on the back burner. I've been told in no uncertain terms by my line manager not to publish with PDSE until it has an impact factor. It's not about the intellectual value of PDSE; it is about the mind-numbing metrics obsessions of my masters. I feel I should be braver, but I feel there may yet be more important hills to die on...

Introduction
Some years ago my dear friend Hamish Macleod emailed me the article entitled 'Peter Higgs: I wouldn't be productive enough for today's academic system' (Aitkenhead 2013). The Nobel Prize winning physicist, and one of my intellectual heroes, said that he had a hard time working at the University of Edinburgh because he 'published fewer than 10 papers after his groundbreaking work, which identified the mechanism by which subatomic material acquires mass, [which] was published in 1964'. Higgs said that, after that work and prior to winning the Nobel prize in 2013, he had been treated like 'an embarrassment to the department when they did research assessment exercises.' Speaking of today's employment criteria, he says: 'Today I wouldn't get an academic job. It's as simple as that. I don't think I would be regarded as productive enough.' Responding to a question about the conditions which contributed to the Nobel Prize winning discovery, Higgs continued: 'It's difficult to imagine how I would ever have enough peace and quiet in the present sort of climate to do what I did in 1964' (Aitkenhead 2013). If people like Peter Higgs are indeed an embarrassment to their schools and departments, then who is their pride?
Higgs might have been an embarrassment to his department before he won the Nobel Prize for Physics, yet after he received the Nobel Prize, the same department has written accolades in his honour. How lovely is that? Higgs' work has been around since 1964, but his supposed 'peers'academics of roughly the same status, working at the same departmenthave managed to recognize its importance only after he received the Nobel Prize. However tempting, I do not want to turn this article into argumentum ad hominem: this appalling behaviour of Higgs' colleagues is a mere reflection of the wider pathetic, miserable, Janus-faced nature of the so-called 'objective' scientific community. Instead of playing the blame game, therefore, I am merely using the case of Peter Higgs to send a sobering message to those who still believe in the myth of scientific neutrality. Even the 'hardest' experimental sciences, such as physics, are games of hunger for the many and prestige for the few.
In a way, Peter Higgs is a lucky man. In 1964 he developed a theory that could not be proven without a complex apparatus which was built in the early twenty-first century. The 49 years between publication of Higgs' theory and its experimental confirmation is pretty long by human standards, yet Higgs has at least managed to get the world's most prestigious recognition during his lifetime. However, what happens to the prospective author of Postdigital Science and Education quoted at the beginning of this article, and indeed to all of us working in contemporary academia? How does our work, perhaps less significant than Higgs' but still somehow important, find its way through the mazes of 'research excellence'? And how do we find the room to do good work in the first place?
The dominating discourse has it that measuring excellence is beneficial both for knowledge development and for researchers (Hayes 2019). We want to support excellence over mediocrity, and we do not want slackers to take away our precious resources.
As for examples such as Peter Higgs (before he won the Nobel Prize), we can only hope that these examples are just unfortunate aberrations in an otherwise just system of reward and punishment. Unfortunately, they are not. While it is easy to agree with good intentions behind the dominating discourse of measuring excellence, it is now abundantly clear that the privileging of certain disciplines and methodologies over others, and accepting simplistic notions of impact, result in some deep flaws in the system. In this article I continue the theme of quantum physics, drawing on Karen Barad's interpretation of the work of Niels Bohr and Erwin Schrödinger, to explore some ontological and epistemic issues which might inform our understanding of measuring research excellence in a postdigital context.

Ontology and Epistemology, Sparkled with a Bit of Political Economy
The Double-Slit Experiment In Meeting the Universe Halfway: Quantum physics and the Entanglement of Matter and Meaning Barad (2007) interprets two famous quantum physics experiments: the double-slit experiment which can easily be performed in your kid's bedroom, and the thought experiment popularly known as the Schrödinger's cat experiment. The first half of the double-slit experiment consists of a coherent light source, such as a simple classroom laser-pointer, which illuminates a plate (this can be as simple as a sheet of paper), with two parallel slits (one above and one below) (see Fig. 1). As the light wave passes through the slits, it splits into two waves, and each of these waves spreads from its respective slit. The two waves interfere with each other. Where a peak meets a trough, the waves cancel each other; where a peak meets a peak, the waves reinforce each other. Consequently, the wall behind the plate will show a stripy pattern of multiple wave reinforcements (bright stripes) and cancellations (dark stripes) called Fig. 1 The double-slit experimentlight (Wikimedia Foundation 2007a). The image is in the public domain, and anyone has the right to use this work for any purpose, without any conditions, unless such conditions are required by law the interference pattern. Now for the second half of the double-slit experiment, let us replace the light source with a simple homemade straw-gun with paper bullets and start shooting! (see Fig. 2). Some bullets will pass through the upper slit, and some bullets will pass through the lower slit. After a while, our wall will end up with two heaps of paper bullets, formed as two strips, behind each slit. Two strips on the wall indicate that our paper bullets are particles; multiple interference strips on the wall indicate that light is a wave. Conducted in physics classrooms all over the world, this simple experiment clearly distinguishes particles from waves.
To enter the world of quantum physics, let us replace our laser beam and straw-gun with a source of electrons. When we turn on the source, the wall will show multiple interference strips of a wave. So far so goodwe can now conclude that electrons are waves. Just out of curiosity, let us now check which electron passes through which hole in the plate. To do so, we will equip each slit with a detector. And, surprise! When we turn on the detectors, the interference pattern characteristic for waves turns into the twostrip pattern characteristic for particles. As soon as we added the detectors, our electrons have started to behave like particles. Revealing this curious behaviour, the double-slit experiment shows one of the key mysteries of quantum mechanics. At a quantum level, in some experiments we can observe wave-like properties, and in other experiments we can observe particle-like properties. While we can never observe the wave-like properties and particle-like properties in the same experiment, we do need both models to explain all behaviours. On a quantum scale our concepts of wave and particle do not correspond to reality. In this way, twentieth century physicists have developed the concept of wave-particle duality. Fig. 2 The double-slit experimentparticles (Wikimedia Foundation 2007b). The image is in the public domain, and anyone has the right to use this work for any purpose, without any conditions, unless such conditions are required by law According to Barad (2007: 109) 'the indeterminable nature of measurement interactions is based on [Bohr's] insight that concepts are defined by the circumstances required for their measurement. That is, theoretical concepts are not ideational in character; they are specific physical arrangements.' (italics from the original) Based on this interpretation of Bohr's work, Barad has developed a more general theory called agential realism. As can easily be seen from the double-slit experiment, quantum particles change their behaviour when they are being observed. Therefore, the act of measurement is inseparable from the object of measurement; the human observer cannot be separated from observed non-human phenomena. Barad's agential realism elaborates Bohr's 'insights in a posthumanist direction that decentres the human' and suggests 'that Bohr's notion of a phenomenon be understood ontologically' (Barad 2007: 333). Thus, writes Barad, I take the primary ontological unit to be phenomena, rather than independent objects with inherent boundaries and properties. In my agential realist elaboration, phenomena do not merely mark the epistemological inseparability of 'observer' and 'observed'; rather, phenomena are the ontological inseparability of intra-acting 'agencies.' That is, phenomena are ontological entanglements. (Barad 2007: 333) (italics from the original) Barad's agential realism carries two important implications for measuring research excellence. First, it brings about a non-representationalist understanding that measurement always depends on the act of measuring. The act of measuring research excellence changes the nature of what is being measured, i.e., constitutes research practices which are different from research practices that might develop in a non-observed environment. More generally, the act of measuring research excellence brings the very concept of research excellence into being. Different ways of measuring will bring about different research; principles, practices, and policies of measuring research excellence carry a lot of ontological significance.
Second, Barad claims that '[r]eality does not depend on the prior existence of human beings; rather, the point is to understand that "humans" are themselves natural phenomena.' (Barad 2007: 336) Therefore, she concludes that '[i]t doesn't make sense to hold onto an anthropocentric conception of measurement; on the contrary, a commitment to a thoroughgoing naturalism suggests that we understand measurements as causal intra-actions (some of which involve humans).' (Barad 2007: 338) Measurement does not only influence practice; it is an inseparable element of the thing or phenomenon being measured. Yet at least since Romanticism, Western science is based on 'the dominant, romantic, highly individualistic and irrational account of "personal anarcho-aesthetics"' exemplified in the notion of the creative individual (Peters and Jandrić 2018: 343). This produces the dominant contemporary discourse where the main task of measuring excellence is to recognize creative, hard-working researchers and weed out 'slackers'. Yet sides in the apparent binary between productive researchers and slackers are dialectically intertwined parts of the larger system of knowledge making and dissemination, which cannot be separated from researched phenomena and from each other.

Schrödinger's Cat Thought Experiment
In 1935 Erwin Schrödinger offered his contribution to the problem of measurement by the way of his famous Schrödinger's cat thought experiment (Fig. 3). In his paper 'The present situation in quantum mechanics' Schrödinger introduces the experiment as follows: A cat is penned up in a steel chamber, along with the following device (which must be secured against direct interference by the cat): in a Geiger counter, there is a tiny bit of radioactive substance, so small, that perhaps in the course of the hour one of the atoms decays, but also, with equal probability, perhaps none; if it happens, the counter tube discharges and through a relay releases a hammer that shatters a small flask of hydrocyanic acid. If one has left this entire system to itself for an hour, one would say that the cat still lives if meanwhile no atom has decayed. The first atomic decay would have poisoned it. The psi-function of the entire system would express this by having in it the living and dead cat (pardon the expression) mixed or smeared out in equal parts. (Schrödinger 1935(Schrödinger /1983 With this vivid image of a cat smeared out in equal parts around the box, which has become an inseparable part of popular culture, Schrödinger makes some important points about the nature of measurement. This smearing (hopefully) does not arrive from Schrödinger's vicious character, but refers to a situation where the cat's indeterminate state results from 'a superposition of "alive" and "dead" states of the cat.' This does not imply 'that the cat is either alive or dead and that we simply do not know which', or that the cat is a part-dead, part-alive zombie, or that the cat is neither-alive-nor-dead vampire-like creature. 'Rather, the correct way to understand what the "smearing" stands for is to realize that the cat's fate is not simply metaphorically entangled with the radioactive sourceit is literally in an entangled state.' (Barad 2007: 278) For the cat, the act of measuring is the act of transition from this indeterminate, entangled state into one of the two determined states: dead or alive. So the key question pertaining to measurement is: what happens in the moment of this transition? Schrödinger does not subscribe to one or another 'mystical' explanation; and neither does Barad. In language of physics, '[i]t seems as if the wave function has somehow "collapsed" from a superposition (or entanglement) to one in which all the coefficients except one of them are set to zero' (Barad 2007: 280). Or, in simpler but less detailed words, the act of measurement kills potentiality by selecting only one of many possible options.
For Schrödinger, this murder of potentiality is the beginning of the problem of measurement. The trouble is, that the Schrödinger's cat experiment is not reproducible each time the box is opened, the cat has a new chance to surface dead or alive.
Thus for Schrödinger the problem of measurement is resolved as follows: what appears to be a discontinuous change in the wave function is not due to some distinctive law of nature governing measurement interactions that creates a discontinuous change in the wave function; but rather what is actually going on is that the wave function of the 'object' becomes entangled with the 'measuring system' (…) such that they are no longer separate systems. Only upon observation by a cognizing agent can we speak of a resolution of the entanglement. (Barad: 2007: 284) (italics from the original) Following Barad's interpretation of Schrödinger, academic work before measurement can be understood as 'a catalog of the maximum knowledge of a system that it is possible to obtain in principle'. Unlike Bohr's ontological view to the double-slit experiment, Schrödinger's solution to the problem of measurement is decidedly epistemological. The act of measuring research excellence creates an entanglement of our knowledge about academic work with our knowledge of our measuring instruments such as the UK's Research Excellence Framework. 1 Measuring excellence resolves this entanglement, but reduces our knowledge of a researcher's work to a single result ('good' or 'bad' research). Research excellence exercises are highly competitive, so 'good' researchers are typically supported in furthering their work, while 'bad' researchers are often directed towards other duties such as teaching or administration. More often than not, those measures kill potentials for further development of their research.

Postdigital Postdigital Measurement of Excellence in Higher Education
We now live in a postdigital world, 'where digital technology and media is [no longer] separate, virtual, 'other' to a 'natural' human and social life' (Jandrić et al. 2018: 893). Together with development of postdigital conditions in the past few decades, measuring research excellence follows trends similar to educational research in general. Much has been written about policies such as New Public Management and neoliberalization of the university (Peters and Jandrić 2018), domination of one-sided discourses of excellence (Hayes 2019) and the dialectically intertwined notion of student experience . These trends are underlined by a disappearance of human agency; much too often, it is posited that quality improvements arrive magically by applications of this, or that, technology (Hayes and Jandrić 2014). In this context, Sarah Hayes questions why there is often no explicit reference to the very humans who will enact the tasks stated, and in many cases, no reference to those who authored said policy documents either (Bartholomew and Hayes 2015;Hayes 2019: viii). These problems are exacerbated by data-driven algorithmic systems that process various forms of assessment in seemingly objective but actually very skewed ways. The problem can be classified in two main steps (Jandrić 2019). First, problematic measurements perform problematic constructs into being; second, these problematic constructs are operated on by algorithms and systems that are opaque and biassed in important ways.
Speaking of data, Ben Williamson asks a fundamental question 'who owns educational research?' and argues: The central argument is that as educational data science has migrated from the academic lab to the commercial sector, ownership of the means to produce educational data analyses has become concentrated in the activities of for-profit companies. As a consequence, new theories of learning are being built-in to the tools they provide, in the shape of algorithm-driven technologies of personalization, which can be sold to schools and universities. (Williamson 2017: 105) In his analyses of student assessment Cormac O'Keefe examines quality of collected data. He suggests that standardized tests, which are now routinely used in various contexts from admissions to final exams, produce large datasets about studentsand these datasets suffer from various biases. Looking at 'the role of psychometric practices and educational testing theories ', O'Keefe (2017: 123) shows 'that large-scale digital assessments as tests such as PIAAC [The Programme for the International Assessment of Adult Competencies] do more than produce data about ability'. More importantly, '[t]hey perform the concept of ability into being." (O'Keefe 2017: 133). These days, measuring methods and strategies across academia have become almost completely McDonaldized (Ritzer et al. 2018). Therefore, Williamson's analysis of ownership over academic data and O'Keefe's analyses of student assessment can easily inform the argument about measuring research excellence.
AIs arrive on top of the commercialised entanglement between data excellence and bring in an additional level of indeterminacy.
According to Liza Daly, 'artificial intelligence is the umbrella term for the entire field of programming computers to solve problems. I would distinguish this from software engineering, where we program computers to perform tasks.' (Daly 2017) This simple definition describes an important paradigm change in the inner workings of the computer. Traditional computers, including the most sophisticated expert systems of yesterday, consisted of long lines of code which determined their behaviour: for every input, such systems would do predetermined calculations and provide an output. In contrast, AI systems are provided with some initial rules of behaviour, and then they are 'taught' by large datasets. Then, a computer independently establishes various connections between input data and produces 'intelligent' solutions to new problems in non-predetermined ways. This is the essence of machine learning, which is broadly defined as 'the science of getting computers to act without being explicitly programmed' (Ng 2018) (Jandrić 2019: 31-32) By design, AI systems transfer a lot of agency from human beings to inner workings of the computer. Data bias inevitably superposes with AI bias: problematic measurements perform problematic constructs into being and are then operated on by algorithms and systems that are opaque and biassed in important ways. As can be seen from examples arriving from various walks of life (see Jandrić 2019; Peters and Jandrić 2019), results of these operations are far from trustworthy. As I write these words there are no adequate technical solutions for data / AI biassesperhaps because the nature of these biasses is not only technical but also normative / ethical (Fuller and Jandrić 2018: 190). At present tech companies resolve data / AI biases 'manually', by introducing various practices based on human labour such as algorithm auditing (Hempel 2018;IBM Research 2018; see also Jandrić 2019: 33). However, these practices are far from perfect, and applications of AI-based decision-making systems in areas such as healthcare remain far from fair (Eubanks 2018).

Conclusion
Measuring research excellence brings a particular concept of research excellence into being. Research is dialectically related with the process of academic publishing, which 'is a form of "social production" that takes place across the economy, politics and culture, all of which are in turn accommodating both old and new technology in our postdigital age' (Jandrić and Hayes 2019: 381). For as long as 'good research' is defined by journal impact factors, a place of publication and publication popularity will be more important than an article's content. For the prospective PDSE author quoted at the beginning of this article, this means producing publications for 'better' journals. Yet how 'better' journals are defined is in itself open to question in a postdigital context. Beyond academic publishing, the mutually constitutive relationship between measurement and research has resulted in the underappreciation of the value of Higgs' work for 49 years. On this basis alone, it is worth reviewing how measurement and research are currently being linked.
All researchersdespite their position in the Hunger Games of research excellenceare dialectically intertwined parts of the larger system of knowledge making and dissemination. Higgs became famous only because other scientists did not become (as) famous. There is only so much room at the top, and the notion of excellence is heavily rigged towards those who are already there. For instance, one of the main difficulties of starting a new academic journal is the cruel game illustrated by this article's leading quotation where academics need to publish in 'good' journals in order to secure their jobs. 'Our postdigital age is one of cohabitation, blurring borders between social actors and scientific disciplines, mutual dependence, shifting relationships between traditional centres and margins, and inevitable compromiseand this calls for deep reconfiguration of politics and practice of knowledge production and academic publishing as we know it.' (Jandrić and Hayes 2019: 390). Producing standard science is very hard, but boring; it takes a lot of courage, expertise, and luck, to break the glass ceiling of measured excellence and make a real change. This pushes some academics towards various strategies of gaming the system, which are harmful for science and the society at large (see Fuller and Jandrić 2019).
The last problem with measuring research excellence is the vicious murder of potentiality. Who knows what the prospective PDSE author might have written had he not been coerced into playing by the rules? Defined as the 'dirty little industrial machine' (Peters in Jandrić 2017: 378), an academic article is always of same format, same length, same methodologyand top quoted journals have mastered this art of the same to perfection. While measuring research excellence often brings about vicious consequences to researchers' careers, quantum mechanics offers a straw of hope. Schrödinger's cat experiment teaches that the entanglement of our knowledge and measurement systems can collapse differently in each experiment, so every day offers a new chance.
This brief overview of issues pertaining to measuring research excellence in a postdigital context reveals the same issues as our short trip to mazes of quantum physics. Arguably, these concordances have always been there, yet in the postdigital condition they surface more prominently than ever. So how far can we extend these concordances; what can quantum physics teach us about measuring research excellence? This question can be answered in many different ways, yet its full elaboration reaches far beyond the scope of this article. Following Barad's advice, I am not interested in developing direct links between the double slit experiment, Schrödinger's cat experiment and measuring research excellence. Instead, 'I am interested in understanding the epistemological and ontological issues that quantum physics forces us to confront, such as the conditions for the possibility of objectivity, the nature of measurement, the nature of nature and meaning making, and the relationship between discursive practices and the material world.' (Barad 2007: 24) Arguments presented in this article, therefore, are not about applying quantum physics to measuring research excellence; they are presented to help us reach a meta-level of epistemology and ontology which can be applied to both systems of thought and their entanglements.
Which messages from quantum physics could possibly be of use for this analysis of research excellence in a postdigital context? According to Barad, [w]hat is needed is an analysis that enables us to theorize the social and the natural together, to read our best understandings of social and natural phenomena through one another in a way that clarifies the relationship between them. (…) considering them together does not mean forcing them together, collapsing important differences between them, or treating them in the same way, but means allowing any integral aspects to emerge (by not writing them out before we get started). (Barad 2007: 25) Moving from theory to practice, we are now left with two main approaches. The first approach, which is probably most reasonable for many of us including the prospective author quoted at the beginning of this paper, is business as usualbut with more awareness of issues pertaining to postdigital measurement and occasional fixes. The business as usual approach is by definition unable to make significant changes, yet awareness of these things could result in occasional pockets of freedom or an odd personal advance resulting from knowing how to game the system. The second approach is to try and redefine theory and practice of research measurement according to these philosophical insights. My best bet is on attempts starting from systems theory and cybernetics, but there's plenty of opportunity in other fields. Working in, against, and beyond contemporary academia (Holloway 2016), we need to simultaneously engage with both approaches. While I fully appreciate the importance of solid work done under the business as usual scenario, my heart is with those who dare to imagine radically different futures.


Impact of Intangible Characteristics of Universities on Student Satisfaction

The aim of this research was to investigate the impact of the intangible qualities of the universities on student satisfaction. To do this, we have collected data from 7 different major public and private universities of the Kurdistan Region of Iraq. We have used 170 data to proposed further analysis. The partial least square method (PLS) was used to test the hypothesis. The results reveal that career opportunities and a friendly atmosphere are the main two elements that foster the reputation of the universities. The second interesting result of this research is that social activities impact the reputation of universities but not the friendly atmosphere while social activities impact a friendly atmosphere but not the reputation significantly. Finally, we have suggested the implications to the practitioners in

Introduction
Academic profession in Kurdistan region of Iraq has gone through a lot of changes since its liberation from Saddam's Iraqi regime. Various structural changes in the education field shaped the academic profession all over the world over the past few decades (Enders & de Weert, 2009). One such developments is the cut-throat competition among the universities. There are thousands of good higher education institutions and universities around the world. Potential students of higher education institutions all over 39 Tishk International University, Sulaimani, Kurdistan Region of Iraq 40 Tishik International University, Erbil, Kurdistan Region of Iraq the world find it difficult to choose a university which can fulfill their educational needs and demands. On one hand, students are compelled to shortlist higher education institutions or universities, for instance, while studying in Alevels in UK (Moogan et al, 1999). On the other hand, universities face tough competition and raised-expectations of all stakeholders, especially the potential customers or students. Due to factors like globalization and increased competition, universities now realize that in order to attract, retain and satisfy their customers, or students, they are supposed to work as efficiently as commercial companies do, in any service industry (Cheong & Tam, 1997).
Traditionally, higher education institutes were supposed to produce limited number of scholars who would teach and produce quality research. With the passage of time, more and more people realized the importance of education and joined universities. Private universities came into being and gave rise to a new culture of education. Universities changed their strategies and started targeting the masses, instead of focusing on niche of intellectuals (Wan et al, 2015). Universities no longer operate as public service institutions, but now most of them are for-profit organizations, equipped with business acumen to attract target customers.
Things are not very different for higher education institutions in Kurdistan region of Iraq as well. The dynamic environment and economic factors contributed to transformation of universities, especially the private institutions to operate as commercial companies, trying to maximize their market share (Budur, Rashid & Poturak, 2018). The universities not only face competition from local higher educational institutions, but also from the better-ranked universities abroad. A vast number of students travel to overseas countries to have a good quality of education (Poturak, 2014). It is quite challenging for universities to attract students, to meet their expectations and keep their satisfaction level high, once they have joined the university (Hushyar Sherwani, 2018). Whilst the number of private university students is increasing every year, the universities still strive to enhance their market share by using various strategies that attract prospective students to campus.
The purpose of this study is to determine which dimensions of an educational experience a university should improve in order to significantly increase overall student satisfaction. In this study, student satisfaction is measured along eleven independent variables. The results of this research will help in finding out which variables have most significant impact on overall student satisfaction.

Overview of the higher education sector in Kurdistan
Some of the renowned higher education institution operating privately in Erbil, Sulemani and Duhok cities of Kurdistan region are as follows: Cihan university, Ishik university, Komar university, American university of Iraq, Human Development university, International university of Erbil, Bayan university and Qalam university. Major public universities in the region include Salahadin University in Erbil, Sulaimani University and Sulaimani polytechnic university in Sulaymaniyah, Koya university in Koya, and Raparin university in Raparin. Some of the public universities have been operating for the past 6 decades. However, private higher education sector is relatively new in Kurdistan. The first private university was founded in the region in year 2002. Most private universities offer an undergraduate degree with duration of three to four years, which is called as bachelor degree. The exception to this duration is degrees in medicine, dentistry, pharmacy and architecture, where longer and more extensive study duration is required. Public universities in Kurdistan usually have annual examination system, whereas private universities have adopted the semester system for bachelor as well as master degrees.
According to Kotler (1997), if the choices available are various and complex, the decision making can be termed as extensive problem solving. In the consumer behavior field of study, different types of factors (reputation, image, extra services, customer relationships, etc.) have been defined as influential on the customer preferences (Sahin and Singh, 2017;Budur, 2018;Jaf et al, 2019). Beside the management is another influential factor on the employees to promote organizational effectiveness (Budur and Demir, 2019 a,b) and in turn attract customers for the competitive advantage in the market (Demir and Aydinli, 2016). There are various factors affecting the student's decision-making process when it comes to choosing a university such as cost, facilities on campus, scholarships, social and scientific activities, environment, and the university's reputation or brand name (Budur et al., 2018). These factors eventually lead to student's satisfaction or dissatisfaction with the university. In order to get a greater share of the pie, it is very important for the higher education institutions to examine the source of satisfaction of students with the university they choose. Once the university knows the important factors that impact students' choice of university, they can make improvements in those areas and attract and retain the maximum number of target customers or students.

Literature review
A substantial amount of research has been carried out in order to better comprehend the factors that affect the decision-making process of a student to pursue higher education. While early work was mainly done in developed countries, for instance, Kotler (1975) and Chapman (1981), focusing on factors influencing the university selection on both student's as well as university's side, respectively. Subsequent research has been contributed by scholars in developing countries as well, for instance, Budur et al (2018) increased the breadth of factors considered by studying the same in Kurdistan region of Iraq.
Past research concluded that even though selection of a university hardly has any impact on final outcomes for the students, as personal study habits and intelligence are more important determinants of success in higher education (Need and De Jong, 2001). However, due to higher education being a highly competitive industry, universities face challenge of attracting new students and students get mind boggled to see so many alternatives. Both parties need increased information to help them succeed in their respective quest.

Student satisfaction
Satisfaction is the perception between expectation and experience with the service or product (Demir and Mukhlis, 2017;Budur et al., 2019c;Torlak et al., 2019). The authors (Demir, 2019a(Demir, , 2019b have suggested that the satisfied customers are more inclined to repurchase that service again. Student satisfaction can be defined as a subsequent short-term attitude of students after educational experience (Elliott & Healy, 2001;Demir, 2017). Hartman and Schmidt (1995) pointed out multi-dimensional nature of student satisfaction. Students select universities that meet their academic, financial and social needs. Moreover, Demir and Guven (2017) suggested that being certified by official authorities such as international standards organization (ISO) had significant and positive impact on the students' satisfaction.
Students are satisfied when a university they select meets or exceeds their expectations. According to Browne et al (1998) student satisfaction is evident from students' actions. They recommend the university to their friends and relatives if they are satisfied with it. Another dimension of student satisfaction is their intention to retain or repurchase the services of the same university if they are given another chance.
Student satisfaction depends on many factors such as clarity of student goals (Hartman & Schmidt,1995), trust in and built by staff, perceived quality or reputation of university (Athiyaman, 1997), career support services such as career counselling (Kotler & Fox, 1995), cost of education (Patton, 2010) and extent of interaction between the students and university staff (Browne et al, 1998). Other factors that lead to students' satisfaction with the choice of university include location, facilities or infrastructure, future career prospects and quality of life and social activities (Veloutsou & Paton, 2004). Moreover, student centeredness, campus climate or atmosphere, and instructional effectiveness also have an influence on overall student satisfaction. Last but not the least, need and merit-based scholarship also play an important role in selection and retention of the students at higher education level (Single Jr., 2004).

Cost and Scholarships
While elite higher education institution is less cost sensitive, as students always get drawn to them irrespective of cost, other universities which are not in top tier do not enjoy such privilege. Therefore, demand for most higher education degrees is quite elastic and poses serious potential problems for the universities (Shurden et al, 2010). This leads to conclusion that cost is an important factor that affects a student's decision-making process regarding university selection or in other words, student overall satisfaction with educational experience. Increasing the cost of higher education can drive the students away from the university and in turn can damage their chances of survival in competitive higher education industry. On the contrary, scholarships and financial aid decrease the overall cost of education and attract more students (Elliot and Healey, 2001).

Reputation in the market, internationality and career support/prospect
University's reputation among masses and its overall ranking substantially influence student's purchase decisions and satisfaction. Usually students rely on word-of-mouth about the university and its services and bad comments decrease their willingness to choose the university (Elliott & Healy, 2001). Scholars have also established significant relationship between university reputation and student choice of university (Abbott & Ali, 2009).
Graduates of highly reputed universities have better career prospect. They find jobs easily and are preferred by the employers as well. Career prospects and support are among significant factors that affect students' decision-making process (Munisamy, Jaafar & Nagaraj, 2014). Internationality of a university is measured by the number of international faculty and international students that a university has. Internationality promotes diversity in the university, and helps students learn about new cultures, develop various thinking and problem-solving techniques and improve decision-making process (Sherry, Thomas, & Chui, 2010). Teachers from different cultural and ethnic backgrounds share various kind of experiences, knowledge and perspectives with the students.

IT Services and Scientific Activities
Computer and IT services such as types of internet, online student portals, online management systems and other e-services improve a student's perception of the university's quality (Gatfield et al., 1990). Elliott and Healy (2001) also confirmed the influence of IT services on satisfaction level of the students. Scientific activities include conferences, workshops, seminars, exhibitions and projectbased learning activities organized by the university. Scientific activities not only help universities improve their performance as mentioned by Montilla (2004), but also influence students' university selection and overall student satisfaction (Kotler & Fox, 1995;Budur et al., 2018).

Campus Climate or Atmosphere and Social Activities
University is not simply considered as a place that offers a degree, but it is perceived as a platform where you learn to be a good citizen and a better person. Students are drawn to the universities which have nice, friendly and positive atmosphere (Montilla, 2004). Some studies even concluded that it is one of the most important factors influencing a student's decision of choosing a university (Elliot & Healey, 2001;James et al., 1999). Moreover, if students perceive that the university offers a good social life on campus, they will be more likely to choose it over a university with no entertainment services and tedious and boring social life (Moogan & Baron, 2014). Hypothesis Based on the literature above, we have developed hypothesis of the current study as; H1 Academic staff has significant impact on the cost perceptions H2 Academic staff has significant impact on the friendly atmosphere H3 Academic staff has significant impact on the reputation H4 Career opportunities has significant impact on the cost perceptions H5 Career opportunities has significant impact on the friendly atmosphere H6 Career opportunities has significant impact on the reputation H7 Cost perceptions has significant impact on the reputation H8 Friendly atmosphere has significant impact on the reputation H9 IT services has significant impact on the cost perceptions H10 IT services has significant impact on the friendly atmosphere H11 IT services has significant impact on the reputation H12 Reputation has significant impact on the satisfaction H13 Scientific activities has significant impact on the cost perceptions H14 Scientific activities has significant impact on the friendly atmosphere H15 Scientific activities has significant impact on the reputation H16 Social activities has significant impact on the cost perceptions H17 Social activities has significant impact on the friendly atmosphere H18 Social activities has significant impact on the reputation

Sampling
The research was studied in Kurdistan Region of Iraq. Therefore, we have gathered data from the major private and public universities of the region. Initially, we have distributed 500 questionnaires to the students who studied at one of the major public or private universities of the region. Besides, only 200 students have filled the survey questionnaire and turned it back. Remaining 300 students haven't responded back. Moreover, we have observed that 30 of the returned data was invalid due to the technical problems such as incomplete and impulsive filling of the questionnaire. Hence, there are 170 data for the further analyses.

Measures
In this study, we have aimed to investigate the impact of intangible qualities of the universities on the student satisfaction. To do this, we have used the questionnaire of Budur, Rashid, and Poturak (2018). The survey contained dimensions such as; academic staff (four items), career opportunities (three items), social activities (three items), scientific activities (three items), IT services (three items), friendly atmosphere (three items), cost perceptions (three items), reputation (five items), student satisfaction (three items).

Procedures
The aim of this research was to investigate the impact of intangible qualities of the universities on the student satisfaction. To do this, we have collected data from major universities of Kurdistan Region of Iraq. Initially, validity and reliability analysis were proposed to prepare data for the analysis. Secondly, Partial least squares method was employed to test the hypothesis.

Demographic information
The study contains 170 sample. Among those, 64 percent were males while 35 percent were females. Participants have been collected from various public and private universities of the region. The university names haven't been mentioned in the study due to the privacy. It has been observed that 22 percent of the students were paying 1000-2000 $ tuition fee to their universities, 29 percent were paying 2001-3000 $, 17 percent were paying 3001-4000 $, 10 percent were paying 4001-5000 $, 7 percent were paying 5001-6000 $, and 14 percent were paying 6000 $ or more to their universities. Further details can be seen on the Table 1.

Validity and reliability
Partial least squares method is more convenient than structural equations modeling when sample size is less than 200 (Chin and Newsted, 1999). Barclay et al., (1995) have suggested that 70samples are enough to perform partial least squares in case there are no formative constructs. In this study, we have no formative constructs in our model. Therefore, 170 sample is enough to perform partial least squares method. In the model of the current study, there were 25 questions under nine dimensions. We have proposed analysis for internal consistency, reliability, and discriminant validity of the questionnaire. Table 2 represents the results of internal consistency and reliability analysis.
Individual item reliability has been tested via factor loading under each dimension. Igbaria, Guimares, and Davis (1995) have suggested the minimum threshold of factor loadings to be 0.5 in order to call them "very significant". The results of the analysis have shown that items' factor loadings under each construct were above 0.5. Hence, it can be concluded that the reliability of each item under concerning construct have been satisfied.
Another test was internal consistency. Fornel and Larker (1981) have suggested 0.7 for each composite reliability in order to be accepted as reliable. It was observed that both composite reliability and Cronbach's Alpha values were above 0.7. However, average variance extracted for each construct have exceeded 0.5. Therefore, we can conclude that internal consistency has been achieved.
Discriminant validity was tested in order to prove that each construct was at appropriate distance comparing to each other. To understand that, we need to make sure that square roots of average variance extracted are above the correlations of a construct with other dimensions. Table 3 shows that the correlation values between variables were below the square root of average variance extracted for concerning dimension. Thus, it can be concluded that there are enough spaces between variables to be considered as a separate dimension.

Testing hypothesis
Partial least squares method has been used to test the hypothesis. To do this, we have employed Smart PLS software. Given in the Table 4, it can be observed that there is some hypothesis that have been accepted while some others were rejected. Adjusted R It has been revealed from the analyses that academic staff had positive and significant impact on the cost perceptions of the students and friendly atmosphere inside the university while it did not have a significant impact on the reputation directly. Rather, this impact was mediated by friendly atmosphere. Therefore, it can be said that quality academic staff that behaves friendly with the students enhance friendly atmosphere inside the university and that way effects the reputation positively. The results show that H1 and H2 have been accepted and H3 has been rejected.
It has been observed that career opportunities that students possibly might have impacted the cost perceptions of the students, friendly atmosphere inside the university, and reputation positively and significantly. Hence, it can be revealed that career opportunities are one of the most important determinants that foster positive perceptions of the students about the university. According to these results, H4, H5, and H6 have been accepted.
We could not find any significant relations between cost perceptions and reputation. Thus, the students do not evaluate the reputation of a university directly via tuition fee of the university. As friendly atmosphere had significant impact on the reputation, it can be said that the friendly atmosphere is more important than the cost perceptions of the students for the reputation of a university.
It has been observed that IT services impacted cost perceptions significantly and negatively. The result reveals that when the IT services are better, students think that the fee they pay is not expensive for that university. Besides, IT services did not have any significant impact on the friendly atmosphere and reputation directly. Rather, that impact is mediated by the cost perceptions. Based on these results, H7, H10, and H11 have been rejected while H8 and H9 have been accepted.
The results show that reputation had significant impact on the students' satisfaction. However, it explained 43 percent of the overall variance of the student satisfaction. Therefore, it can be said that if the reputation of a university is high, students are more satisfied of selecting that university. Thus, H12 has been accepted.
Based on the results, it was observed that number of scientific activities such as seminars, workshops, scientific trips…etc. impact the cost perceptions and reputation while didn't have significant impact on the friendly atmosphere. Therefore, it can be revealed that students evaluate their university highly reputed based on the number of the scientific activities organized. Besides, the scientific activities do not enhance friendly atmosphere as social activities do. As a result, H13 and H15 have been accepted while H14 has been rejected.
Finally, it has been observed that social activities had significant and positive impact on the friendly atmosphere of a university while didn't impact the cost perceptions and reputation significantly. Hence, H17 has been accepted while H16 and H18 have been rejected.

Discussion
Based on the results of the current study, we have found that reputation has significant and positive impact on the student satisfaction of university. This result reveal that student is satisfied when his/her university is respected in the market. This result shows similarity with the (Athiyaman, 1997). The author also has suggested that the reputation was one of the significant determinants for the student satisfaction.
Secondly, we could not find any significant relation between cost perceptions and satisfaction of the students. These results are controversial with the study of (Patton, 2010). They have suggested that the cost of university had significant impact on the student satisfaction. The reason here might be that the tuition fees are not high for the private and public university students. There are other priorities for them comparing to the cost.
The results show that career opportunities are the most important determinant of reputation of a university. The results show similarity with (Kotler & Fox, 1995), who suggested that the career support services such as career counselling has significant impact on the reputation and student satisfaction.
We have found that the quality of academic staff and their relation with the students impact the reputation of the university over friendly atmosphere. These results show similarity with the studies of (Browne et al, 1998).
Further, we have found that the social and scientific activities have significant impact on the friendly atmosphere, reputation, and cost perceptions of a university. However, social activities are not significantly related with the cost perceptions and reputation but friendly atmosphere while scientific activities are not significantly related with the friendly atmosphere but the reputation and cost perceptions of a university. These results show partial similarity with the study of (Veloutsou & Paton, 2004). They suggested that social activities increase quality of life in the campus and consequently effect the student satisfaction.

Conclusions
The aim of this research was to investigate the impact of intangible qualities of a university on the student satisfaction. As intangible qualities, we have selected academic staff, social activities, scientific activities, career opportunities, IT services, reputation, and cost perceptions.
The results reveal that the reputation of a university is one of the biggest determinants on the student satisfaction. This construct explained 43 percent of overall variance on the student satisfaction. However, friendly atmosphere was very important for the reputation of a university comparing to cost perceptions. Besides, cost perception didn't have significant impact on the reputation of a university. Based on these results, administration of universities is suggested to increase friendly atmosphere where there is no discriminations and all students are equally evaluated. Secondly, they need to create the career opportunities for the students as internships, part-time and/or full-time employment at highly reputed firms. Third, they need to organize scientific activities for the students such as projects Olympiads, workshops, seminars, conferences in order to increase the practical knowledge of students.
In order to increase the friendly atmosphere, we have found that career opportunities and social activities play important role. As the friendly atmosphere play important role in reputation, the administration of universities is suggested to organize social activities such as cultural and sports competitions, festivals…etc. in order to keep staff and student together in a friendly atmosphere.


Design Thinking Applications in Teaching Programming to Gifted Students

The study aimed to present suggestions for how a design thinking (DT) approach can be applied in the processes of teaching programming to gifted students and to reveal its effects on the teaching process. The case study method was used. 5 different DT tasks were defined to create solutions for an unstructured problem by using programming tools and DT processes. DT activities were applied to 25 gifted students (13 girls, 12 boys) at the Science and Art Center (BILSEM) in the city center through the summer term. Data were collected through interviews, observation forms, and the DT Rubric which was developed by the researchers. The findings showed that gifted students improved their DT skills to a certain level, learned the academic content, enjoyed the process itself, and experienced some problems working in teams. At the end of the teaching process, the students emphasized that a good designer should be a respectful person who can work well within a team. Additionally, according to the students' views, different programming tools and environments namely Scratch, Arduino IDE and Lego Mindstorms EV3 can be used in the prototyping phase of the DT processes. Updating DT tasks to include DT mindsets and taking into account the leadership qualities of gifted students during the implementation process may be suggested.

INTRODUCTION
Gifted students are those who perform at a higher level than their peers in at least one of the following areas: academic areas such as general cognitive skills, mathematics, and science; psychomotor skill areas; creativity; leadership; and/or visual or performance-based art (Clark, 2015;Marland Report, 1972;Ministry of National Education [MoNE], 2018; National Association for Gifted Children, 2019). According to Renzulli (1978), there are three sets of features that define gifted individuals, namely general and special talent, motivation (devotion to the task), and creativity. It is an obligation rather than a responsibility to provide this elite group of people who perform or have the potential to perform better than their peers with education services that they need and will be able to develop their talents in (Öngöz & Sözel, 2018).
Strategies of grouping, acceleration, and enrichment are used in gifted student education. Gifted students have the chance to work with students who possess similar characteristics to them by applications of grouping (Kanlı, 2008). The leading role in the education of gifted students in Turkey belongs to the Science and Art Centers (BİLSEM) and their grouping samples (Ataman, 2004). The scope of acceleration activities includes practices such as, class skipping, early school entry, participation in upper classes in a specific area or areas, and narrowing the curriculum (Gür, 2017;VanTassel-Baska & Brown, 2000). The range of educational programs will be extended for enrichment purposes and students will be given the opportunity to focus on specialized topics and activities (Sak, 2014). Some examples of enrichment are a teacher inviting a person who is a specialist in a certain area to class in order to provide students with in-depth information, or a teacher giving research articles to students which are not a part of the current curriculum (Şahin, 2015a).
The concept of differentiation is used as an umbrella concept which covers all grouping, acceleration and enrichment strategies in the education of gifted individuals. Differentiation is a learning experience where distinct ways to explore the curriculum is used to enable students to understand the material, the teaching process and activities are coordinated in a way that allows students to make substantive learning and construct the knowledge, and choices are presented to students to allow them to show what they have learned in different ways (Tomlinson, 1995). In other words, to meet the learning needs of students who have different learning skills, interests, and readiness levels, differentiation can be applied to an education program's content, process, environment and product dimensions (Tortop, 2015). Taking into account the students' interests, learning styles and prior knowledge, the content can be differentiated. The learning environment can be differentiated in terms of material, time and space. The process dimension is influenced by the paths and methods students use in thinking and using information while participating in the learning process. Differentiation can be realized in the process dimension in terms of strategy, system, methodology, high-level thought, research, self-expression, and self-regulation skills. The product can be differentiated according to criteria such as depth, originality, unusualness, innovation, and is related to the real world (Şahin, 2015b, 2018a, 2018b).
By applying these models, it is aimed that gifted students will find creative solutions to unstructured problems in a social context that is close to them, and they will develop high-level thinking and metacognitive skills (Alhusaini, 2018). Design thinking is one of the models that serve to the aforementioned targets.
Design thinking is a human-centered approach that aims to find creative and innovative solutions to various social and commercial problems by using design tools and mindsets (Kelley & Kelley, 2013, Lor, 2016. During the process of design thinking, learners work on targets that are not clearly defined and unstructured problems that have no solutions stated yet (Jonassen, 2000). The effectiveness in bringing 21st-century skills and characteristics to students creates the educational value of design problems (Koh, Chai, Wong & Hong, 2015).
The design thinking process relies on the principles of empathizing in order to understand user needs, defining the needs, making trials, prototyping, receiving feedback from users, redesigning the process (Darbellay, Moody & Lubart, 2017) and expressing yourself through creative ways besides using words and symbols (Brown & Wyat, 2010). Since there is no common description of design thinking in literature, there is no single way to follow the design thinking process. Institutions such as Stanford University Hasso Plattner Design School, IDEO and Design Council have developed many design process models. In all models; collecting information in order to understand the problem, using creative thinking skills in the process and being experiential during the process were always highlighted (Chesson, 2017). "D.school" describes the design thinking process phases as empathizing, defining, brainstorming (ideate), prototyping, and testing, respectively (Bootcamp Bootleg D.School, 2011). The design thinking process is a iterative process, and phases can be repeated one after the other if needed (Lor, 2016). In Figure 1, the stages of the design thinking process and their relations with each other are shared. The design thinking process starts with empathizing with the user. Insights of what people feel and what people think will be developed through empathy (Carroll, 2015). In order to develop insights, students observe how people behave, and how they interact with other people and the environment. Also, they can record projections regarding the answers to questions asked. This way, empathy will be established with the user (Carroll et al., 2010). When students empathize, they are not limited by their own experiences. On the contrary, they can develop new ideas and products through internalizing others' experiences (Root-Bernstein & Root-Bernstein, 1999).
The second phase of design thinking is defining the need. In this phase, action-based problem statements will be stated after analyzing and synthesizing the data obtained during the empathy phase. Problem situations are expressed as Point of View (POV) statements which are formed by combining "User + User's Need + Insight" (Carroll et al., 2010). The problem defining phase supports creative thinking skills in the context of evaluating a situation or problem from different angles, redefining present models and enabling the production of new information by developing multiple points of view (Henriksen, Richardson & Metha, 2017).
Brainstorming is a phase aimed at producing many ideas in various categories devoted to finding a solution for the defined problem. Students can participate in brainstorming processes in different groups or individually (Painter, 2018). In the Prototyping phase, a solid product is put together by taking action on the new ideas that were developed in the brainstorming phase.
Any kind of thing that has a physical component such as an object, role-play activity, an interface, a visual scenario draft inherits the feature of being a prototype (Bootcamp Bootleg D.School, 2011;Carroll, 2015, Odabaşı, Dursun, Ersöz & Kılınç, 2018. In the testing phase, which is the final stage of the design thinking process, the user is able to experience the developed prototype and give feedback to the designers. Solutions developed according to user feedback are evaluated and will be improved accordingly (Carroll, 2015). Additionally in this phase, gathering more information about the user and the improvement of POV statements after testing can be realized (Bootcamp Bootleg D.School, 2011). All the actions executed by the designer or the designing team in the design thinking processes are supported by the individual's attitude and mindsets. These processes are effective on the qualification of design thinking (Carroll, 2015;Chesson, 2017;Lor, 2016). In literature, design thinking mindsets are stated as follows; people-oriented, action-oriented, metacognitive awareness, multidisciplinary understanding of cooperation, open-mindedness, tolerance to uncertainty, teamwork, risktaking, learning-oriented, learning from mistakes/experiences, and creative self-confidence (Dosi, Rosati & Vignoli, 2018).
In literature, design thinking is generally considered to be a teaching method and skill set, and it has been proven that the design process has positive effects on students' academic learning, cognitive, affective and social skills. Students who experience the design thinking process have the opportunity to learn about the specialties of the designers, academic content, design process, teaching-mentoring knowledge and can improve their academic success. At the end of the design thinking process, learners develop creative thinking, critical thinking, problem-solving, design thinking, meaningful learning, and metacognitive skills. In the design thinking process, students have the opportunity to progress their affective skills such as empathy, creative selfconfidence, risk-taking, assertiveness, self-sufficiency of knowledge generation, curiosity, being human-oriented and setting a career goal. Furthermore, social skills like working in cooperation, self-expression and having social interaction of learners can be improved during the design thinking process (Aflatoony, Wakkary & Neustaedter;Bouchard, 2013;Carroll et al., 2010;Carroll, 2015;Duman & Kayalı, 2017;Henriksen et al., 2017;Koh et al., 2015;Kwek, 2011;Noel & Liub, 2017;Scheer, Noweski & Meinel, 2011;Rauth, Köppen, 6 Design thinking can be applied in different contexts such as mathematics (Painter, 2018), graphic design (Duman & Kayalı, 2017), social sciences (Koh et al., 2015), STEM education (Carroll, 2014) and teacher education (Carroll, 2014;Odabaşı et al., 2018). One of those contexts is computer science education which aims at revealing the designer, developer and active participant characteristics of learners and which is addressed with works of developing solid products and software developing (Kert, 2018). During software development (in other words, the programming process), students understand the nature of unstructured problems by cooperative work, empathize, define problems, and establish, develop, and improve their program to find new, appropriate and useful solutions (Romero, Lepage & Lille, 2017).
Programming has been thought to the gifted students of primary, secondary and high schools in the Information Technologies and Software lesson at BİLSEM. In this scope, framework programs and activity books are prepared to be included the programming skills and served as a guide to the teachers who work at BİLSEM (MoNE, 2017). BİLSEM Information Technologies and Software Lesson Framework Program can be criticized for not having any differentiation strategy, and for designing activities as technology-oriented rather than pedagogical. Because technology-oriented learning-teaching processes that lack pedagogical aspects lead to inefficiency during programming education (Kert, 2018). It can be stated as a deficiency not to execute example activities in the teaching programs that are designed for the gifted students who are studying in groups by formal education (MoNE, 2019). Example activities designed for gifted students are a necessity for the BİLSEMs that are in the phase of restructuring (Ayverdi, 2018). Building and sharing activities with teachers, that utilize differentiation strategies for gifted students, eliminate pedagogical deficits, focus on accomplishments and provide meaningful learning experiences are incredibly important. In addition, designed activities should be evaluated by empirical researches (Plucker & Callahan, 2014).
In the scope of this study, design thinking is discussed as a differentiated teaching method and skillset and applied during programming education processes for gifted students. The aim of the study is to present how a design thinking approach can be applied in programming education and to reveal its effects on the teaching process.

Research Design
The study examines how design thinking is applied during programming education processes for gifted students. As in-depth examination of the DT experiences of the gifted students is intended, case study, which is one of the qualitative research methods, was used. A case study is an in-depth description and analysis of a bounded system. A bounded system can be single person who is a case example of some phenomenon, a program, a group, an institution, a community, or a specific policy (Merriam, 2014).

Participants
Activities included 25 gifted students who studied at Science and Art Center (BİLSEM) in the city Centre for one summer period. The study group consisted of 13 male and 12 female students. At the time of the study, 8 of the students were in 5th grade, 7 of them were in 6th grade, 3 of them were in 7th grade and 7 of them were in 8th grade.

Data Collecting Tools
Design thinking worksheets (Empathy Map, POV statements, User Feedbacks), note sheets used in the brainstorming process and prototypes developed during the DT process were examined with DT Rubric (Annex-4) developed by the researchers. The steps proposed by Andrade (2000) and Mertler (2001) of DT Rubric were used. Firstly, 5 criteria were identified as performance criteria, evaluating each step of the design thinking process (empathy, defining, and brainstorming, developing prototypes, testing). Secondly, it was decided the type of DT rubric as to be "analytical rubric". To determine whether the students gained the DT skills while performing the DT tasks, not only the products they developed, affected this decisionmaking process. Afterwards, performance levels were determined and level definitions were cleared. Performance levels were determined from the weakest to the most competent by scoring the lowest level of performance as 1 and the highest level of performance as 4. After collecting the reviews of 8 gifted students, 1 teacher and 2 experts working on DT field, rubric was finalized. During the implementation of activities process, use of design thinking skills by gifted students is examined by DT Observation Form (Annex-5). The observation form was developed by researchers using the resources on the online platform "d school K12 Lab". Semistructured interviews were conducted with gifted students before and after the implementation 8 of the activities. One student from each grade level (5th, 6th, 7th and 8th grades) participated in the interviews. In the interviews; questions such as how are they going to structure a design project, what kinds of tools they will use for prototyping, what they think about possible problems may be encountered during design process and what they think about the personality traits that a member of design team should inherit are directed to students.

Design of DT Activities
The activities in this study were designed to be used in information technology and software course for gifted students. There were five study units which covered block-based, text-based, physical and mobile programming, and at the end of each study unit, students were asked to develop unique projects according to the programming environment covered in each module.
Prior to the project development process, the students received technical training regarding design thinking, block-based, text-based, physical, and mobile programming. The achievements of the activities are as follows: 1. Develop an original project in a block-based programming environment using design thinking process. For each achievement, "Design Thinking (DT) Tasks" were also determined which aimed to attract students' curiosity, increase participation in activities, and providing opportunities to demonstrate information processing skills also related with daily life as well. These tasks are stated as follows respectively; "redesigning the experience of playing computer games", "redesigning the healthy eating experience", "redesigning the experience of water use", "redesigning the experience of living safely" and "redesigning the learning experience in science and technology lesson". In all DT tasks; DT digital presentation, Empathy Map

Application Process
Each of the activities lasted six lesson hours and was administered to gifted students for one day at Science and Art Center (BİLSEM). Prior to the exercise, the Design Thinking process was reminded to the students by using a digital Design Thinking Presentation. After that, students were divided into 5 groups of 5 people. Students were given their DT task, Empathy Map, Point of View and User Feedback templates and other tools related to DT task. Students were required to name their group and hold a small discussion about the DT task. During the activities process, phases of the DT process were followed.

Execution Phase-Empathy
Students are told that their group is a "design team" and will work on the DT task. A volunteer in the design team is selected as "user". The other four people are called "designers". Designers will have an interview with the user to empathize. The task assignment is done by the designer students for the interview. Students take on the duties of 1) the main interviewer 2) the person asking continuity questions (such as "For what?" and "Why?") 3) note taker and 4) time keeper.
Students are given time to prepare the interview questions (10 minutes) and to conduct the interview (25 minutes). To monitor timing "Online Stopwatch Web 2.0 Tool" is used and projected onto a screen visible to students. Students are expected to complete the empathy map within fifteen minutes at the end of the interviews. At the end of this process, one spokesperson from the groups is asked to briefly summarize the empathy map to the whole class, and revisions are given to the groups about their inferences on their empathy map. If needed, students make revisions on their empathy maps. An example interview questions prepared by students for the first DT task during the phase of empathy is shown in Figure 2, and the empathy map is shown in Figure 3.
When Figure 2 is examined, it is seen that students try to ask open-ended questions which may reveal user experiences, but they do not use continuity questions as much. When the empathy map showed in Figure 3 is examined, it is seen that the students place what they say and do in the left section of the template, and in the right section they make deductions according to the information they get from the user.

Execution Phase -Defining
This is the phase where the problem or need is defined. Groups filled their POV worksheets in 15 minutes. In the Defining phase, students are asked to write down at least 3 POV statements.
Each group is required to share POV statements to the class by one spokesperson and a discussion is held to check if the statements are correct POV statements. After that, students are asked to select one POV statement and proceed to the prototyping phase with that. When the POV template given in Figure 4 for the first DT task is examined, it is seen that the students write the user's need in the first part of the template by using verbs and in the second part they develop insight towards the need written in this statement.

Execution Phase-Brainstorming
After students write their POV statements, they do brainstorm to provide a solution for the need that presented itself in these statements. Before starting the brainstorming phase, students are reminded that they can develop each other's ideas (in their group), that ideas should not be judged, that the quantity of ideas is important, and that it is important for everyone to listen to each other's ideas. Students are given 10 minutes to produce at least 25 ideas. The students write down their ideas on sticky papers and cluster the papers in a dedicated area. Afterwards, if they wish to do so, students can cluster their ideas into categories. The images of the students' process of brainstorming are shared in Figure 5 (a, b). The activity leader(s) check whether the ideas that the students produce are solutions to the needs of the user and count the ideas. All members of the group which produced the most ideas receive a badge called "Ideate Card" by the activity leader(s). At the end of the brainstorming process, the group members put their signatures on the 3 ideas they liked the most. Thus, the idea on the sticky paper with the most signatures is determined and prototype development phase starts. a b Figure 5. Brainstorming process

Execution Phase -Prototyping
Students develop prototypes for their chosen ideas at this phase by using the appropriate programming environment as well as tools and materials that are suitable to the DT task given.
The prototypes developed by students for the DT tasks are presented in Figure 6 (a, b), Figure   7, Figure 8 (a, b), Figure 9 (a, b) and Figure 10 (a, b). a b Figure 6. Example prototypes developed for 1st DT task Figure 6 shows the prototypes developed by students in the Scratch block-based programming environment for the DT task. In Figure 6-a, a fast-progressive game was developed which has objectives and a story for the user. In this prototype, the user is imprisoned in a room. By using directional keys to move, the user tries to get to the key without touching the police character.
When the key is taken, the door of the room opens and the character becomes free. The group who designed the second game developed a survival game for their users. In this game, the user is hunting in a forest, collecting goods, building houses, lighting fire and so on to survive (6b).

Figure 7.
Example prototypes developed for 2nd DT task Figure 7 shows the codes of the program that the students developed in Python programming language to redesign the healthy eating experience related to the second DT task. When the codes are examined, it can be seen that the user is asked to input their height and weight information. According to the information entered by the user, the program calculates the body mass index and returns feedback to the user. The user is then asked about the calories of the meal he or she eats for one meal course. According to this information, a suggestion is provided to the user regarding which sports to do and how long they should do it for in order to burn the target number of calories. A food menu is also offered to user. In Figure 9-a, a safe home system was developed, and in Figure 9-b, a robotic system was developed which can communicate with the police quickly using wireless technology in emergency situations. a b Figure 10. Example prototypes developed for 5th DT task In the first prototype, a mobile application was developed that asks the user questions about science and technology lesson contents when the user shakes the phone, and gives feedback to users regarding their answers (Figure 10-a). In the second prototype, an application was developed that asks the user to input functions of organelles in the cell, and after the user provides their answer, the app informs the user according to the information entered ( Figure   10-b).

Execution Phase-Testing
In the testing phase, the developed prototype is presented to the user and the user's feedback is received. This process provides information regarding whether the developed prototype meets the needs of the user or not. In the testing phase, there are some rules that both the designers and users need to pay attention to. Designers are required to briefly explain their prototype without sharing details, and allow the user to interact with the prototype without interrupting.
They observe user carefully. Users; express their views about the prototype clearly and in a straightforward way. They explain all of the details they liked and disliked, and they behave clearly and honestly in this process. Group records the feedback of users by using the User Feedback Template. After reviewing the feedback, they received and making revisions on the prototype, their work is presented to the user again, and then the final version of prototype is shared. Figure 11 presents an example of a user feedback template filled in for the 1st DT task.

Data Analysis
The DT rubric was scored by 2 instructors during the implementation of the activities and compliance between the raters was evaluated. The intra-class correlation coefficient (ICC) was calculated for the compliance between the raters and this value was obtained as .825. According to Koo and Li (2016, p.158), the ICC found between 0.75 and 0.90 is indicative of sufficient reliability. The boxes in the DT Observation Form are marked with the level of which gifted students are involved in the DT phases. For DT observation form, the compliance between the observers was examined by calculating "Weighted Kappa Coefficient" and this value was calculated as .68. Accordingly, there is a sufficient level of compliance among the observers and this value shows that results are purged from the chance factor (Şencan, 2005). Conducted interviews examined through content analysis.

FINDINGS AND DISCUSSION
Design thinking worksheets (Empathy Map, POV statements, User Feedbacks), note sheets used in the brainstorming process and prototypes developed during the DT process were examined with DT Rubric. In Table 1, the scores obtained from the criteria of DT rubric are shared. Table 1 Groups' Scores from Each Criteria of DT Rubric  understood that students exhibited behaviors at Level 2 (n = 2), Level 3 (n = 14) and Level 4 (n = 9). Again, in the defining phase, students showed behaviors at Level 2 (n = 4), Level 3 (n = 16) and Level 4 (n = 5). In the process of brainstorming and prototyping, students presented the behaviors in Level 3 and Level 4. 12 students are in Level 3 and 13 students are in Level 4 in the phase of brainstorming. During the prototyping phase, 17 students are at Level 3 and 8 students are at Level 4. In the last phase of the test; 8 students showed behaviors belonging to Level 2, 11 students in Level 3 and 6 students in Level 4.
During the semi-structured interviews with four gifted students both prior and after the activities, they highlighted the tools of prototyping, DT process and features of designers. After execution of activities, as distinct from their prior thoughts, students mentioned that they learned the process, shared the results, enjoyed the experience, made their POV statements and repeated the phases during the process. As for prototyping tools; programming tools like Scratch, Appinventor etc., 3D design software and easy-to-find tools (cardboard, drinking straw, crepe paper, etc.) are mentioned. According to the students' views, a good designer should be able to empathize, work with teams, value the thinking process, and respect others' ideas. The fact that these characteristics of designers are mentioned only after the execution of the activities can be explained by effects of activities on the students' thoughts about the designer characteristics.
There is no study in literature that states and search upon DT activities implemented on gifted students specifically and the effects of activity achievements to students. Ayverdi (2018)  In other studies, as a result of the DT process, students learned academic content (Carroll et al., 2010;Carroll, 2015;Kwek, 2011;Painter, 2018) and had problems working with the team students have chance to improve 21st century skills in the phases of empathy, defining, brainstorming, prototyping and testing (Henriksen et all., 2017). Many studies provide evidence that DT activities improve 21st century skills (Carroll, 2015;Diefenthaler, Moorhead, Speicher, Bear and Cerminaro, 2017;Koh et al., 2015;Scheer et al., 2011). The development of students' DT skills after the implementation of the activities in this study is also a supportive indicator of this situation.

CONCLUSION AND SUGGESTIONS
After the execution of the DT activities, students stated in the interviews that they enjoyed DT activities, they learned the process, they had small arguments with team members during design process and it is hard to work with a team. After participating in DT process, students shaped the characteristics of a good designer according to their DT experiences, and concluded that a good designer should be able to work with a team and should be a respectful person. It is thought that obstacles experienced regarding the team work may result from almost all gifted students' tendency to be a leader and desire to manage team on their own. Additionally, the situations of some students being outsider to team work, not fulfilling their duties and not respecting to opinions of peers caused to having problems. Based on teacher observations, majority of students exhibit the behaviors of Level 3. Scores of students obtained from DT Rubric are also very high. At the end of execution of activities, it is observed that gifted students have improved their DT skills, learned the academic content and experienced some problems regarding working with the team.
For further studies, activities may develop to place DT thinking ways in detail. Students' roles like teamwork, participation in the DT process may be observed in detail more. The process used by the gifted students to reach the solutions for the design problems and their creativity levels can be investigated in detail. During the interviews with the students; collaboration can be portrayed like solidarity, good or bad in teamwork, contribution to the team and so on. The maximum number of students in a group should be 5 and the leadership skills of gifted students should be taken into account in group work. To increase the motivation of students in the process of brainstorming, the principles of gamification can be utilized such as giving badges to the group that produced the most amount of ideas. While DT is applied in programming teaching processes, block-based programming environments such as Scratch, Mblock and App Inventor, text-based programming environments such as Python, Arduino IDE, and for physical programming Lego Mindstorms and Arduino tools can be preferred.

Annexes Annex 1. Empathy Map Template
Group Name : Lesson : Information Technologies and Software Activity Name : Instructions: After the interview, please fill out the "Empathy Map" with your group members.

What They Said / Done My Thoughts / Feelings
Interviewer's observations that are seen and heard stated in this section. The sentences that come out of the interviewee's mouth are included in the quotes without being changed. ...
Based on the data in the what they said / done section, deductions are made about the user and assumptions are noted. ...

Annex 2. POV Template
Group Name : Lesson : Information Technologies and Software Activity Name : Instructions: Make POV statements by completing the sentences in the table below.

Annex 3. User Feedback Template
Group Name : Lesson : Information Technologies and Software Activity Name : Instructions: Record user feedbacks by filling out below table.

What worked out?
What did not work out?

… …
What should be improved? What has surprised us? … ….