Validation of the Students ’ Cognitive Engagement Measure : Evidence from University Students of Karachi

The goal of the study is to determine the validation of the Cognitive Engagement measure of students in the context of Pakistan. The rationale for such an investigation is that while enrollments in Pakistani higher education are increasing, passing out numbers are very low; this problem necessitates an investigation of student cognitive engagement as well as other factors. Because the Construct of Engagement arose in response to the issue of lower student turnout. The Cognitive Engagement Subscales of Attitudes towards Mathematics (ATM) were used to collect data for this study, and the convenience sampling technique was used. The sample included 528 university students from ten different universities in Karachi. The Structural Equation Modeling (SEM) approach was used for data analysis. The results revealed 21 items that were compatible with the Pakistani context, with Shallow Strategy Use (SSU) being the most endorsed subscale with a mean of 3.7. This study provided a validated instrument for future studies to determine cognitive engagement levels. Future research should look into other disciplines and the relationship with other variables.


Introduction
Student engagement is a concept that represents the degree to which a student actively engages in learning activities (Fredricks et al., 2011). The concept of student involvement has received a lot of attention in education research, policy, and practice in recent years. This might be because of its documented connections with desirable scholastic and non-scholastic outcomes such as academic success (Reyes et al., 2012), school completion (Archambault et al., 2013), and physical and psychological well-being (Steele & Fullagar, 2009). Natriello was one of the first researchers to establish a formal explanation of the concept of student involvement (Mosher & MacGowan, 1985). Engagement, according to Natriello (1987), "exists when pupils participate in the activities given as part of the school curriculum.
However, the problem has mostly been explored via the lens of disengagement, which appears as absenteeism (i.e., unexplained absence), indifference (i.e., a low level of effort), and criminality (e.g., cheating, stealing). Several factors such as students' origin, learning environment, and school policy can influence involvement, which in turn affects students' academic achievement and social behaviors (e.g., disrupting classroom activities).
Although Natriello (1987) conceptualization of student involvement as a purely behavioral variable comprised of school participation and behavior was limited, it did open the door for future conversations in the education field. Researchers have sorted out or derived models from motivational theories and models, therefore it is not a novel construct. The notion of Engagement, like motivational theories, has three dimensions: emotional, behavioral, and cognitive.
Researchers have sorted out or derived models from motivational theories and models, therefore it is not a novel construct. The notion of Engagement, like motivational theories, has three dimensions: emotional, behavioral, and cognitive. Interest, assimilation, attentiveness, and active participation in learning activities are all examples of cognitive engagement, as is the desire to learn new things more than academic requirements. Because engaging or disengaging students cognitively affects their behavior and academic engagement (Fredrick's & Paris, 2004). Cognitive engagement plays a dual role. In the last ten years, researchers have begun to focus on the concept of student engagement.
University enrolment in Pakistan started increasing since 2001-2002 in the subsequent years; during 2010-2015 Pakistani higher education witnessed a 78 percent increase in the number of universities and degree-awarding institutions, consequently, student enrollment increased profoundly by 174 percent (HEC, 2015). The main reason was a change in government policies towards higher education. The private sector was provided incentives to enter (Economic Survey of Pakistan, 2015) and mushroom growth of private universities started. That increase in the number of universities led to an increase in enrollment to great extent; nevertheless, the enrollment rate is still lower than in other South Asian countries (HEC, 2015). See Table 1 for detailed enrollment information of respondents. However, it evident from statistics available in different HEC publications is that student turnout has been decreasing (Sultana, 2017;Muslim et al., 2017;Zaheer et al., 2016). This could be due to higher dropout rates (Khanam et al., 2016) or low student achievement. Moreover, since proper planning and long-term perspective was not kept into consideration, this increase in human capital has not significantly contributed to economic development (Asghar & Zahra, 2012). Such a problem "exerts a harmful effect on the investment in education by creating a non-productive and non-innovative environment" (Khanam et al., 2016).
This issue of lower student turns out necessitates an investigation of the level of student cognitive engagement as well as other factors that contribute to the low passing rate. Few studies in Pakistan, have included cognitive engagement aspects in part, such as Ullah et al. (2014), who investigated three cognitive engagement variables: in-depth, surface learning, and monitoring in their study. In Items 21, 22, and 23, Iqbal et al. (2009) included cognitive engagement factors. Similarly, some researchers, for instance (Nomaan et al., 2016;Amir & Kamal, 2011;Nausheen, 2016;Munir & Rehman, 2016), have used the Motivated Strategies for Learning Questionnaire (MSLQ) in their studies, that investigated subscales of cognitive engagement, self-regulation, elaboration, and cognitive strategies, among other variables (Fredricks et al., 2012;Pintrich, 2004). It is worth noting that all Pakistani researchers did not use the term "cognitive engagement." The primary purpose of this study was to confirm the appropriate factor structure of the Cognitive engagement scale in the context of Pakistan, in order to meet the validity and reliability (Schimmenti et al., 2020) by employing structural equation modeling.

Background of Study
Higher education in Pakistan is in a precarious state. It is not because people are inherently inferior in talent or moral values in comparison to any other nation in the world, but because of long colonial rule and imitation of other nations' systems, as well as political imbalance, that have spoiled some of the virtues and brought a bad name to the people's intellectual capacities. In Pakistan's history, university education got off to a shaky start. When Pakistan was founded in 1947, it had only one university: the University of Punjab in Lahore. Karachi University was established in 1950. Universities grew in the years that followed. The Lahore Institution of Management Sciences (LUMS) was the first private university in Pakistan, founded in 1984, followed by the Agha Khan University in 1985. The first significant growth in the number of public universities occurred under the Zulfiqar Ali Bhutto era, from 1971 to 1977 (Hoodbhoy, 2009).

Research Questions
Q.1 Is the Cognitive Engagement scale valid and reliable for Pakistani university students? Q.2 What is the widely used cognitive engagement strategy among Pakistani university students?

SAL and SRL Approach
Although the domain of motivation and learning for higher education research comprises several models and viewpoints "A key distinction in the field has been the contrast between two general perspectives".

• Student Approaches to Learning (SAL) • Information Processing
Most European and Australian researchers opt for the SAL approach whereas the IP approach is more common among North American researchers. It is noteworthy that although various models have been derived from the IP approach in the recent era Self-Regulated approach (SRL) has replaced the IP approach. The rationale behind this replacement is that SRL is more comprehensive as opposed to the IP approach that only includes cognitive factors. Whereas the SRL approach includes motivation and environmental factors as well (Pintrich, 2004).
It appears that different research traditions have become closer in recent years. Researchers from many traditions are now accepting current themes that emphasize active, constructivist, situational, and collaborative components of learning (Lonka et al ., 2004). There is the emergence of another approach that is student engagement, it is not new rather separates the models from both SRL and SAL perspectives. Student engagement has the same three dimensions as motivational theories say, affective, behavioral and cognitive. A clear definition of (CE) is not clearly defined in available literature (Fredricks et al., 2016). Although, researchers following the SAL perspective describe CE in terms of deep and shallow cognitive strategies.

Cognitive Engagement
Because "cognitive and emotional engagement is potentially mediators of academic and behavioral engagement" (Reschly & Christenson, 2012), engaging or disengaging students intellectually and effectively precedes their behavior and academic involvement.
Although there is little study on cognitive engagement, it focuses on broad concepts like thinking and the desire to put in the work required to achieve and learn complicated abilities and ideas. Following are the key components of student cognitive engagement

Self-regulation
Certain key assumptions of self-regulated learning were outlined by Zimmerman (2000). Although there are many different Self-regulation models, Pintrich claims that this assumption is the same in all of them. The first assumption is that learners actively engage in knowledge acquisition, making attempts to use both accessible resources and past knowledge. The next assumption is that a student has some influence over their learning processes and occasionally external circumstances as well, or that learners can self-regulate their learning process to some level (Pintrich, 2004).
The next premise is that a learner has a purpose to achieve and to do so, they manage their cognition, motivation, and behaviour" to govern their self-regulatory, motivational, and behavioral techniques and activities (Vermetten et al., 1999). The last premise is that a learner does not utilize her self-regulated methods just because of her personal history and priorities, nor alone because of the learning environment. To achieve her aim, a student uses self-regulated learning to relate her goals or objectives to classroom surroundings.
H1: The facture structure of the self-regulation sub-scale is sufficiently valid and reliable with the current sample.

Deep Strategy Use
Deep processing is a term used to describe a learner's efforts to foster the formation of progressively sophisticated structures of knowledge. For instance, if a student is using deeper thought while accomplishing any academic task, the students display elevated levels of (CE). This way of thinking has been related to higher academic accomplishment and a better understanding of course material (Entwhisle & Ramsden, 2015;Nolen, 1988;Miller et al., 1996;Garcia & Pintrich,1991;Schunk,1985). Deep processing techniques include rephrasing or summarizing material, usage of drawings or diagrams to help in problem-solving, evaluation of understanding, comparison, and contrast , Hofer et al., 1998.
H2: Facture structure of deep strategy use is sufficiently valid and reliable with the current sample.

Shallow Strategy Use
On the other hand, Shallow engagement, explains a learner's surface-level cognitive participation focused on replicating rather than elaborating on learning content. These tactics are intended at internalizing knowledge as is, rather than adopting metacognitive processes to deepen an individual's grasp of course content. Repetitive repetition and rote memorizing of material, underlining or highlighting text while reading, and reviewing notes are all examples of shallow processing procedures (Zimmerman & Pons, 1986;Miller et al., 1996;Weinstein & Mayer, 1986;Hofer et al., 1998;Meece et al., 1988;Nolen, 1988;Ravindran, et al., 2005).
H3: The facture structure of shallow strategy use is sufficiently valid and reliable with the current sample.

Persistence
Persistence refers determination and hard work of a student towards academic activities in the face of difficulties. High levels of persistence significantly contribute towards the accomplishment of the tasks and/or course completion (Sakurai et al., 2012). Some researchers categorize persistence under behavioral engagement, whereas others as cognitive engagement (Fredricks et al., 2011), such as Miller et al. (1996) referred to persistence as a construct of cognitive engagement. Whereas Skinner and Pitzer (2012) categorized persistence as behavioral engagement.
H4: Facture structure persistence use is sufficiently valid and reliable with the current sample.

Measurement of Cognitive Engagement
Measurement of CE varies in accordance with operational definitions of the construct Cognitive engagement has been defined as (1) perceptions of the importance or value of schooling, learning goals, and prospects; (2) cognitive strategy use (how thoroughly students study material); (3) self-regulatory or metacognitive strategies (how students manage knowledge acquisition such as organizing and seeking information); and (4) doing extra work and going above and beyond schoolwork. Such cognitive engagement measures consider motivation, self-regulated learning, and strategy use. When we go for notions of deep and shallow for CE, we find its roots in the work of Marton (1986) and his fellow researchers in 1970. Lonka et al. (2004) in 1976 Marton and Saljo (1976) coined the terms of deep and shallow learning strategies. To explore the constructs of deep and shallow learning they used qualitative methodology. Marton (1986) developed a qualitative method named phonomyography that uses the lens of students' perspective developing through the interaction of students experience and the learning environment (Lonka et al., 2004). In terms of the value of self-report surveys, self-report data has considerably added to our understanding of motivation and cognitive engagement.
The first handbook on student engagement was published in 2012 wherein Fredricks and McColskey (2012) discussed in detail the surveys instruments that include subscales measuring cognitive engagement along with theoretical origins. In 2015, Green reviewed self-report surveys being used over 20 years, she also reviewed the studies that do not mention the CE but the work they have done falls under the umbrella term CE. Although Greene et al.(2004) follows the SAL school of thought in her review, she brought under discussion the self-report surveys from both SAL and SRL approaches. Such as details about the Motivated Strategies for Learning Questionnaire (MSLQ) developed by Pintrich (1991) belong to the SRL approach.

Research Design
A descriptive method has been chosen for the current study. Descriptive research is defined as a research method that describes the characteristics of the population or phenomenon studied. This methodology focuses more on the "what" of the research subject than the "why" of the research subject (Siedlecki, 2020). However, the study's research topics include the description of university students' fundamental ideas and cognitive activities. Descriptive research is undertaken in many educational settings to "describe, compare, contrast, classify, analyze, and interpret the entities and occurrences" (Cohen et al., 2011) concerning people, practices, or content. A common descriptive method is survey design.

Measure
This study used Cognitive Engagement subscales of Attitudes towards Mathematics (ATM) an extensively validated measure in different cultures and subject domains (Fredricks & McColskey, 2012). As far as the utility of self-report surveys is concerned "self-report data have made significant and important contributions to the understanding of motivation and cognitive engagement" (Greene et al., 2004). This measure consisted of four subscales: self-regulation (9 items), deep strategy use (7 items), shallow strategy use (4 items), and persistence (8 items). Comprises 5-point Likert scale ranging from strongly disagree to strongly agree.
The Cognitive Engagement subscales of the Attitudes towards Mathematics (ATM) were used in this study, which is a widely validated measure across cultures and subject domains (Fredricks & McColskey, 2012). In terms of utility, "self-report data have made significant and important contributions to the understanding of motivation and cognitive engagement".
This scale had four subscales: self-regulation (9 items), deep strategy use (7 items), shallow strategy use (4 items), and persistence (8 items). A 5-point Likert scale ranging from strongly disagree to strongly agree is used.

Sample
The population of this study comprises all university students of Pakistan. For the current study, the convenience sampling technique has been employed, although it is a non-probability sampling technique still extensively used in social sciences research. The results of the convenience sample analysis can only be applied to the research participant group. It is critical to note that connections and effects discovered in a convenience sample cannot be applied to a target population. Convenience sampling, on the other hand, is less expensive, faster, and easier than other types of samples. Convenience sampling can be used to create hypotheses and objectives for use in more rigorous research projects when no other sample method is practical (Stratton, 2021).
Moreover, a list of all enrolled students is very difficult to obtain for a trio of researchers with limited time and resources, therefore random sampling was not a feasible option. Data were collected from 6 public and 4 private universities in Karachi city, 600 forms from 650 were returned, 528 were complete and free from common errors. The response rate remained 81%.
In order to confirm the validity of the cognitive engagement scale; exploratory factor analysis was conducted with SPSS software. AMOS is a statistical software programme created by IBM. The Amos programme is specially developed to aid in the testing of hypotheses about the relationship between variables. We may use this programme to determine the strength of the association between variables, including latent and manifest variables. How substantial is the link between variables, and how well does the hypothetical model suit the real-world data? The benefit of Amos is that we don't need a sophisticated syntax or computer language to use it (Purwanto et al., 2021).

Results and Data analysis
Before initiating factor analysis in order to see whether missing data can be imputed or not for the reason being that AMOS requires complete data set for analysis, Missing Completely at Random (MCAR) test was performed, which yielded insignificant for all four variables i.e., self-regulation, deep strategy use, shallow strategy use, and persistence. Insignificant MCAR test refers that missing data pattern is random and can be imputed (Little, 1988). Then expectation-maximization procedure was employed to impute missing data.
Research question two is about the mean level of endorsement of the constructs. It is shown from table 3 that shallow strategy use (SSU) has the highest mean value (3.74) as compared to other constructs of higher-order thinking; self-regulation (3.67), deep strategy use (3.57), and persistence has the lowest value (3.11). It implies that students included in the sample are employing shallow cognitive strategies and have lower rates of persistence. Table 3:

Descriptive Statistics
Initially, exploratory factor analysis was conducted Orthogonal Varimax rotation was used as was in Miller et al. (1996) to scrutinize the dimensionality of factors. The value of Kaiser-Meyer-Olkin (KMO) 0.86 was sufficient to conduct analysis (Leech et al., 2005;Barkus et al., 2006). The Bartlet test of Sphericity was found significant at 0.05, sufficient to reject the null hypothesis. Items having lower loadings (k < 0.30) were deleted (Bandalos & Finney, 2018). A total of 21 out of 28-factor loadings were identified to be sufficient to moderate.
"Internal consistency is the amount to which individuals who answer in one manner to things tend to respond in the same way to other items meant to test the same construct," says the author (Fredricks & McColskey, 2012). If Cronbach's alpha is 0.07 or greater, reliability analysis is acceptable (Leary, 2004). In this study, the values were self-regulation (0.80), DSU (0.73), SSU (0.77), and persistence (0.77). (0.72). The CR values in the table are all more than 0.7, indicating composite measure dependability (Raykov, 1997).

Construct Validity
Confirmatory Factor Analysis was used in the second phase to validate the factor structure since it "is a good method for evaluating construct validity" (Kline, 2005). Different measures were employed to assess model fit, the Comparative Fit Index (CFI), Root Mean Squared Error of Approximation (RAMSEA), Goodness of Fit Index (GFI), and Adjusted Goodness of Fit (AGFI) (Fan & Sivo, 2005), RMSEA (.058) and RMR (0.062) supported model fit (Hu & Bentler,1999), while GFI (.912) and AGFI (.887 >0.8) also indicate good model fit (Kline, 2005;Anderson, 1995), and TLI (.878) and CFI (.89) around 0.90 show satisfactory fit (Kline, 2005;Anderson et al., 1995). Table 4 shows the results. Validated convergent and discriminant validity (Kline, 2005) results appear in table 3. Inter-factor correlations ( Table 3) and figure 1 were determined to be weak to moderate. The results show that SSU is the most approved subscale, with a mean of 3.7.

Discussions and Conclusion
In this study negative relationship was found between persistence and shallow strategy use. This relationship is consistent with literature although not direct instead it is needed to understand the relationships of another construct. There are certain predictors/and or associated variables of shallow or surface-level strategies such as performance goals i.e., learning for the sake of good grades to get appreciation from family, and peers/teachers instead of the zest of learning . In this regard, it is argued students who are inclined towards performance goals, when confront with difficulties they show low persistence and their performance falls, and they are more at risk of dropping out. On the other hand, students interested in learning goals tend towards deep strategy use, self-regulation and even in the face of difficulties show higher levels of persistence, for the reason being that they feel pleasure in learning new things and remain open for challenges as well (Miller et al., 1993).
As far as mean levels of endorsement are concerned shallow cognitive engagement was found most endorsed. This result is very much consistent with what researchers have been indicating in various studies (Iqbal et al., 2010) another study employing a larger sample of 1850 university students found a lack of self-regulation, loss of confidence, and anxiety problems, although judgment cannot be made mere on student's self-reports, without thorough examination this result at least show students' perspective (Saleem & Mahmood, 2013). When students are facing such problems their risk of dropout increases, and mental discomfort decreases their learning and achievement (Greene et al., 2004;Fredricks et al., 2011).
To some extent existence of shallow engagement is of point of concern because students need to memorize a few things, as well as follow certain patterns as for research writing students must follow certain styles. The problem may occur when shallow engagement becomes problematic when it is a predominant construct as compared to other cognitive engagement constructs fall under the deep engagement phenomenon. It is evident from various research studies that deeper level processing i.e., self-regulation, deep strategy use, and persistence is associated with a higher level of performance. Whereas shallow engagement is associated with lack of interest, low achievement, and higher dropout rates (Garcia & Pintrich, 1991;Greene et al., 2004;.

Academic Implications
Cognitive involvement is an essential notion that has received little investigation in Pakistan and is still developing in the international arena. This research will contribute to both Pakistani and international research. Despite the study's shortcomings, it will equip other Pakistani researchers with well-validated tools. In the future, it is suggested that this instrument be improved with a larger sample size to ensure validity, as well as alternative approaches such as observation and teacher-administered surveys.
In this study shallow engagement was found most endorsed construct, to enhance students' deep cognitive engagement it is needed to collaborate students' expectations from learning activities with classroom instructions. Such as introducing topics/lessons in a way that is in line with students' future, to increase student achievement (Greene et al., 2004).

Research Limitations and Future Recommendations
Because this is a survey study, it is impossible to determine individual or small group concerns in educational settings. In the future, researchers may conduct qualitative investigations to provide a more in-depth study of the underlying condition.
It is impossible to guarantee that replies will be supplied honestly or unbiasedly, as with any self-report survey. Furthermore, because this study used convenience sampling, the results cannot be extrapolated to the target group of university students. Random sampling and longitudinal data studies can contribute a more robust piece of knowledge to the cognitive engagement construct.