The Development of Holistic Assessment in Multiple Linear Regression Literacy

The purpose of this study is twofold: 1) to find out the development process of holistic assessment in multiple linear regression literacy; 2) to develop a valid, practical, and effective holistic assessment in multiple linear regression literacy. Employing Research and Development model, it indicated that holistic assessment model is developed based on preliminary research and development of the assessment as mentioned earlier in multiple linear regression literacy following the revision that was done after the validity test. The development of holistic assessment was strengthened by carrying out a trial to the students at the Study Program of Statistics of Faculty of Mathematics and Natural Sciences, Universitas Negeri Makassar. The trial was conducted in the lesson of multiple linear regression literacy in an integrated manner. Holistic assessment encompasses the information on students' learning outcomes from the cognitive, affective, and psychomotor aspects that are oriented to the lesson of multiple linear regression literacy. This model of assessment is able to monitor the progress of students' learning outcomes. The holistic assessment in multiple linear regression literacy is proven valid based on the result of instrument assessment by the experts. Besides, this assessment model is also considered practical and effective to apply, as signified by the observation of holistic assessment implementation in the lesson of multiple regression linear literacy as well as the result of the trial.


Introduction
Going through the learning process in university is commonly devoted to shaping and building students' competencies, i.e., knowledge, skills, and attitude. Students' learning achievement undergoes an internalization process that involves cognitive, affective, and psychomotor domains. The aspects of learning achievement are quite complex and need to be assessed by using appropriate and accurate instruments.
An assessment of learning achievement is part of the evaluation in the learning process. The assessment is the last part of a learning process that is preceded by planning and implementation; it also should be correctly designed in order for it to be able to give information for the significance of a lesson. On that ground, the planning, implementation, and assessment stages, as a complete unit, should be well-designed continuously. An assessment serves as the most crucial element in the learning process to determine the success level of a process and learning outcomes.
Learning outcomes are achieved after the students take the learning process, in which the outcomes are assessed and analyzed through a process, i.e., assessment. Assessment is an essential component that is inseparable from a learning system. The assessment of students' learning outcomes includes knowledge, skills, and attitude that is performed in balance so that it can be utilized to determine students' position as in accordance with the standard. Assessment components are related to each other, and they are performed to improve learning outcomes.
In statistics subject, the lecturer sometimes assesses the students by merely considering cognitive and psychomotor domains, without even concerning the affective domain. There is a statistical literacy term containing five components that should be taken into account, namely understanding the statistical concept, insight into the statistical application, numeracy and doing graph skills, interpretation capabilities, and visualization and communication skills. Statistical literacy can assist the process of learning assessment by considering knowledge, skills, and attitude in order that the assessment system is more measurable and complete.
The global-oriented holistic assessment takes in the assessment of the knowledge, skills, and attitude of the students.

Research Problems
Based on the above background, the current reality is that there is not yet a holistic assessment that measures the multiple linear regression literacy. By using a holistic assessment, it will cover all three domains The research questions as follows: 1. How is the process of developing a Holistic assessment model on multiple linear regression literacy? 2. How is the holistic assessment model for valid, practical and effective on multiple linear regression literacy?

Literature Review
Measurement is an attempt to specify numbers to individuals systematically. Such a specification intends to describe the characteristics of people's ability in particular fields, thus the measurement result should have errors as small as possible.
According to [1], the purpose of a measurement is to interpret the characteristics of empirical data in an analyzable form. The focus of measurement is providing numbers to empirical data based on certain rules/procedures. The procedures are called the measurement process, i.e., an investigation of characteristics that underlies empirical events and gives numbers over the characteristics. A good measurement result will generate useful data; the data are then processed to become information that functions as a consideration in making policy. However, if the measurement does not reach a good result, the data will also remain poor, no matter what techniques of data analysis are employed. Hence, measurement plays a vital role in various fields, especially education.
After doing the measurement, the next process to be executed is assessment. Suharsimi (in [2] issues an opinion that assessment comprises three objects: (1) input, (2) transformation, and (3) output. Input (of the students) is considered ingredients to be processed; transformation is regarded as a kitchen to process the ingredients, and; output is the processing result to be used.
[3] defines assessment as "an application of various ways by using instruments to get information about the extent to which students' learning outcomes or competencies have been achieved". Putra also mentions the essence of assessment, including (1) students can discover their success in the learning process, (2) teachers can find out which students who can successfully proceed to the next level, and pay close attention to the unsuccessful ones; teachers can understand the appropriate material to be taught along with the learning method, (3) the school is able to find out whether or not the learning condition has met the expectation; the information about the curriculum from teachers can serve as the consideration for the school's future goals; annual assessment results from year to year serve as the school's guideline of the assessment process.
[1] brings out the fact that assessment is part of the learning program evaluation. It is also usually limited to the interpretation of a test score or other measurement results (non-test measurement). An accurate scoring is highly required to gain a correct assessment result so that assessment errors will not occur.
Holistic assessment, according to [4]: "In holistic or global grading, the teacher responds to a student's work as a whole, then directly maps its quality to a notional point on the grade scale. Although the teacher may note specific features that stand out while appraising, arriving directly at a global judgment is foremost. Reflecting on that judgment gives rise to an explanation, which necessarily refers to criteria. Holistic grading is sometimes characterized as impressionistic or intuitive." [5] also state that: The holistic judgments may require smaller cognitive effort from the Decision Maker (DM) in answering questions concerning her preferences than direct elicitation of the value function, e.g.,. However, there is typically more than a single value function compatible with the holistic statements. Obviously, the ranking of the alternatives can vary depending on the compatible value function used, and often the set of compatible functions must be reduced in size by introducing additional preference information to obtain a complete preorder or to determine the most attractive alternative.
Grady (2004) (in [6] claims "holistic assessment entails using a number of assessments and then combining these assessments using a rationale that is defensible to the objective of assessing understanding". Ekawati opines that holistic assessment is able to enhance students' ability in analyzing their works during the learning process. Therefore, the progress of learning achievement will be noticed. This assessment model can create a classroom where its students can criticize themselves. The positive impact of holistic assessment is that the students are able to develop investigation skills, enabling them to encounter any sorts of assessments effectively. Besides, they also can understand the process of constructing the answers to the provided questions. [7] defines multiple regression analysis as a score prediction analysis of two or more independent variables towards the dependent variable in order to prove whether or not there is a functional or causal relationship between two or more independent variables (X 1 ), (X 2 ), (X 3 ), ... , (X n ) and one dependent variable. [8] writes the patterns of multiple regression relationships, which are 1) each independent variable stands alone in influencing the dependent variable. There is no significant relationship among the independent variables; 2) each independent variable does not stand alone, yet they cooperatively affect the dependent variable; 3) independent variables indirectly influence the dependent variable so that there is an intervening variable that bridges the relationship between independent variables and the dependent variable.
Multiple regression analysis encompasses the same completion stages as the stages in simple regression analysis. Despite this, multiple regression analysis has more independent variables and oftentimes makes an assumption. Here are the forms of multiple regression equations.
Two independent variables: � = + 1 1 + 2 2 Three independent variables: � = + 1 1 + 2 2 + 3 3 k amount of independent variables, thus, that the equation will be: Learning statistics subject needs a statistical way of thinking; statistical literacy is utilized to build up such a way of thinking. Students should reach various statistical literacy competencies. According to [9], statistical literacy is the ability to read and interpret data, i.e., the ability to use statistics as evidence in giving arguments. [10] in the article, explains the basic competencies of statistical literacy, as follows: Understanding of statistical concepts. For instance, the concept of mean and standard deviation comprises the calculation formula and conveyed information.
Insight into the application of statistical concepts. Statistical values can be applied in recognizing a situation or condition, and in what situation that specific statistical values do or do not have a meaning.
Numeracy and doing graph skills. Numeracy skill refers to calculating other means and statistical values; in contrast, doing graph is a way easier with the existence of statistical packages and computers. Interpretation capabilities. The calculation result of statistical values will be meaningless if they cannot be correctly interpreted.
Visualization and communication skills. No matter how good the results of statistical analysis and interpretation are, if they cannot be applied to stakeholders, it will be useless.
In their article, [11] describe that: "Someone who has a statistical literacy is equipped to recognize the effect of sample size on the statistical significance. Thus, in the large-sized data, assigning people which have statistical literacy should anticipate that the statistical significance of almost meaningless because the relationship between the two variables are expected to be statistically significant". [7] opines that multiple regression analysis is the development part of the simple regression analysis. It is to predict the value of the dependent variable (Y) if there are two or more independent variables.

Type of Research
This research and development study relied on the development model by [12] since it comprised some advantages, including (1) to develop the holistic assessment; (2) the stages were systematic and brief; (3) having clear linkages among the stages.
The procedures of the Research and Development study, as declared by [12] were in the form of cycles, i.e., a study on research findings, creating products based on the findings, testing the products in the field, and revising the tested products. Validity sheets given to the experts were used to assess the research instruments. This study was conducted at Universitas Negeri Makassar (UNM) by attempting to develop a holistic assessment. The second-semester students at the Study Program of Statistics, Faculty of Mathematics and Natural Sciences, UNM participated in this study as the subjects.

Development Process
The implementation of this research went through three stages. Firstly, the preliminary stage as the initial activity before continuing to the next stages. There were two activities carried out in this first stage, namely (a) an analysis of theories or concepts about the holistic-based assessment model, along with the findings of previous studies related to the assessment model in university; (b) a field survey regarding the needs analysis, i.e., an evaluation model that was employed by lecturers in the process of learning statistics, the holistic-based assessment process. Secondly, the stage of designing holistic assessment media and research instruments of the assessment mentioned previously. Thirdly, the stage of development by performing different trials and revisions of the application of the holistic-based assessment model as well as research instruments, so that theories and practices were integrated.

Data Collection Instruments
The validity, effectiveness, and practicality of the model of holistic assessment required research instruments that consisted of validity sheets, assessment sheets of model and application appropriateness, observation sheets, questionnaires of students' and teachers' responses, and observation sheets of the holistic assessment process.
This research examined the validity of the implementation, management observation, instruments, and guidebook of holistic assessment, as well as students' responses; those were formulated in validity sheets. The data of holistic assessment validity came from three experts.
The practicality of a learning model was viewed from the consistency of two assessment results, including (1) the result of assessment conducted by experts and practitioners based on theories and experiences revealed that the developed model could be undergone in the field (intended↔perceived) well; (2) the result of assessment done by observers indicated that the model implementation during the learning process fell into a good category, as viewed by three aspects of observation, namely introductory, core, and closing activities. The effectiveness of a holistic assessment was measured by three primary aspects of implementation observation, i.e., students' behavior, learning outcomes, and responses towards the process of the holistic assessment.

Verification
The student holistic assessment model conducted by the lecturer is as follows: 1. Designing an assessment strategy by the lecturer according to the Semester Learning Plan (SLP) made by the lecturer; 2. Designing test instruments to assess cognitive and psychomotor aspects as well as designing non-test instruments to assess affective aspects. 3. The assessment of attitude aspects is done through observation at each meeting; 4. The assessment of knowledge aspects is done through independent assignments, group assignments, written tests, and oral tests; 5. The Skills assessment is done through practice in running applications to analyze data in accordance with the assessed competencies; 6. At the last meeting a written test was conducted to assess the cognitive and psychomotor aspects of the Some of the main characteristics of the holistic assessment model, such as: 1. This assessment model is carried out in an integrated way on learning multiple linear regression literacy. 2. This assessment model includes a load of information about student learning outcomes from cognitive, affective and psychomotor aspects. 3. This assessment model is oriented to how multiple linear regression literacy learning can better monitor the progress of student learning outcomes.
The novelty of the holistic assessment model in multiple linear regression literacy, the development of this assessment model is very important because it is based on facts in the class and in accordance with theoretical studies, it can be explained as follows: 1. The definition of statistical literacy in this study includes 5 competencies, namely: the ability to master concepts, applications, calculations, interpretations, and visualization & communication. 2. Implications of these findings: the assessment of learning outcomes starts from the reality that is in the classroom.

Validity
Following its stages, i.e., design and development, this study designs the module and its instruments. The prepared instruments encompass expert assessment and field instruments; both are utilized as the formats for the experts and field data collection. The instrument of expert assessment consists of two formats of assessing all field instruments and the module. Similarly, the field instrument also has two formats of test and non-test instruments.
The validity test is implemented after having the design of the initial model. In addition to the initial model, assessment sheets for experts with their suggestions should also be prepared. Each expert assesses the instruments to be applied in the study through the provided sheets; such an assessment is necessary to improve the instruments themselves. The validity test is conducted time after time, especially when it comes to corrections from the experts. Minor revisions from the experts towards the test and non-test instruments play an essential role in the development of the instruments. Considering all aspects of validity, all instruments used in this study are categorized valid in the initial validity test. Nevertheless, there are some suggestions from the experts to be taken into account for the improvement of the assessment model. The minor revisions are carried out following the experts' suggestions, which then the revised instruments pass through the second test of validity.
The criteria of a valid module include (1) the average score ( ) of all aspects which are at least in a valid category, (2) the score of each aspect minimally falls under a valid category [13]. It is concluded from the validity test that the assessment model in multiple linear regression literacy is valid.
Below are the results of the validity test from three experts: The instrument of Personal Attitude  The average score out of nine assessment aspects of the personal attitude instrument measures at 3.1 (76.9%) and is categorized valid.

The instrument of Learning Outcomes in Multiple Linear Regression Literacy
The instrument of learning outcomes in multiple linear regression literacy scores 3.4 (86.1%) out of three assessment aspects and falls into a valid category.  The instrument of Students' Responses towards Holistic Assessment of Multiple Linear Regression Literacy  The instrument of students' responses towards the holistic assessment of multiple linear regression literacy scores averagely 3.4 (85.2%) out of nine assessment aspects and is considered valid.  The instrument of Holistic Assessment Implementation in the Topic of Multiple Linear Regression Literacy The average score out of nine assessment aspects of the instrument of holistic assessment implementation in the topic of multiple linear regression literacy reaches 3.6 (88.9%) and represents a valid category.
The summary of validity results of the test and non-test instruments' average score of the holistic assessment in multiple linear regression literacy is clearly elaborated in the following table 1.

Practicality
The study program of Statistics, Faculty of Mathematics and Natural Sciences, UNM has been carrying out a holistic assessment for five meetings; the implementation is observed. The observation intends to find out the practicality of holistic assessment in multiple linear regression literacy, in which the aspects being observed are introductory, core, and closing activities. The observation result of the implementation of holistic assessment in multiple linear regression literacy.
The implementation of the introductory activity of the assessment in the trial stage during five meetings scores averagely 1.7; this score is from one observer. Since the average score ranges from the interval 1.5 ≤ KM ≤ 2.0, thus the introductory activity in holistic assessment has been fully implemented. There are some components of the assessment introductory activity that have not been performed in the first and second meetings due to the lack of understanding of holistic assessment in the lesson of multiple linear regression literacy. However, the next meeting has accomplished all components of the introductory activity of the aforementioned lesson. Hence, the assessment model of the multiple linear regression literacy has met the practicality criteria.
The implementation of the core activity of the assessment in the trial stage during five meetings scores averagely 1.6; this score is from one observer. Since the average score ranges from the interval 1.5 ≤ KM ≤ 2.0, therefore, the core activity in holistic assessment has been fully performed. There are some components of the assessment core activity that have not been undergone in the first meeting; this blames the lack of understanding of holistic assessment in the lesson of multiple linear regression literacy. Nevertheless, the next meeting has accomplished all components of the core activity of the lesson as mentioned above, signifying that the assessment model of the multiple linear regression literacy has satisfied the practicality criteria.  The implementation of the closing activity of the assessment in the trial stage during five meetings scores averagely 1.6; this score is from one observer. Since the average score ranges from the interval 1.5 ≤ KM ≤ 2.0, therefore, the closing activity in holistic assessment has fully passed through. There are some components of the assessment core activity that have not been undergone in the first meeting; this blames the lack of understanding of holistic assessment in the lesson of multiple linear regression literacy. However, the next meeting has accomplished all components of the core activity in the lesson mentioned previously, meaning that the assessment model of the multiple linear regression literacy has satisfied the practicality criteria. As based on the result of observation done in five meetings, the total score of the implementation of all activities arrives at 1.7, or in the interval 1.5 ≤ KM ≤ 2.0. On that ground, the holistic assessment of multiple linear regression literacy has been completely implemented.

Effectiveness
The effectiveness of holistic assessment is measured by three primary aspects of implementation observation, i.e., students' behavior, learning outcomes, and responses towards the process of a holistic assessment. Measuring the effectiveness is viewed from the trial in the observation of students' behavior in the learning process. In the trial stage for the first and second observations, it is revealed by the lecturer that student discipline falls into "good" and "very good" category, respectively. Although the percentage representing the good category still serves as the highest, yet it decreases the percentage of moderate category of student discipline.
Students' behavior in the first observation arrives at a moderate category, yet it rises to a very good category in the second observation. Further, their attitude in the first and second observations also falls into a very good category, particularly in the second observation that the number of students with a great attitude is excessively increased. The percentage representing the number of students mentioned earlier is 71.4%, a very good category.
As resulted from a multiple-choice test and multiple linear regression literacy, students' learning outcomes in the aspect of concept understanding, insight into application, numeracy skill, and interpretation capability achieve a high category of 49.4%, 63.6%, 49.4%, 66.2%, respectively. Visualization and communication skills, on the other hand, reach a very high category of 50.6%. Under the assessment given by those three experts, this study does a revision, which is then continued to another trial. The trial is conducted to 77 students at the Study Program of Statistics, Faculty of Mathematics and Natural Sciences, UNM. The students' learning outcomes of multiple linear regression literacy comprise five aspects, as described below.

Understanding of Statistical Concept
This aspect consisted of five-question items, in which the correct answer scores 1 and the incorrect one scores 0; the score is then converted to scale 10-100. The lowest, highest scores, mean, and standard deviation are 60, 100, 84.42, and 13.62, respectively. The table of frequency distribution shows that out of 77 respondents given a pre-test of statistical literacy learning outcomes in its relation to the aspect of regression concept understanding, 11 respondents (14.3%) get a moderate learning outcome, 38 respondents (49.4%) reach a high learning outcome, and 28 respondents (36.4%) arrive at a very high learning outcome. Thus, the trial result is overall measuring at a high category of learning outcomes. To sum up, students' learning outcomes of multiple linear regression literacy in the aspect of regression analysis concept understanding in the site area generally fall into a high category.

Insight into Statistical Application
This aspect comprised five-question items, in which the correct answer scores 1 and the incorrect one scores 0; the score is then converted to scale 10-100. The lowest, highest scores, mean, and standard deviation are 60, 100, 85.19, and 10.96, respectively. The frequency distribution in Table 4.19 reveals that out of 77 respondents provided by a test of multiple linear regression literacy learning outcomes in its relation to the aspect of insight into regression application, four respondents (5.2%) get a moderate learning outcome, 49 respondents (63.6%) arrive at high learning outcome, and 24 respondents (31.2%) achieve a very high learning outcome. Hence, the trial result is overall measuring at a high category of learning outcomes. Accordingly, students' learning outcomes of multiple linear regression literacy in the aspect of insight into regression application in the site area get a high category.

Statistical Numeracy Skill
This aspect consisted of six-question items, in which the correct answer scores 1 and the incorrect one scores 0; the score is then converted to scale 10-100. The lowest, highest scores, mean, and standard deviation are 66.67, 100, 86.58, and 11.48, sequentially. The table of frequency distribution brings out the fact that out of 77 respondents given a test of multiple linear regression literacy learning outcomes in its relation to the aspect of numeracy skill, 12 respondents (15.6%) achieve a moderate learning outcome, 38 respondents (49.4%) reach a high learning outcome, and 27 respondents (35.1%) arrive at a very high learning outcome. Therefore, the trial result is overall measuring at a high category of learning outcomes, meaning that students' learning outcomes of multiple linear regression literacy in the aspect of numeracy skill in the site area fall under a high category.

Statistical Interpretation Capability
This aspect consisted of six-question items, in which the correct answer scores 1 and the incorrect one scores 0; the score is then converted to scale 10-100. The lowest, highest scores, mean, and standard deviation are 50, 100, 85.06, and 100, consecutively. Based on the table of frequency distribution, out of 77 respondents given a test of multiple linear regression literacy learning outcomes in its relation to the aspect of interpretation capability, one respondent (1.3%) achieves a low learning outcome, 51 respondents (66.2%) reach a high learning outcome, and 25 respondents (32.5%) arrive at a very high learning outcome. In brief, the trial result is overall getting a high category of learning outcomes, signifying that students' learning outcomes of multiple linear regression literacy in the aspect of interpretation capability in the site area represent a high category.

Statistical Visualization and Communication Skills
This aspect consisted of eight-question items, in which the correct answer scores 1 and the incorrect one scores 0; the score is then converted to scale 10-100. The lowest, highest scores, mean, and standard deviation are 50, 100, 81.82, and 12.43, respectively. The table of frequency distribution shows that out of 77 respondents given a test of multiple linear regression literacy learning outcomes in its relation to the aspect of visualization and communication skills, one respondent (1.3%) gains a low learning outcome, 10 respondents (13.0%) reach a moderate learning outcome, 27 respondents (35.1%%) arrive at a high learning outcome, and 39 respondents (50.6%) get a very high learning outcome. For this reason, the trial result is overall measuring at a very high category of learning outcomes, highlighting that students' learning outcomes of multiple linear regression literacy in the aspect of visualization and communication skills in the site area fall under a very high category.
In order to measure the effectiveness of the instruments, students' responses towards holistic assessment in the multiple linear regression literacy are also examined. Students' responses in the cognitive, affective, and psychomotor aspects are in a very interesting category of 68.83%, 72.73%, and 81.82%, sequentially. Meanwhile, their responses in the aspect of multiple linear regression literacy skills achieve an interesting category of 74.0%.
As resulted from the trial, students' attitude is categorized "good" and "very good"; students' learning outcomes in all aspects arrive at a high category; students' responses towards holistic assessment get a high category. On that ground, the holistic assessment in multiple linear regression literacy is proven to be effective.
The important stage in model finalization is specifying the product of this study result. The specification is required in order for the product, in this case, the holistic assessment, to be sustainable and right-targeted; such a specification is determined by various aspects, namely capacity and time of use, and user. As based on its specification, holistic assessment is able to measure five statistical literacy competencies namely, statistical concept, application, numeracy, interpretation, and visualization and communication; it also can be used by the lecturers of statistics or mathematics.

Conclusions
This study draws several conclusions, including 1) the holistic assessment model is developed based on preliminary research and development of the aforementioned assessment in multiple linear regression literacy following the revision that is done after the validity test. The development of the holistic assessment was strengthened by carrying out a trial for the students at the Study Program of Statistics, Faculty of Mathematics and Natural Sciences, UNM. The trial was conducted in the lesson of multiple linear regression literacy in an integrated manner. The holistic assessment, oriented to the lesson of multiple linear regression literacy, encompasses the information on students' learning outcomes from the cognitive, affective, and psychomotor aspects. This model of assessment is also able to monitor the progress of students' learning outcomes; 2) it is concluded from the 3988 The Development of Holistic Assessment in Multiple Linear Regression Literacy validity test that the holistic assessment in multiple linear regression literacy is valid based on the result of instruments assessment by the experts. Besides, this assessment model is also considered practical and effective to apply as revealed by the observation of the holistic assessment implementation in the lesson of multiple regression linear literacy as well as the result of the trial.