Portfolio Assessment: Learning Outcomes and Students’ Attitudes

This paper is aimed at (1) investigating whether the implementation of portfolio assessment had an impact on students’ writing ability, and (2) obtaining the students’ attitudes towards portfolios. The method used in this study was quasi-experimental research design, and the data were collected through a set of writing tests (pre-test and post-test) and a questionnaire. The results show that the implementation of portfolio assessment increased the students’ writing ability. It was also found that the students’ knowledge of global issues (content and organization) also increased more significantly than the local issues (grammar, vocabulary, and mechanics). In addition, the results of the questionnaire proved that the secondary level students had positive attitudes towards the * Corresponding author, email: saiful22@unsyiah.ac.id Citation in APA style: Sulistyo, T., Eltris, K. P. N., Mafulah, S., Budianto, S., Saiful, S., & Heriyawati, D. F. (2020). Portfolio assessment: Learning outcomes and students’ attitudes. Studies in English Language and Education, 7(1), 141-153. Received December 12, 2012; Revised January 31, 2020; Accepted February 27, 2020 https://doi.org/10.24815/siele.v7i1.15169 ©Universitas Syiah Kuala. All rights reserved. T. Sulistyo, K. P. N. Eltris, S. Mafulah, S. Budianto, S. Saiful & D. F. Heriyawati, Portfolio assessment: Learning outcomes and students’ attitudes | 142 implementation of the portfolio assessment. Therefore, it is suggested that the portfolio assessment should be implemented in the teaching-learning process especially in English writing since it could give regular feedback, and help the students in monitoring their writing progress.


INTRODUCTION
Writing is considered a difficult language skill to master especially for English as a foreign language (EFL) students. Muslim (2014) says that writing is not easy for both native and non-native students, and Negari (2011) claims that learning to write is difficult especially writing in a second or a foreign language in academic contexts since students do not know enough about how to generate ideas for writing. Considering writing as a complex process, the teachers' assessment and feedback are two of the main processes in improving students' writing ability. In this case, it can be done by keeping portfolios that can be used both as a learning and an assessment tool to help students overcome their problems in writing and enhance their learning in a period of time (Tahriri et al., 2014).
Some studies revealed that portfolio assessment has a significant impact on students' writing ability as well as their improvement in terms of content, organization, vocabulary, grammar, and mechanics (Efendi et al., 2017;Roohani & Taheri, 2015;Samad et al., 2017;Shokraie & Tabrizi, 2016;Tabatabaei & Assefi, 2012). The other findings found that portfolio assessment has a positive impact on students' writing performance in general and particularly in terms of content, organization, grammar, vocabulary, and mechanics (Obeiah & Bataineh, 2016;Prastikawati et al., 2016;Shokraie & Tabrizi, 2016;Tabatabaei & Assefi, 2012). In addition, Samad et al. (2013) explain that portfolio assessment allows EFL learners to participate actively in class with their peers or teachers, and it enables them to evaluate their own strengths and weaknesses during the teaching-learning process.
On the other hand, a study by Ghoorchaei et al. (2010) show that portfolio assessment mostly improved students' knowledge about organization, while the least improvement is in the vocabulary aspect. However, based on their study, it was also found that the aspect of mechanics does not show great improvement after the implementation of portfolio assessment. Another study conducted by Berliana et al. (2013) indicate that portfolio assessment only increases students' writing ability in terms of global issues namely content and organization. Moreover, portfolio assessment technique has a significant positive effect only on EFL learners' vocabulary learning by getting feedback of teachers (Omidi & Yarahmedzehi, 2016). On the contrary, the study conducted by Roohani and Taheri (2015) find that portfolio assessment does not give more impact on students' vocabulary. They also report that portfolio conferences might limit time which could be spent on other activities.
In general, the previous studies showed inconsistent findings. Besides, the time management for portfolio implementation is difficult to be applied (Lam, 2015). Thus, it is necessary to conduct further research dealing with how to implement portfolio assessment for EFL students in order to measure the effects of portfolio assessment on students' writing ability in terms of local issues (vocabulary, grammar, and mechanics), and global issues (content and organization). In this case, the researchers focused on investigating the effect of portfolio assessment on students' recount writing ability and how portfolio cultivated their reflective thinking based on their perceptions towards portfolio. Accordingly, this research is hoped to provide additional insights dealing with the effects of portfolio assessment on EFL students' writing ability including their attitudes towards portfolio assessment.
Based on the background of the study, the major research questions are as follows: 1. Does portfolio assessment have any impact on students' writing ability in terms of global and local issues? 2. What are the students' attitudes towards portfolio assessment?
The following hypotheses are also formulated: • Alternative hypothesis (Ha): Portfolio assessment has a significant effect on students' writing ability in terms of local and global issues • Null hypothesis (Ho): Portfolio assessment does not have a significant effect on students' writing ability in terms of local and global issues

LITERATURE REVIEW
A portfolio is a systematic and organized collection of students' work in a file folder or box that lets teachers and students monitor the progress and achievement of knowledge and ability in a certain area (Moya & Malley, 1994). They also explain that a portfolio is basically adopted from the field of arts used to show the samples of an artist's work. A portfolio can also provide informational needs and assessment requirements of schools for educational assessment in assessing students' ability which is used to achieve teaching-learning goals (Efendi et al., 2017). Furthermore, Applebee and Langer (1983) in Richards and Renandya (2002) define portfolios as a cumulative collection of the students' work that they have done since it shows a student's work from the beginning of the term to the end, and it gives both teachers and students a chance to assess how much the writing has progressed. In short, portfolio gives more opportunities for students and teachers to discuss students' difficulties and progress in learning writing, and it is a mean for teachers to assess students' process in writing as well as their texts.
A portfolio can also provide informational needs and assessment requirements of schools for educational assessment in assessing students' ability which is used to achieve teaching-learning goals (Efendi et al., 2017). Basically, there are three stages in conducting portfolio as suggested by Hamp-Lyons and Condon (2000) in Ghoorchaei et al. (2010), namely collection (students are expected to collect their final draft in a portfolio), selection (students are expected to select the best two or three final draft for grading), and reflection (students are expected to reflect upon on the first and final drafts). In this case, keeping portfolio means students should accomplish the several stages in writing called process of writing which takes some corrections (feedback) in order to get a good product (Nassirdoost, 2015).
In addition, a portfolio can also be used to support cooperative teaching by offering an opportunity for students to share and comment on each other's work (Lucas, 2007). Aydin (2010) in Khodashenas et al. (2013) state that portfolio has a good impact for students because it can help them to analyse their work, they can write in different styles, and it can improve their grammatical competence. Another important aspect of writing is assessment is that assessment is an integral part of the teaching-learning cycle that lets a teacher discover how far students have understood the materials during the teaching-learning process (Brown, 2000). Assessment is a mean of obtaining information about students' ability, knowledge, understanding, attainments, or attitudes (Richards & Renandya, 2002). This paper, then, focuses more on portfolio assessment that will provide an opportunity for students to evaluate their own learning and give useful information for teachers to improve their instructional plans and practice.
Nowadays, portfolio assessment has been popular in the educational area specifically in EFL which has a purpose to improve students' English ability. Portfolio assessment basically offers a student-centred approach. Dealing with the educational purpose, portfolio assessment has been implemented in the 2013 Curriculum which should be done by teachers in assessing students' learning process as well as their product (The Ministry of National Education of Indonesia or Kemendikbud, 2016). Moreover, Roohani and Taheri (2015) report that portfolio assessment has gained more attention in language education used for writing assessment, and many researchers have started to investigate the effectiveness of this method in writing assessment. Similarly, Tabatabaei and Assefi (2012) also clarify that portfolio assessment is a tool for assessing the writing processes as well as the final product of writing in order to improve the students' writing performance. They also claim that this technique is effective and can be implemented in writing classes. In brief, portfolio assessment is a tool to evaluate students' achievement in language learning in the period of time, and it also helps students to examine their own writing progress covering global issues and local issues.
Global issues of texts commonly consist of two aspects: content and organization. According to Karani (2008), content is about formulating main ideas and supporting sentences into an effective paragraph, while organization is about organizing the ideas or the events happened in the story logically, clearly, and easily. In contrast, local issues are mostly related to sentence structure covering vocabulary, grammar, and mechanics. Karani (2008) further explain that vocabulary is one of the important points in constructing a good paragraph by using appropriate words. Then, grammar covers the patterns of sentences construction and the good order of words in a sentence sequence, while mechanics are related to spelling and punctuation used in a sentence.

METHODS
Since the purpose of this study was to investigate the impact of applying portfolio assessment (independent variable) on students' writing ability (dependent variable), a quasi-experimental design (Sugiyono, 2013;2007) was administered by combining quantitative and qualitative data involving two groups namely portfolio assessment group (experimental group) and process-based approach group (control group). Class A, which consisted of 31 students, was the experimental group and treated using portfolio assessment. Whilst, 31 students in Class B acted as the control group and were trained using the process-based approach. All students were eight graders of a junior high school in Malang, Indonesia. The treatments for each group were conducted in eight eighty-minute-long meetings.
There were two main instruments employed to obtain quantitative and qualitative data: a set of writing tests and a questionnaire. The writing tests, which includes a pre-test and a post-test, were given in order to get quantitative data in terms of measuring students' writing ability. In this test, students were asked to write a recount text based on certain topics.
The pre-test was used to ensure the homogeneity of the level of knowledge of students in both groups (EC and CG) before the treatment (portfolio assessment) was given. The participants were to write a personal recount paragraph with a title 'My Bad Day'. In this test, some pictures were given to trigger students to complete the test.
The post-test was administered to evaluate students' writing improvement after the treatment of portfolio assessment in the experimental group and the process-based approach in the control group were given. The participants were to write a personal recount about students' personal experience in the past. In this case, the participants were to choose one of the interesting themes provided to write such as 'My Unforgettable Experience when I was in the First Grade in a High School and My Daily Activities at School in the Last Semester'. Both pre-test and post-test required the students to compose a recount paragraph consisting of 150 to 180 words in 80 minutes.
The results of each test were scored by two raters based on a scoring rubric covering global issues (content and organization) and local issues (vocabulary, grammar, and mechanics). For the qualitative data, a 10-item questionnaire using 4-Likert scale consisting of strongly agree, agree, disagree, and strongly disagree was administered.

The Effects of Portfolio Assessment on Students' Writing Ability
The independent sample t-test was utilized to investigate whether there was any significant impact of portfolio assessment on students' writing ability. The results of independent sample t-test can be seen in Table 1. Based on Table 1, the significance value was 0.000<0.05, so the alternative hypothesis 'Portfolio assessment has a significant effect on students' writing ability in terms of local and global issues' was accepted. Thus, it can be concluded that portfolio assessment affects the students' achievement in their paragraph writing ability and it is an effective way to be applied in the class. The primary data of this research were used to find out the significant difference in the test score before and after the treatment was given. The test measures two aspects related to writing: global and local issues. The scores of the pre and the post-test were compared to see the difference. Further, the result of the post-test was used to find out the t-value. The significant difference between the test scores was presented in Figure 1.  Figure 1 shows that the mean score difference of the pre-test from both groups was not statistically significant. The experimental group gained 59.639 and the control group gained 57.984. However, after the treatment, the experimental group performed better which was indicated by an increase in the mean score. For the post-test, the mean score gained by the experimental group was 78.416 while the control group was 69.190.
Whilst, the significant difference between global and local issues could be seen in Figure 2.  Figure 2 shows that the students from the experimental group got higher scores in both global and local issues. Further, the global issues, such as content and organization were higher than the local ones. Basically, the score for the global issues gained by the experimental group was 18.65, and that of the local issue was 12.72. Whereas, the students from the control group got only 16.52 for the global issue and 11.55 for the local ones. Hence, the total score of both issues gained by the experimental group was 31.37, while the control group gained 28.07.
During the treatment, it was found that students who were taught using portfolio assessment were actively involved in the writing activities by questioning what they did not understand yet, for example, about some vocabulary items of past tense. Moreover, most students could express their ideas though they were still confused about verb change from present to past form. Nevertheless, they were so enthusiastic when revising their draft after getting some teacher feedback because they could find out their mistakes and errors in their draft. By getting the feedback, they could correct those mistakes in order to produce a good product of writing. This is in harmony with Efendi et al. (2017) who report that portfolio assessment can increase students' motivation because they know their mistakes and what they should fix after getting the feedback. In the same way, Tabatabaei and Assefi (2012) assert that portfolio assessment can promote students' motivation because it gives an opportunity for teacher and students to discuss their problems during the teaching-learning process.
In addition, writing is basically a complex process covering selecting, combining, and arranging ideas into a good product. Dealing with the complicated process, according to Lundstrom and Baker (2009), writing itself has some aspects which have been categorized into two issues namely global issues which consisted of content and organization, and local issues, such as vocabulary, grammar, and mechanics. Relating to that statement, it was found that during the treatment most students were confused and got difficulties when writing a recount text with correct grammar. Due to this problem, it might be caused by some factors, such as the lack of vocabulary knowledge, the unfamiliarity with the activities, different writing styles, and the changes of verb form which were mostly used in the past form. This condition is basically similar to the study of Eridafithri (2015) and Karani (2008) where most EFL learners particularly Indonesian got problems in writing because of grammar and vocabulary. In fact, the students sometimes wrote a sentence based on Indonesian repertoires which might create misunderstanding in meaning. Thus, to overcome the problems encountered by students when making ineffective writing, the teacher then gave some feedback on their worksheet by applying portfolio assessment in order to correct their mistakes and enhance their writing ability.
Additionally, a study conducted by Prastikawati et al. (2016) find that the mean scores for the five writing aspects of the experimental group got a significant improvement after the treatment. In line with Obeiah and Bataineh (2016), Shokraie and Tabrizi (2016) and Tabatabaei and Assefi (2012), students of the experimental group showed a great improvement in writing ability in terms of content, organization, vocabulary, grammar, and mechanics. Related to the previous finding, the result of this study showed that using portfolio assessment improved students' overall writing ability in terms of five important aspects in writing, such as content, organization, vocabulary, grammar, and mechanics. However, the study of Roohani and Taheri (2015) shows that portfolio assessment does not affect students' writing ability in terms of vocabulary and mechanics. Also, Uçar and Yazıcı (2016) findd that there is no significant improvement in portfolio assessment on students' writing ability in terms of mechanics. Moreover, based on the data computation of those five aspects, the aspect of content increased most. The improvement score of content in post-test was in line with the study of Obeiah and Bataineh (2016), Prastikawati et al. (2016), and Shokraie and Tabrizi (2016), which indicate that the greatest effect of portfolio assessment is on the aspect of content. Meanwhile, the least improvement of those aspects in this study is grammar and mechanics. In this case, students encountered some problems related to grammar and mechanics when constructing sentences in past tense. Hence, grammar and mechanics are other problems encountered by most EFL students when writing a recount text related to patterns of sentence construction in past tense, and the use of capitalization, punctuation, and spelling (Karani, 2008). In conclusion, there were two aspects, namely grammar and mechanics which were categorized as poor improvement though the portfolio assessment technique has a significant effect on students' writing ability.

Local issues
Based on the observation during the research, some students still translated sentences directly from Indonesian to English when writing a paragraph of a personal recount. The condition is in accordance with Yi and Aung (2011) who reveal that many Myanmar or EFL students usually translate their mother tongue into English when writing, so their texts might contain many mistakes in terms of forms and meanings.
In addition, the results of this study indicate that students in both groups got a low score on the local issue though their writing ability has improved. In this case, the local score gained by the experimental group was 12.72, and by the control group, it was 11.55. Based on the data computation of local issue, it was found that there were two aspects categorized as the least effect particularly grammar and mechanics gained by students of both groups. In this case, grammar was classified as a serious problem faced by students in writing recount text. Similarly, a study done by Karani (2008) finds that the most serious problem encountered by students deals with grammar compared to the other aspects. Further, based on the observation, it was found that this problem mostly came up when the students wrote their experience by using past tense with regular and irregular verb forms. The next problem encountered by students was about mechanisms in terms of spelling, punctuation, and capitalization. As a result, they still made some errors in spelling words when writing. Besides, some of them did not give more attention to punctuation and capitalization, for example, period (.) at the end of a sentence. Furthermore, the students sometimes wrote the first letter of a word at the beginning of the paragraph or a title, and the name of someone or place in a small letter. Therefore, their improvement in this aspect was only 0.56; it was from 4.09 to 4.65. The result is in line with the study of Roohani and Taheri (2015) revealing that the aspect of vocabulary and mechanics did not increase significantly. Similarly, Uçar and Yazıcı (2016) find that there was no significant improvement for the mechanic aspect of students in the experimental group and control group. However, a study by Shokraie and Tabrizi (2016) investigating the effect of portfolio assessment on EFL learner's L2 writing performance in Iran prove that portfolio assessment could improve students' writing ability in terms of mechanics, such as spelling, punctuation, and capitalization.
The last problem was about vocabulary which usually came up when students should compose the recount paragraph with an appropriate word choice. Most students were confused about using proper words in their worksheet because most of them used the words taken from a dictionary. For example, the sentence 'Dia naik bis' which has been translated by students as 'He climbed the bus'. Basically, the structure and grammar of the sentence 'He climbed the bus' is true but the word choice is not proper. It should be 'He got on the bus'. Dealing with this problem, Karani (2008) states that vocabulary, particularly about diction, is one of the obstacles for students when composing sentences. In the same way, Yi and Aung (2011) also explain that lack of vocabulary may be a problem for students when constructing sentences. Based on the results of vocabulary improvement in the experimental group, it could be concluded that portfolio assessment is effective to improve students' vocabulary mastery. This finding is consistent with a study by Nassirdoost (2015) proving that portfolio assessment affects Iranian EFL learners' vocabulary achievement and motivation. Similarly, Omidi and Yarahmedzehi (2016) assert that the implementation of portfolio assessment is effective for and influential to students' vocabulary achievement. In contrast, a study conducted by Roohani and Taheri (2015) investigates the effect of portfolio assessment on Iranian EFL learners' expository writing ability showed that portfolio assessment did not give any significant effect on students' vocabulary mastery. Similarly, Shokraie and Tabrizi (2016) find that the least influence of the portfolio assessment is students' vocabulary mastery.

Global issues
Global issues consist of two writing aspects: content and organization. The result of global issues of this study showed that they increased better than those of local issues. Figure 2 shows the total score of global issues of both groups is 35.17 while the local issue was 24.27, and the difference was 10.9. In this case, the global issues gained by the experimental group was 18.65 and the control group was 16.52.
This finding is similar to Berliana et al. (2013) who find that content and organization increased most. The improvement of this aspect is due to some factors, such as the content of recount text which was stated clearly and understandable because it talked about students' own personal experiences. This finding was also in line with the study of Obeiah and Bataineh (2016), Prastikawati et al. (2016), and Shokraie and Tabrizi (2016), which indicate that the greatest effect of portfolio assessment is the aspect of content.
The second aspect of this issue was the organization. In this case, the enhancement of organization was 1.1 which from 6.13 to 7.23. This might be due to the organization of recount text which also engaged students themselves in the story. Thus, they could organize the ideas and the events happened in the past logically by using some chronological orders or linking words. Regarding the impact of portfolio assessment on students' writing ability in terms of organization, Ghoorchaei et al. (2010) in their investigation about the impact of portfolio assessment on Iranian EFL students' essay writing, find that organization was one of the sub-skills which had improved most. According to them, writing a portfolio could be effective for EFL classes because it could be used to increase the development of EFL students' writing ability. Similarly, as mentioned earlier that the finding of a study done by Berliana et al. (2013) shows that organization was one of the aspects which increase more.
According to them, the improvement in this aspect might be due to the simple part of descriptive text which only consisted of identification and description. Besides, the number of students was quite small in each group, so it was easy to control the idea of each student that would be described and organized in their work.

The Students' Attitudes towards Portfolio Assessment
The questionnaire was given to find out the students' attitudes towards the implementation of portfolio assessment in the writing class. In this questionnaire, there were 10 statements consisting of knowledge, motivation, and portfolio effectiveness. Moreover, the 4-Likert scale for this questionnaire that consisted of four categories: strongly agree, agree, disagree, and strongly disagree was employed.
The Appendix shows that most students believed that portfolio assessment can help them increase their writing ability. The results of the questionnaire indicated that students' attitudes towards portfolio assessment were generally positive. The result is basically in line with Yi and Aung (2011) who find that all students agreed that portfolio assessment improved their overall writing ability because of an ongoing process where they should write again and again. Besides, according to them, it could be due to the teacher's comment or feedback which could help the student to reflect the mistakes that they had noticed before.
Regarding the statement that students believed portfolio assessment could help them to organize their ideas well, a study conducted by Taki and Heidari (2011) also indicates that the participants agreed that organization was the most important area in writing. However, 6% of participants or about two students thought that they could not use the chronological order appropriately when writing their experiences. In addition, there were 61% of participants or 19 students claiming that they could write the story with a good structure and grammar. Similarly, a study done by Taki and Heidari (2011) reveals that about 50% of the students believed that their grammar improved after learning English writing by using a portfolio-based writing assessment. Indeed, the students were strongly satisfied with this method of writing assessment. On the other hand, there were four students or 13% of participants who did not agree if the portfolio assessment could help them in increasing their writing ability in terms of vocabulary and grammar. The result is in line with Ghoorchaei et al. (2010) who find that most interviewees said that the aspect which improved least is vocabulary. According to them, improving vocabulary basically requires more reading time.
As mentioned earlier, teacher's feedback is a good way for students to increase their language learning particularly in writing because it could help them to fix their mistakes (Efendi et al., 2017;Khodashenas et al., 2013). Dealing with this perception, 55% of students also agreed that portfolio assessment could help them to measure their progress in writing. In other words, portfolio assessment gave a chance for students to monitor how much their writing increased. Furthermore, students could be more motivated after getting teacher feedback directly (Efendi et al., 2017). In line with Chung (2012), portfolio assessment could be used to increase students' motivation. According to him, if the portfolio could be used well, students could see their learning growth and strengths as well as their weaknesses.

CONCLUSION
The implementation of portfolio assessment improved the students' writing ability covering local and global issues. A portfolio allows students to build their selfreflection by comparing their texts from time to time, so they are aware of their growth as well as strengths and weaknesses in writing activities. Nevertheless, the results of this study show that global issues increased better than local issues. In addition, there are two aspects of local issues that did not really increase significantly, such as grammar and mechanics. Students also have positive attitudes towards the implementation of portfolio assessment in the writing class. They agree that portfolio assessment cultivates their motivation in writing because they are able to correct their mistakes after getting teacher feedback. They are also more interested in learning when this activity is applied in the writing class.
Due to some limitations of this study, such as the number of the students involved, perhaps further studies should be conducted by adding the number of students as well as a moderate variable such as students' different learning styles in order to obtain more thorough insights related to the power of portfolios in writing classes.