By Rosangela Bando, Francisco Gallego, Paul Gertler
Let’s imagine a dilemma: Fernanda, a principal at an elementary school in Mexico, is thinking of buying her daughter a laptop. Her daughter is halfway through fifth grade. Fernanda thinks this is a good time to introduce her daughter to the digital world: A computer should allow her daughter to excel in school. But is it a good idea?
Then Fernanda thinks about her role as a school principal. Instead of buying one laptop for her own child, should she try to raise funds to get laptops for all her fifth and sixth-grade students? She wants children in her school to excel academically. But is the fundraising effort worth the time and expense?
Educators and policymakers wrestle with questions like this every day. In 2015, the Organisation for Economic Co-operation and Development (OECD) published a skeptical report about information technologies (IT) in education, stating that using computers in classrooms generally has not led to improvements in reading, mathematics, or science in the countries that invested heavily in them. This was a disappointing message, considering that between 2006 and 2012 member countries of the Inter-American Development Bank (IDB) distributed more than 10 million laptops in public schools in Latin America. In light of this report, those investments in laptops, at first glance, may not seem like money well spent.
The switch from print to digital learning, however, might be effective if other potential gains are considered, like improved digital literacy, lower communication costs, better Internet access, and cost-effectiveness. To investigate this, jointly with Dario Romero a current Ph.D. student at Columbia, we conducted a randomized evaluation to measure the impact of a Honduras program called Educatrachos on students’ educational performance.
The Educatrachos program and evaluation
The Educatrachos program, launched by the Honduran Ministry of Education in 2013, replaced standard mathematics and Spanish textbooks with XO 1.75 laptops in high-poverty areas of Honduras. Since schools that received laptops did not receive textbooks, this study was one of the first randomized evaluations to look at technology as a substitute for, rather than a complement to, traditional instruction materials.
The government selected third- and sixth-grade classrooms at 271 schools to participate in this study (a total of 9,600 students). We randomly assigned half the schools to receive the laptops and the other half to receive traditional textbooks and conducted tests to assess the impact of the program on academic performance. By randomly assigning schools to the program, we knew that schools that received laptops were statistically similar to schools that received textbooks, so we could tell that any differences between the two groups of schools were a direct result of the laptop program.
The results show that the Educatrachos program effectively increased access to and use of computers at school. Similar to the findings from the OECD report, we found that laptops and traditional textbooks resulted in no differences in student performance—and at a large enough scale, basic computers that store multiple electronic copies of textbooks on their hard drives could actually be more cost-effective than physical textbooks. This means that despite no significant difference in student learning, schools could still gain from using laptops instead of books.
Our cost-effectiveness analysis
We conducted a preliminary analysis using administrative data on the costs of the laptops used in the Educatrachos program to determine whether the switch was cost-effective for schools. The total cost of Educatrachos was US$95 per child, which included the cost of the laptops (expected to last four years), technical assistance, Internet and operation of the laptops, and training for principals. Importantly, to measure benefits, we factored in not only cost savings for two textbooks (US$14 each), but also increases in future job earnings due to improved digital literacy for students who used the laptops (a conservative premium of 1 percent, or US$35 over the student’s working life).
Even when factoring in these benefits, though, there was a net cost of US$32 per student. This would indicate that the Educatrachos program was not cost-effective.
Consider, however, the capacity of the laptop. Even the most basic laptop can store far more than two textbooks on its hard drive. In fact, Educatrachos laptops came with 23 pre-loaded books in addition to the math and Spanish textbooks. In the case of this program, making the switch from print to digital for three additional books would be enough to make the program tentatively cost-effective.
Many other factors could also influence this cost comparison. For instance, does the Honduran educational curriculum have three more textbooks to digitize? Our research only tested the effects of math and Spanish textbooks; would digitizing textbooks for other subjects, such as science or history, have a positive, neutral, or negative effect on learning? Moreover, it is important to consider the lifespan of the textbooks: If they are useful for a longer period (in this context, textbooks were replaced every year), their cost could be much lower than US$14.
Let’s think back to Fernanda, our elementary school principal. She’s trying to decide if she should fundraise for laptops in her classrooms. Armed with the evidence, Fernanda might decide to move forward—but knowing that computers may only improve student learning with appropriate use, she will invest in effective training and content too.
Education systems are rapidly moving toward greater digitalization. As the OECD’s Furthermore, evaluations of educational software programs (of Computer-Assisted Instruction and in India) that allow for individualized instruction have revealed large increases in test scores for both stronger and weaker students in the same classrooms. J-PAL’s Latin America and Caribbean office, which provided support for our evaluation, is currently advancing projects on this topic, in the hopes that the efforts put into bringing information and communication technology into the classrooms may finally translate into durable and significant learning gains for students.
Our study demonstrates that, and there are other gains to be had. Although the Educatrachos program was not cost-effective in its current form, providing laptops with more content could make them more cost-effective in the future. More research about the potential gains (or harmful effects) from computer use is needed, but we hope these findings are a promising step forward to help guide future policy.
About the authors:
Rosangela Bando is an economist in the Office of Strategic Planning and Development Effectiveness at the IDB.
Francisco Gallego is Associate Professor of Economics at the Pontifical Catholic University of Chile and serves as Scientific Director of the Latin America and Caribbean office of the Abdul Latif Jameel Poverty Action Lab (J-PAL).
Paul Gertler is the Li Ka Shing Professor of Economics at the University of California Berkeley, where he holds appointments in the Haas School of Business and the School of Public Health, and is a J-PAL affiliate.