  Beta testers are the first end users outside a software company to use its
product. They have been used for decades and are rightly credited not only with
finding and reporting bugs, but also with improving general product usability
through their feedback and/or the ways they use the product. In this paper, we
investigate whether beta testers represent standard users well enough to allow
for the extrapolation of testing data to standard users. We have investigated
records of beta testers and standard users of home security solution developed
by the IT security software provider ESET. With more than 600 000 participants
from more than 180 countries, we present what we believe to be the first
large-scale comparison between standard users and beta testers. We compared
several aspects of both populations, such as hardware, operating system,
country of origin and EULA reading time, all taken from system data. Other
attributes, such as age, gender, privacy perception and computer proficiency
self-evaluation, were available thanks to a user questionnaire. We conclude
that - at least in our study - beta users represent standard users well in
terms of hardware and operating system in large scale beta testing. However,
populations differ significantly in the distribution of users and testers
between countries. This may cause a problem when a testing includes
localization and usability issues that may be influenced by regional
differences.

Computers and Society