  Continuous Integration (CI) requires efficient regression testing to ensure
software quality without significantly delaying its CI builds. This warrants
the need for techniques to reduce regression testing time, such as Test Case
Prioritization (TCP) techniques that prioritize the execution of test cases to
detect faults as early as possible. Many recent TCP studies employ various
Machine Learning (ML) techniques to deal with the dynamic and complex nature of
CI. However, most of them use a limited number of features for training ML
models and evaluate the models on subjects for which the application of TCP
makes little practical sense, due to their small regression testing time and
low number of failed builds.
  In this work, we first define, at a conceptual level, a data model that
captures data sources and their relations in a typical CI environment. Second,
based on this data model, we define a comprehensive set of features that covers
all features previously used by related studies. Third, we develop methods and
tools to collect the defined features for 25 open-source software systems with
enough failed builds and whose regression testing takes at least five minutes.
Fourth, relying on the collected dataset containing a comprehensive feature
set, we answer four research questions concerning data collection time, the
effectiveness of ML-based TCP, the impact of the features on effectiveness, the
decay of ML-based TCP models over time, and the trade-off between data
collection time and the effectiveness of ML-based TCP techniques.

Software Engineering