  Decision forest, including RandomForest, XGBoost, and LightGBM, is one of the
most popular machine learning techniques used in many industrial scenarios,
such as credit card fraud detection, ranking, and business intelligence.
Because the inference process is usually performance-critical, a number of
frameworks were developed and dedicated for decision forest inference, such as
ONNX, TreeLite from Amazon, TensorFlow Decision Forest from Google, HummingBird
from Microsoft, Nvidia FIL, and lleaves. However, these frameworks are all
decoupled with data management frameworks. It is unclear whether in-database
inference will improve the overall performance. In addition, these frameworks
used different algorithms, optimization techniques, and parallelism models. It
is unclear how these implementations will affect the overall performance and
how to make design decisions for an in-database inference framework.
  In this work, we investigated the above questions by comprehensively
comparing the end-to-end performance of the aforementioned inference frameworks
and netsDB, an in-database inference framework we implemented. Through this
study, we identified that netsDB is best suited for handling small-scale models
on large-scale datasets and all-scale models on small-scale datasets, for which
it achieved up to hundreds of times of speedup. In addition, the
relation-centric representation we proposed significantly improved netsDB's
performance in handling large-scale models, while the model reuse optimization
we proposed further improved netsDB's performance in handling small-scale
datasets.

Databases;Machine Learning;Performance