DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts
Learning from non-stationary data streams subject to concept drift requires
models that can adapt on-the-fly while remaining resource-efficient. Existing
adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or
simple voting schemes that fail to optimally leverage specialized knowledge.
This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture
that addresses these limitations through a novel co-training framework.
DriftMoE features a compact neural router that is co-trained alongside a pool
of incremental Hoeffding tree experts. The key innovation lies in a symbiotic
learning loop that enables expert specialization: the router selects the most
suitable expert for prediction, the relevant experts update incrementally with
the true label, and the router refines its parameters using a multi-hot
correctness mask that reinforces every accurate expert. This feedback loop
provides the router with a clear training signal while accelerating expert
specialization. We evaluate DriftMoE's performance across nine state-of-the-art
data stream learning benchmarks spanning abrupt, gradual, and real-world drifts
testing two distinct configurations: one where experts specialize on data
regimes (multi-class variant), and another where they focus on single-class
specialization (task-based variant). Our results demonstrate that DriftMoE
achieves competitive results with state-of-the-art stream learning adaptive
ensembles, offering a principled and efficient approach to concept drift
adaptation. All code, data pipelines, and reproducibility scripts are available
in our public GitHub repository: https://github.com/miguel-ceadar/drift-moe.
68833d33846d2e78b7548bb3
Miguel Aspis
claimed_verified
2025-07-29T12:51:10.922Z
68888cd33a918ab000899d6d
/avatars/7f41ab95dfeb668069a4c0cea4b1132c.svg
Miguel Aspis
user
m-aspis
68833d33846d2e78b7548bb4
Sebastián A. Cajas Ordónez
claimed_verified
2025-07-25T08:38:10.028Z
628ddf04986ae70e823298f7
https://cdn-avatars.huggingface.co/v1/production/uploads/628ddf04986ae70e823298f7/P6GyCswDo3dDMd59DEkWC.png
Sebastián Andres Cajas Ordóñez
user
sebasmos
68833d33846d2e78b7548bb5
Andrés L. Suárez-Cetrulo
claimed_verified
2025-07-25T08:38:44.525Z
674f4a1689fcf1699e6f1713
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/dMrPBgCzZBLcmW2Xue94n.jpeg
Andres L. Suarez Cetrulo
user
suarezcetrulo
68833d33846d2e78b7548bb6
Ricardo Simón Carbajo
DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts
Learning from non-stationary data streams subject to concept drift requires
models that can adapt on-the-fly while remaining resource-efficient. Existing
adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or
simple voting schemes that fail to optimally leverage specialized knowledge.
This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture
that addresses these limitations through a novel co-training framework.
DriftMoE features a compact neural router that is co-trained alongside a pool
of incremental Hoeffding tree experts. The key innovation lies in a symbiotic
learning loop that enables expert specialization: the router selects the most
suitable expert for prediction, the relevant experts update incrementally with
the true label, and the router refines its parameters using a multi-hot
correctness mask that reinforces every accurate expert. This feedback loop
provides the router with a clear training signal while accelerating expert
specialization. We evaluate DriftMoE's performance across nine state-of-the-art
data stream learning benchmarks spanning abrupt, gradual, and real-world drifts
testing two distinct configurations: one where experts specialize on data
regimes (multi-class variant), and another where they focus on single-class
specialization (task-based variant). Our results demonstrate that DriftMoE
achieves competitive results with state-of-the-art stream learning adaptive
ensembles, offering a principled and efficient approach to concept drift
adaptation. All code, data pipelines, and reproducibility scripts are available
in our public GitHub repository: https://github.com/miguel-ceadar/drift-moe.
DriftMoE, an online Mixture-of-Experts architecture, addresses concept drift in non-stationary data streams through a co-training framework that enhances expert specialization and resource efficiency.
Mixture-of-Experts MoE neural router incremental Hoeffding tree experts co-training framework symbiotic learning loop multi-hot correctness mask concept drift data stream learning adaptive ensembles