Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference
<|reference_start|>Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference: The vector field of a controlled differential equation (CDE) describes the relationship between a control path and the evolution of a solution path. Neural CDEs (NCDEs) treat time series data as observations from a control path, parameterise a CDE's vector field using a neural network, and use the solution path as a continuously evolving hidden state. As their formulation makes them robust to irregular sampling rates, NCDEs are a powerful approach for modelling real-world data. Building on neural rough differential equations (NRDEs), we introduce Log-NCDEs, a novel, effective, and efficient method for training NCDEs. The core component of Log-NCDEs is the Log-ODE method, a tool from the study of rough paths for approximating a CDE's solution. Log-NCDEs are shown to outperform NCDEs, NRDEs, the linear recurrent unit, S5, and MAMBA on a range of multivariate time series datasets with up to $50{,}000$ observations.<|reference_end|>
@article{walker2024log,
    title={Log Neural Controlled Differential Equations: The Lie Brackets Make a
  Difference},
    author={Benjamin Walker, Andrew D. McLeod, Tiexin Qin, Yichuan Cheng, Haoliang
  Li, Terry Lyons},
    journal={Proceedings of the 41st International Conference on Machine
  Learning, 2024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.18512},
    primaryClass={cs.LG}
}