Spatio-Temporal Few-Shot Learning via Diffusive Neural Network Generation
<|reference_start|>Spatio-Temporal Few-Shot Learning via Diffusive Neural Network Generation: Spatio-temporal modeling is foundational for smart city applications, yet it is often hindered by data scarcity in many cities and regions. To bridge this gap, we propose a novel generative pre-training framework, GPD, for spatio-temporal few-shot learning with urban knowledge transfer. Unlike conventional approaches that heavily rely on common feature extraction or intricate few-shot learning designs, our solution takes a novel approach by performing generative pre-training on a collection of neural network parameters optimized with data from source cities. We recast spatio-temporal few-shot learning as pre-training a generative diffusion model, which generates tailored neural networks guided by prompts, allowing for adaptability to diverse data distributions and city-specific characteristics. GPD employs a Transformer-based denoising diffusion model, which is model-agnostic to integrate with powerful spatio-temporal neural networks. By addressing challenges arising from data gaps and the complexity of generalizing knowledge across cities, our framework consistently outperforms state-of-the-art baselines on multiple real-world datasets for tasks such as traffic speed prediction and crowd flow prediction. The implementation of our approach is available: https://github.com/tsinghua-fib-lab/GPD.<|reference_end|>
@article{yuan2024spatio-temporal,
    title={Spatio-Temporal Few-Shot Learning via Diffusive Neural Network
  Generation},
    author={Yuan Yuan, Chenyang Shao, Jingtao Ding, Depeng Jin, Yong Li},
    journal={arXiv preprint arXiv:2402.11922},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.11922},
    primaryClass={cs.LG}
}