Online Estimation of a Joint Distribution from batches of data I want to implement an algorithm for the online estimation of a joint probability distribution from a sequence of mini batches sampled from the real distribution. The distribution is discrete and non parametric. I need to keep updated my estimation by embedding the information from the batches and gradually approximate the real distribution if possible.
What I was thinking about was a sort of EMA: $P(x,y)_{t+1} = \alpha \cdot P(x,y)_t + (1-\alpha) P(x,y)_{B}$ where $P(x,y)_{B}$ is estimated from the current sample batch.
Is it a good idea? I hope I was able to explain clearly my problem, in any case I remain available for further explanations.
Thank you in advance!

What you are trying to do (i.e. updating a posterior distribution based on a stream of observations) is called filtering. It is a well-studied problem for which several solutions exist. Here are a few reviews that might be helpful for you:

*

*Review on particle methods: Kantas, N., Doucet, A., Singh, S. S., Maciejowski, J., & Chopin, N. (2015). On particle methods for parameter estimation in state-space models.

*Review on non-linear filtering: Kutschireiter, A., Surace, S. C., & Pfister, J. P. (2020). The Hitchhiker’s guide to nonlinear filtering. Journal of Mathematical Psychology, 94, 102307.

*A recent paper showing a efficient implementation of particle filtering for experimental settings: https://arxiv.org/pdf/2201.07539.pdf
