Infinite Factorial Dynamical Model
We propose the infinite factorial dynamic model (iFDM), a general Bayesian nonparametric model for source separation. Our model builds on the Markov Indian buffet process to consider a potentially unbounded number of hidden Markov chains (sources) that evolve independently according to some dynamics, in which the state space can be either discrete or continuous. For posterior inference, we develop an algorithm based on particle Gibbs with ancestor sampling that can be efficiently applied to a wide range of source separation problems. We evaluate the performance of our iFDM on four well-known applications: multitarget tracking, cocktail party, power disaggregation, and multiuser detection. Our experimental results show that our approach for source separation does not only outperform previous approaches, but it can also handle problems that were computationally intractable for existing approaches.
The authors introduce a new Bayesian nonparametric model, more specifically an infinite factorial dynamic model (iFDM). This model is based on the Markov Indian buffet process and it allows for a possibly unbounded number of hidden Markov chains, each of which evolve independently according to some dynamics. The resulting inference problem is solved using particle Gibbs with ancestor sampling. The application in mind is that of source separation and the approach is evaluated on four examples; multitarget tracking, cocktail party, power disaggregation, and multiuser detection. The iFDM contains several interesting simpler models as special cases. Why is the state of the underlying Markov chain restricted to be of order one? There is a comment in the multitarget tracking example that the existing BNP approaches cannot handle this. What is the problem exactly?You are using a lot of particles, do you experience problems with degeneracy?You can probably improve the performance quite a lot by making use of a fully adapted filter, rather than the bootstrap as you are currently doing. Since your models are analytically tractable I think this should be quite straightforward to derive. More specifically this boils down to changing equation (4) and possibly also the resampling step. An early and basic introduction is provided in (Section II.D)http://www.stats.ox.ac.uk/~doucet/doucet_godsill_andrieu_sequentialmontecarloforbayesfiltering.pdf where it is referred to as the optimal proposal. If you google ``fully adapted particle filters'' you will find a lot more material.The authors have considered four different and all relevant application examples. The experimental section shows that the iFDM seems to work and that it can provide interesting results. The only comparison provided is against the FFBS-type algorithm, which we know will perform worse due to its construction. I know that it is a lot of work to implement other solutions to the problem, but if one were to do so it would probably provide an even better understanding of the performance of the model and it would be interesting to see the performance of existing solution to these problems. For example, for the multitarget tracking example, the simplest solution to this problem would probably be to use an extended Kalman filter together with nearest neighbour data association. Since your targets are very well separated I would expect this solution to perform quite well. It would be interesting to compare your performance against this simple standard solution. I have not worked with the cocktail party problem and the multiuser detection problems, but for the power disaggregation problem there are interesting solutions available, see for example the following NIPS paper (which is gaining some influence):Kolter, J. Z.; Batra, S.; and Ng, A. Y. Energy disaggregation via discriminative sparse coding. In Advances in Neural Information Processing Systems (NIPS). 2010. and the subsequent variational inference solution Kolter, J. Z., and Jaakkola, T. Approximate inference in additive factorial hmms with application to energy disaggregation. International Conference on Artificial Intelligence and Statistics. 2012.