Kernel Observers: Systems-Theoretic Modeling and Inference of Spatiotemporally Evolving Processes
We consider the problem of estimating the latent state of a spatiotemporally evolving continuous function using very few sensor measurements. We show that layering a dynamical systems prior over temporal evolution of weights of a kernel model is a valid approach to spatiotemporal modeling, and that it does not require the design of complex nonstationary kernels. Furthermore, we show that such a differentially constrained predictive model can be utilized to determine sensing locations that guarantee that the hidden state of the phenomena can be recovered with very few measurements. We provide sufficient conditions on the number and spatial location of samples required to guarantee state recovery, and provide a lower bound on the minimum number of samples required to robustly infer the hidden states. Our approach outperforms existing methods in numerical experiments.
This paper describes the problem of needing to model a spatiotemporal process using only a small number of physical locations.  The problem is rephrased using systems theory to provide a mechanism for identifying the optimal observation locations. Algorithms and associated proofs are provided to define the strategy and bound the minimum required complexity as a function of the complexity of the evolution of the system.  This is compared to nonstationary kernel-based approximation methods in numerical experiments which show significant improvements in training time.
Overall, not bad.  I think many of the paragraphs are far too long, and that often hurts the readability, but I think that the content is solid.  The introduction of Banach spaces adds nothing, which actually is part of a trend in this paper to add theory where it is not needed.  If the presentation could be tightened up some, with more of a practical slant, this paper would be very good.  As it is, the reason I gave a Technical score 2 rather than 3 was primarily for the issues with the proofs I list below, and not for the actual discussion in the main paper.  The experiments look fine, although Figure 3 is a bit confusing.  I have one significant fear about this paper, and that is why I am placing it here rather than toward the end of my comments: all the comments I have ever read about the Jordan canonical form suggest that it is numerical unstable to compute.  In a setting where that factorization of an admittedly inaccurate matrix ("Note that in practice, the matrix hat A needs to be inferred ...") is interacting with a kernel matrix which is notoriously subject to ill-conditioning, how can one be certain that the algorithms in this article will not be subject to instability?  Some acknowledgement of this should be included, if only to warn potential practitioners, or some proof as to why this will not be an issue.  The lack of discussion of numerical issues was the reason for a rating of 3 instead of 4 on the Potential Impact score.  I am not an expert in systems theory, and as such I may be way off base here, but I feel like much of the discussion regarding the proofs is unnecessarily abstract.  For example, Definition 1 could simply be "K has no zero valued columns", I think. In turn, Remark 1 could simply be "All positive definite kernels which are always positive produce shaded matrices", which would exclude oscillatory or compactly supported kernels, as I think it must.  Also, is a 1974 book really the most up to date reference available for cyclic index of a matrix?  If it is, maybe there are more popular tools for conducting these proofs?  Likewise, the immediate value of Proposition 1 is lost because by that point, the reader may have forgotten the assumption N < < M and be confused as to why one would need to prove that the product of two full rank matrices is full rank (as I did, initially).  Also, an important component of the proposition, that L > = M, is stated almost as a throwaway at the end, rather than as a condition of the setting; it would probably make more sense to start with that as an assumption, along with the assumption that Upsilon contains all (or at least M) unique values.  Also, I think there's a typo in the proof of proposition 1, where a bold K hat is written on line 83, but never defined or heard from again.  And if A hat is full rank with no repeated eigenvalues, I think the Lambda matrix just has M lambda values in it, because the Jordan decomposition would simply be the spectral decomposition?  That would be easier to comprehend than the confusing introduction of the Lambda_O term.  I also would prefer cleaner notation for the matrix-matrix products in this proof, where the use of [K ... K] implies something other than what (I think) is meant. O_Upsilon is defined as a matrix with NL rows and M columns, but the product written directly beneath it implies that the matrix should have N rows and M columns.  That of course can not be true since an NxM matrix with N < < M will not have linearly independent columns.  What I think is meant is a block diagonal matrix of size NL x ML with L copies of K along the diagonal.  Then the decomposition would be                     [K          [Lambda^tau_1 O_Upsilon =     ...                ...                              K] Lambda^tau_L] Or maybe I do not understand what O_Upsilon is, in which case the notation defining it is confusing.  And I think the same issue extends to line 88 where the logic is applied to the Jordan decomposition.  Also, the Lambda^t notation is not obvious: that matrix is not being raised to the t power?  And, actually, the statement about the sizes of P and Lambda^t is not correct or even consistent with the sizes you wrote earlier - here the resulting matrix would be size NxML, which would certainly not have independent columns.  The notation for the first part of the proof of proposition 2 is also confusing, but it at least looks consistent.  The proof of proposition 3 also seems fine if perhaps a bit confusing - maybe a cleaner presentation in more than 1 paragraph would help?  I think it should be possible to show that, with probability 1, the columns of the K matrix, for all positive definite kernels with only positive values, will be linearly independent for points drawn randomly from the domain.  This was shown to some degree in the differential equation setting by [Hon et al, On unsymmetric collocation by radial basis functions], and for the approximation setting by [Hangelbroek et al, Nonlinear approximation using Gaussian kernels]. I mention this only to alert the authors - no such proof need appear in this article.  Minor points: * Figure 5(b) says that it is a box plot, but does not look like one. * The phrase "much lesser" is used. * K, and variants with hats and twiddles and circles, is used too much.  It makes the content hard to discern. * spelling: nonistropic * The last paragraph of page 2 starts with a poorly written sentence