Probing the Compositionality of Intuitive Functions
How do people learn about complex functional structure? Taking inspiration from other areas of cognitive science, we propose that this is accomplished by harnessing compositionality: complex structure is decomposed into simpler building blocks. We formalize this idea within the framework of Bayesian regression using a grammar over Gaussian process kernels. We show that participants prefer compositional over non-compositional function extrapolations, that samples from the human prior over functions are best described by a compositional model, and that people perceive compositional functions as more predictable than their non-compositional but otherwise similar counterparts. We argue that the compositional nature of intuitive functions is consistent with broad principles of human cognition.
This paper tests the hypothesis that people's hypothesis space for learning functions is compositional. Function learning is formalized as Bayesian regression with functional forms given by compositional Gaussian process kernels (summing and multiplying radial basis, linear, and periodic kernels). A spectral mixture kernel provides an alternative hypothesis. A series of experiments demonstrates that people prefer to extrapolate functions (given points on the function) using compositional structure and find compositional functions to be more predictable.
Technical quality The number of experiments reported in the paper is impressive. Moreover, the use of the MCMCP method to probe people's posterior belief distribution is interesting. However, because Experiments 1a, 2a, and 3 use compositional ground truth, the conclusions that can be drawn from these experiments seem limited. It is not clear if people prefer compositional structures in these experiments because the data are generated from them or because they always prefer compositional structures. Furthermore, why are the compositional kernels not compared to the spectral mixture kernel in Experiments 2a and 2b? Perhaps the most convincing demonstration that people prefer compositional structure is Experiment 1b, in which compositional extrapolations are chosen more often even when the ground truth is the spectral mixture.  I wonder if it would be possible to incorporate a prior over the compositional structures that expresses people's preference for linear and periodic functions and (presumably) simpler combinations, and look at the full posterior belief distribution rather than just the likelihood?  A small comment on Experiment 3: I don't think many participants would find drawing a function using a Bezier curve very intuitive. What did the MTurk participants think of this? Maybe this experiment could be conducted on touchscreen devices instead.  Novelty/originality This work introduces the compositional kernel and includes several novel and interesting behavioral experiments.  Potential impact This paper will probably be of interest to many cognitive scientists and some computer scientists as well. However, it seems somewhat unsurprising from the outset that people prefer to think of functions as consisting of compositional structure rather than as a spectral density (unless there is strong evidence to believe the latter). The paper would be improved if some reasons to think otherwise were presented at the beginning.  Clarity and presentation The paper is well-written and clear. However, it would be helpful to include in the Introduction some examples of why function learning is important for an organism. I did not fully understand the math in Sections 2, 3.1, and 3.2, but I appreciated that the authors tried to provide intuitive explanations for the mathematical formalisms at every step. One small comment is that in lines 43-45, there are two symbols, x_asterisk and x_star, and it's not clear what they each denote (or if they're actually the same symbol).