Data is scaling exponentially in fields ranging from genomics to neuroscience to economics. A central question is whether modern machine learning methods can be applied to construct predictive models based on large data sets drawn from complex, natural systems like cells and brains . In machine learning, the predictive power or generalizability of a model is determined by the statistics of training data . In this paper, we ask how predictive inference is impacted when training data is generated by the statistical behavior of a physical system  . We develop an information-theoretic analysis of a canonical problem , spin network inference. Our analysis reveals the essential role that thermal fluctuations play in determining the efficiency of predictive inference. Thermal noise drives a system to explore a range of configurations providing `raw' information for a learning algorithm to construct a predictive model. Conversely, thermal energy degrades information by blurring energetic differences between network states . In general, spin networks have an intrinsic optimal temperature at which inference becomes maximally efficient. Simple active learning protocols allow optimization of network temperature , without prior knowledge, to dramatically increase the efficiency of inference  . Our results reveal a fundamental link between physics and information and show how the physical environment can be tuned to optimize the efficiency of machine learning.
Data is scaling exponentially in fields ranging from genomics to neuroscience to economics. A central question is : can modern machine learning methods be applied to construct predictive models of natural systems like cells and brains based on large data sets? In this paper, we examine how inference is impacted when training data is generated by the statistical behavior of a physical system , and hence outside direct control by the experimentalist . We develop an information-theoretic analysis for the canonical problem of spin-network inference. Our analysis reveals the essential role that the physical properties of the spin network and its environment play in determining the difficulty of the underlying machine learning problem. Specifically, stochastic fluctuations drive a system to explore a range of configurations providing `raw' information for a learning algorithm to construct an accurate model; yet they also blur energetic differences between network states and thereby degrade information. This competition leads spin networks to generically have an intrinsic optimal temperature at which stochastic spin fluctuations provide maximal information for discriminating among competing models, maximizing inference efficiency. We demonstrate a simple active learning protocol that optimizes network temperature to boost inference efficiency and dramatically increases the efficiency of inference on a neural circuit reconstruction task . Our results reveal a fundamental link between physics and information and show how the physical environment can be tuned to optimize the efficiency of machine learning.
R
whether
: can
D
can
R
based on large data sets drawn from complex,
of
R
. In machine learning, the predictive power or generalizability of a model is determined by the statistics of training data .
based on large data sets?
R
ask how predictive
examine how
A
, and hence outside direct control by the experimentalist
R
of a canonical problem , spin network
for the canonical problem of spin-network
R
thermal fluctuations
the physical properties of the spin network and its environment
R
efficiency of predictive inference. Thermal noise drives
difficulty of the underlying machine learning problem. Specifically, stochastic fluctuations drive
R
a predictive model. Conversely, thermal energy degrades information by blurring
an accurate model; yet they also blur
R
. In general, spin networks
and thereby degrade information. This competition leads spin networks to generically
R
inference becomes maximally efficient. Simple active learning protocols allow optimization of network temperature , without prior knowledge, to dramatically increase
stochastic spin fluctuations provide maximal information for discriminating among competing models, maximizing inference efficiency. We demonstrate a simple active learning protocol that optimizes network temperature to boost inference efficiency and dramatically increases
A
on a neural circuit reconstruction task