Roboticists discover alternative physics
So these guys trained &quot;an AI&quot; to identify hidden variables by looking at videos
of physical phenomena, without prior knowledge of physics (except what was
implicitly and obscurely encoded in &quot;the AI&#x27;s&quot; neural net architecture, and its
training set and hyperparameters, by the researchers).<p>The &quot;AI&quot; came up with a slightly off number and the researchers struggle to
understand why. They struggle because &quot;the AI&quot; is a black box that lacks
explainability and cannot produce an explanatory model, only a predictive
model [1]. The simplest conclusion is that their &quot;AI&quot; has not learned any useful
models, and has only learned to accurately predict their test set, to which
the authors had acces throughout the training of &quot;the AI&quot; [2]. The simplest
explanation is that their model isn&#x27;t very good at doing what they wanted it to
do, but they opt to explain it as a scientific mystery, instead. Why?<p>It seems the only reason to try and see a scientific mystery where a simple
failure would suffice as an explanation is because the model is &quot;an AI&quot;. There
seems to be some kind of expectation that &quot;an AI&quot; <i>must</i> have some deeper
understanding of physics than humans, even when we don&#x27;t understand what it&#x27;s
doing, even whe it&#x27;s not doing very well.<p>In short, this seems to be based on very wrong assumptions and to be coming up
with very wrong conclusions, but then again it makes for a great headline and so
here we are, discussing it on HN.<p>____________________<p>[1] To clarify: a predictive model is one that can predict novel events. An
explainable model is one that can explain how it made a prediction. An
explanatory model is one that explains how the world works and why certain
predictions are true, or not. Predictive and explainable models are useful, but
most scientists aim to build explanatory models, because an explanatory model is
necessarily also explainable and ultimately better at predictions, while a
predictive model is not necessarily explainable or explanatory and an
explainable model is not necessarily predictive or explanatory. Deep neural nets
can only build predictive models, which are sometimes explainable, but they
can&#x27;t build explanatory models. That&#x27;s because they can only identify
correlations in data, but not explain those correlations with generalised
theories, based on previous knowledge.<p>To put it plainly, neural nets can&#x27;t come up with scientific theories, but they
can estimate probabilities of things happening. But scientists can and want to
come up with scientific theories, not just predictions.<p>[2] When the experimenter has access to the test data and can tune the learner&#x27;s
model until it scores highly on the test data, that leads to a model that
overfits to the test data. Such a model is useless for prediction over unseen
data (i.e. data not available to the researchers during training).