Body: <p>Does anyone know of research involving the GPT models to learn not only regular texts, but also learn from physics books with the equations written in latex format?</p>
  <p>My intuition is that the model might learn the rules relating equations and deductions, as they can learn statistically what correlates with what. I understand that the results can also be a little nonsensical, like the sometimes surreal paragraphs written by these models.</p>
  <p>Have there been any attempts to do this?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-04T14:39:41.217Z
Comments.Text: It's very unlikely that any machine learning model can really understand physics (in a "complete" way that a human can). Maybe you can use these models to find correlations between mathematical symbols and descriptions, but a full or complete understanding of a difficult subject like physics is unlikely to be accomplished with a single model. However, note that neural networks (or deep learning systems) have already been used to solve math problems (at least, people have tried to do this). See, for example, [this question](https://ai.stackexchange.com/q/7416/2444).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-06T00:59:32.677Z
Comments.Text: There is an interesting paper that tests GPT, BERT and ELMo to "show that the Transformer can be mathematically interpreted as a numerical
  Ordinary Differential Equation (ODE) solver for a convection-diffusion equation
  in a multi-particle dynamic system".  See https://web.stanford.edu/~yplu/pub/TransformerODE.pdf
Comments.UserDisplayName: Brian O'Donnell
Tags: <gpt><natural-language-understanding>
Title: Can in principle GPT language models learn physics?
UserDisplayName: Wolphram jonny
ThreadText: # Can in principle GPT language models learn physics?
  
  (Asked by: username_0 on 2020-09-03)
  
  Does anyone know of research involving the GPT models to learn not only regular texts, but also learn from physics books with the equations written in latex format?
  
  My intuition is that the model might learn the rules relating equations and deductions, as they can learn statistically what correlates with what. I understand that the results can also be a little nonsensical, like the sometimes surreal paragraphs written by these models.
  
  Have there been any attempts to do this?
  
  **Tags:** gpt, natural-language-understanding
  > **Comments:**
  > **username_1**: It's very unlikely that any machine learning model can really understand physics (in a "complete" way that a human can). Maybe you can use these models to find correlations between mathematical symbols and descriptions, but a full or complete understanding of a difficult subject like physics is unlikely to be accomplished with a single model. However, note that neural networks (or deep learning systems) have already been used to solve math problems (at least, people have tried to do this). See, for example, [this question](https://ai.stackexchange.com/q/7416/2444).  
  > **username_2**: There is an interesting paper that tests GPT, BERT and ELMo to "show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system".  See https://web.stanford.edu/~yplu/pub/TransformerODE.pdf  
  
  ---
  
  ## 0 answers
  
  