Answers.Body: <p>There are definitely approaches that are theory driven (like SVMs), and others where the theory comes after the practice (like a lot of deep neural networks). I think it would be difficult to argue strongly that either direction is more common "in general" within AI, or indeed, within any other branch of science. </p>
  
  <p>The approach that is currently in favor will tend to change over time as well. If we develop a new method that we don't understand, it will tend to attract interest from theorists. If they are able to understand it well, that understanding will likely lead to natural next steps that improve the model. As with other areas of science, the <a href="https://en.wikipedia.org/wiki/Empirical_research#Empirical_cycle" rel="nofollow noreferrer">empirical cycle</a> is at work. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-13T21:42:24.793Z
Answers.LastActivityDate: 2019-12-13T21:42:24.793Z
Answers.UserDisplayName: John Doucette
Body: <p>In the <a href="https://nlp.stanford.edu/pubs/glove.pdf" rel="nofollow noreferrer">original GloVe paper</a>, the authors discuss group theory when coming up with the equation (4). Is it possible that the authors came up with this model, figured out it was good, and then later found out various group theory justifications that justified it? Or was it discovered sequentially as it is described in the paper?</p>
  <p>More generally: In AI research, are most things discovered because they work empirically and later justified mathematically, or is it the other way around?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-12T12:01:23.817Z
Comments.Text: This question is not only related to AI but research in general (even though you provide the example of GloVe).
Comments.UserDisplayName: nbro
Tags: <research><papers><glove>
Title: Are most things generally discovered because they work empirically and later justified mathematically, or vice-versa?
UserDisplayName: Damien
ThreadText: # Are most things generally discovered because they work empirically and later justified mathematically, or vice-versa?
  
  (Asked by: username_0 on 2019-12-12)
  
  In the [original GloVe paper](https://nlp.stanford.edu/pubs/glove.pdf), the authors discuss group theory when coming up with the equation (4). Is it possible that the authors came up with this model, figured out it was good, and then later found out various group theory justifications that justified it? Or was it discovered sequentially as it is described in the paper?
  
  More generally: In AI research, are most things discovered because they work empirically and later justified mathematically, or is it the other way around?
  
  **Tags:** research, papers, glove
  > **Comments:**
  > **username_1**: This question is not only related to AI but research in general (even though you provide the example of GloVe).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-12-13. Score: 1
  
  There are definitely approaches that are theory driven (like SVMs), and others where the theory comes after the practice (like a lot of deep neural networks). I think it would be difficult to argue strongly that either direction is more common "in general" within AI, or indeed, within any other branch of science.
  
  The approach that is currently in favor will tend to change over time as well. If we develop a new method that we don't understand, it will tend to attract interest from theorists. If they are able to understand it well, that understanding will likely lead to natural next steps that improve the model. As with other areas of science, the [empirical cycle](https://en.wikipedia.org/wiki/Empirical_research#Empirical_cycle) is at work.
  
  
  ---
  
  