Answers.Body: <p>Your statement that researchers build their network from the ground-up using C++ or some other low level library <strong>couldn't be further from the truth</strong>.</p>
  <p>You could take a look at <a href="http://horace.io/pytorch-vs-tensorflow/" rel="nofollow noreferrer">this analysis</a> showing the popularity of these two frameworks in the top ML conferences. The following Figure is taken from there.</p>
  <p><img src="https://i.sstatic.net/UZOeL.png" alt="" /></p>
  <p>In CVPR-2020, for example, TensorFlow and pytorch combined for over 500 papers! Furthermore, because the two most active research entities (Google and Facebook) are backing these two frameworks, they are used in some of the most impactful research studies.</p>
  <hr />
  <p>I want to give some reasons that support the popularity of these frameworks, but first I'm going to rephrase your question a bit:</p>
  <blockquote>
  <p>Why use TensorFlow/Pytorch in python rather than build your model on your own using C++?</p>
  </blockquote>
  <p><em>Note: The reason I rephrased the question is because <a href="https://www.tensorflow.org/api_docs/cc" rel="nofollow noreferrer">TensorFlow</a> and <a href="https://pytorch.org/cppdocs/" rel="nofollow noreferrer">PyTorch</a> both have a C++ APIs.</em></p>
  <h3>Why are these frameworks so popular in contrast to lower-level programming languages?</h3>
  <p>Some reasons are the following</p>
  <ul>
  <li><p><strong>Rapid prototyping</strong>. Languages link C++, have bloated syntaxes, require low-level operations (e.g. memory management) and cannot be run interactively. This means it takes someone much less time to create and test a model in python than it does in C++.</p>
  </li>
  <li><p><strong>No need to re-invent the wheel</strong>. Some operations are common in most networks (e.g. backpropagation), why re-implement them? Other functionalities are hard to implement on your own (e.g. parallel processing, GPU computation). Do data scientists need to have such a strong technical background to research neural networks?</p>
  </li>
  <li><p><strong>Open-source</strong>. They benefit from being opensource and can offer a great deal of tools at your disposal for building neural networks. You want to add batchnorm to your network? No worries, just import it and add it in a single line! Also, they offer the perfect opportunity for sharing pretrained models.</p>
  </li>
  <li><p><strong>They are optimized</strong>. These frameworks are optimized to run as fast as possible on GPUs (if available) or CPUs. It would be virtually impossible for someone to write code that runs as fast on his own.</p>
  </li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-06T01:20:02.853Z
Answers.Text: Your comment “couldn’t be further from the truth” makes no sense. I said many, not most. This is confirmed from a casual Google search. Don’t exaggerate for effect. Read the question.
Answers.UserDisplayName: Cybernetic
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-05T23:57:09.717Z
Answers.LastActivityDate: 2021-01-05T23:57:09.717Z
Answers.UserDisplayName: Djib2011
Body: <p>Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories. Can existing open source frameworks also be used for this purpose, or are their implementations too limited?</p>
  <p>Can, for example, TensorFlow be used to craft an <em>original</em> network architecture that shows improvements on existing benchmarks? Can original mathematical work be coded into a high-level framework like TensorFlow such that original research on network architectures/approaches be demonstrated in a paper?</p>
  <p>A quick search reveals many papers using C++ in their implementation:</p>
  <ul>
  <li><a href="https://arxiv.org/pdf/1904.04174.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1904.04174.pdf</a></li>
  <li><a href="https://arxiv.org/pdf/1701.03980.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1701.03980.pdf</a></li>
  <li><a href="https://arxiv.org/pdf/2005.07039.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2005.07039.pdf</a></li>
  <li><a href="https://arxiv.org/pdf/2007.14236.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2007.14236.pdf</a></li>
  <li><a href="https://arxiv.org/pdf/1908.05858.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1908.05858.pdf</a></li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-05T22:38:47.787Z
Comments.Text: Hello. Welcome to AI SE! When you say "Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories", can you please provide 2-3 examples (links to papers or official implementations) where that has been the case? It seems to me that most AI researchers use frameworks (such as PyTorch and TensorFlow) to implement their models, so I don't think that sentence is true, that's why I'm asking for examples.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T00:56:22.733Z
Comments.Text: Literally nobody does this, the one exception maybe is YOLO where the guy wrote a C based library.
Comments.UserDisplayName: FourierFlux
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T01:17:18.923Z
Comments.Text: @FourierFlux Literally nobody does this? A 3 second Google search reveals 5 papers off the top. I didn’t say most authors, I said many, which is true.
Comments.UserDisplayName: Cybernetic
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T01:55:30.917Z
Comments.Text: Did you read the papers you referenced?  Except for the spiking neuron one all of the papers were not all focused on developing AI algorithms, but the implementation on specific hardware. Basically nobody who is interested in developing AI algorithms spends their time developing toolkits.
Comments.UserDisplayName: FourierFlux
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-06T02:53:11.193Z
Comments.Text: Look up the papers from top ML/AI conferences such as ICML, ICLR, AAAI, NeurIPS etc. All papers with an experimental content I've read used either Tensorflow, or PyTorch to do their experiments.
Comments.UserDisplayName: SpiderRico
Tags: <neural-networks><tensorflow><python><research><pytorch>
Title: Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?
UserDisplayName: Cybernetic
ThreadText: # Can TensorFlow, PyTorch, and other mainstream ML frameworks be used for research-grade work in AI?
  
  (Asked by: username_0 on 2021-01-05)
  
  Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories. Can existing open source frameworks also be used for this purpose, or are their implementations too limited?
  
  Can, for example, TensorFlow be used to craft an *original* network architecture that shows improvements on existing benchmarks? Can original mathematical work be coded into a high-level framework like TensorFlow such that original research on network architectures/approaches be demonstrated in a paper?
  
  A quick search reveals many papers using C++ in their implementation:
  
  * <https://arxiv.org/pdf/1904.04174.pdf>
  * <https://arxiv.org/pdf/1701.03980.pdf>
  * <https://arxiv.org/pdf/2005.07039.pdf>
  * <https://arxiv.org/pdf/2007.14236.pdf>
  * <https://arxiv.org/pdf/1908.05858.pdf>
  
  **Tags:** neural-networks, tensorflow, python, research, pytorch
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE! When you say "Many authors of research papers in AI (e.g. arXiv) write their neural networks from the ground-up, using low-level languages like C++ to implement their theories", can you please provide 2-3 examples (links to papers or official implementations) where that has been the case? It seems to me that most AI researchers use frameworks (such as PyTorch and TensorFlow) to implement their models, so I don't think that sentence is true, that's why I'm asking for examples.  
  > **username_2**: Literally nobody does this, the one exception maybe is YOLO where the guy wrote a C based library.  
  > **username_0**: @FourierFlux Literally nobody does this? A 3 second Google search reveals 5 papers off the top. I didn’t say most authors, I said many, which is true.  
  > **username_2**: Did you read the papers you referenced?  Except for the spiking neuron one all of the papers were not all focused on developing AI algorithms, but the implementation on specific hardware. Basically nobody who is interested in developing AI algorithms spends their time developing toolkits.  
  > **username_3**: Look up the papers from top ML/AI conferences such as ICML, ICLR, AAAI, NeurIPS etc. All papers with an experimental content I've read used either Tensorflow, or PyTorch to do their experiments.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2021-01-05. Score: 4 | Accepted answer
  
  Your statement that researchers build their network from the ground-up using C++ or some other low level library **couldn't be further from the truth**.
  
  You could take a look at [this analysis](http://horace.io/pytorch-vs-tensorflow/) showing the popularity of these two frameworks in the top ML conferences. The following Figure is taken from there.
  
  ![](https://i.sstatic.net/UZOeL.png)
  
  In CVPR-2020, for example, TensorFlow and pytorch combined for over 500 papers! Furthermore, because the two most active research entities (Google and Facebook) are backing these two frameworks, they are used in some of the most impactful research studies.
  
  ---
  
  I want to give some reasons that support the popularity of these frameworks, but first I'm going to rephrase your question a bit:
  
  > Why use TensorFlow/Pytorch in python rather than build your model on your own using C++?
  
  *Note: The reason I rephrased the question is because [TensorFlow](https://www.tensorflow.org/api_docs/cc) and [PyTorch](https://pytorch.org/cppdocs/) both have a C++ APIs.*
  
  ### Why are these frameworks so popular in contrast to lower-level programming languages?
  
  Some reasons are the following
  
  * **Rapid prototyping**. Languages link C++, have bloated syntaxes, require low-level operations (e.g. memory management) and cannot be run interactively. This means it takes someone much less time to create and test a model in python than it does in C++.
  * **No need to re-invent the wheel**. Some operations are common in most networks (e.g. backpropagation), why re-implement them? Other functionalities are hard to implement on your own (e.g. parallel processing, GPU computation). Do data scientists need to have such a strong technical background to research neural networks?
  * **Open-source**. They benefit from being opensource and can offer a great deal of tools at your disposal for building neural networks. You want to add batchnorm to your network? No worries, just import it and add it in a single line! Also, they offer the perfect opportunity for sharing pretrained models.
  * **They are optimized**. These frameworks are optimized to run as fast as possible on GPUs (if available) or CPUs. It would be virtually impossible for someone to write code that runs as fast on his own.
  
  > **Comments:**
  > **username_0**: Your comment “couldn’t be further from the truth” makes no sense. I said many, not most. This is confirmed from a casual Google search. Don’t exaggerate for effect. Read the question.  
  
  ---
  
  