Answers.Body: <h3>In short : YES, you can build AI models without a framework</h3>
  <hr />
  <h3>Now, onto the details:</h3>
  <p>Before discussing &quot;why&quot;, let's look at what is an ML framework:</p>
  <p>Simply put, an ML framework is a collection of abstractions that allow you to build ML models. If you are from a software engineering background, you may think of them as libraries with built - in modules to accomplish common ML related tasks, including but not limited to:</p>
  <ol>
  <li>Manipulating Tensors</li>
  <li>Loading and Processing Data</li>
  <li>Training a model</li>
  <li>Deploying your model</li>
  </ol>
  <p>For example, <a href="https://keras.io/examples/" rel="nofollow noreferrer">Keras</a> is a popular Deep learning library with many 'ready to use' components.</p>
  <h3>The 'Why' Part</h3>
  <p>To understand <em>why</em> you need a framework, you should have a basic understanding of what building an ML solution entails:</p>
  <ol>
  <li>Loading the data into your computer's memory</li>
  <li>Preprocessing data to some format that is suitable (for example you may have to preprocess images to have some fixed dimensions before feeding it to your model)</li>
  <li>Defining a basic model architecture. This architecture would vary depending on your task</li>
  <li>Training Your model</li>
  <li>Deploying it further</li>
  </ol>
  <p>Let's see how a framework helps at each stage:</p>
  <p><strong>Stages 1, 2 :</strong> Major Deeplearning frameworks provide you with very easy to use interfaces for loading and processing data. These are generally as simple as making a few lines of core API calls.</p>
  <p>For example, in <a href="https://pytorch.org/tutorials/beginner/basics/data_tutorial.html" rel="nofollow noreferrer">PyTorch</a>, loading the famous <a href="https://en.wikipedia.org/wiki/MNIST_database" rel="nofollow noreferrer">MNIST</a> database is as simple as</p>
  <pre><code>
  training_data = datasets.FashionMNIST(
      root=&quot;data&quot;,
      train=True,
      download=True,
      transform=ToTensor()
  )
  </code></pre>
  <p>Without such a simple API, you would be writing a ton of code to manually fetch, download, extract, copy / load your dataset.</p>
  <p><strong>Stage 3:</strong> These DL frameworks also provide you easy interfaces to manipulate underlying Tensors in memory. (Essentially mostDL Models can be decomposed into Matrices/Tensors). This is by far the single largest reason, i.e. <strong>Deeplearning Frameworks are capable of moving and manipulating these Tensors in GPUs</strong>. And you my be well aware about how much Compute Power ML Models need. GPUs are especially good at handling parallel processing tasks. (For example, you may take say 10,000 dimension Tensor and add a single number to it. Conventional programming would involve running a <code>for</code> loop to iterate over each individual element, whereas GPUs can process the entire Tensor parallely)</p>
  <p><strong>Stage 4:</strong> Finally, most DL frameworks also provide you means to deploy your trained model into production. (Although, this is a debatable topic as many people argue that it's better to re - write the model in performant languages like <code>C++</code> before deploying to production)</p>
  <h3>Addressing your concern over what people used before these frameworks came into existence:</h3>
  <p>Well, these popular DL <em>Frameworks</em> per se did not exist a decade ago, but we have had libraries and packages that attempt to perform some of the above tasks.</p>
  <p>For example, the popular python package <a href="https://en.wikipedia.org/wiki/NumPy" rel="nofollow noreferrer">Numpy</a> provided the capability of operating over nd arrays long before Tensorflow came into picture.</p>
  <p>As AI Expert <a href="https://twitter.com/karpathy/status/1638996540214902784?lang=en" rel="nofollow noreferrer">Andrej Karpathy</a> pointed out, during his academic journey, before GPU accelerated frameworks, he relied on tools like MATLAB to perform his experiments.</p>
  <h3>Should you or should you not use a framework?</h3>
  <p>For most cases, it's better to use a framework as it will save you a great deal of pain.</p>
  <p>If, however you want to learn the fundamentals of machine learning, down to the individual tensor and how it is manipulated, it might be useful to implement a Neural Network from scratch without any libraries (you might still need to use stuff like Numpy, or <a href="https://www.youtube.com/watch?v=PGSba51aRYU" rel="nofollow noreferrer">go full on hardcore C/C++</a> )</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-16T10:47:05.453Z
Answers.LastActivityDate: 2023-06-16T10:48:33.930Z
Answers.LastEditDate: 2023-06-16T10:48:33.930Z
Answers.UserDisplayName: Jay
Body: <p>I'm a relatively newbie in this world of Artificial Intelligence, although I am able to use frameworks such as Tensorflow and also understand the general concepts behind training weights and parameters, I wonder about a lot of stuff.</p>
  <ol>
  <li>So is it necessary that people use frameworks like Tensorflow and PyTorch to do ML</li>
  <li>And what do people do before frameworks were introduced?</li>
  </ol>
  
Tags: <machine-learning><training><tensorflow><pytorch><models>
Title: Is it possible to write/build an AI model without using Frameworks?
UserDisplayName: Ryan Wang
ThreadText: # Is it possible to write/build an AI model without using Frameworks?
  
  (Asked by: username_0 on 2023-06-15)
  
  I'm a relatively newbie in this world of Artificial Intelligence, although I am able to use frameworks such as Tensorflow and also understand the general concepts behind training weights and parameters, I wonder about a lot of stuff.
  
  1. So is it necessary that people use frameworks like Tensorflow and PyTorch to do ML
  2. And what do people do before frameworks were introduced?
  
  **Tags:** machine-learning, training, tensorflow, pytorch, models
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-06-16. Score: 3 | Accepted answer
  
  ### In short : YES, you can build AI models without a framework
  
  ---
  
  ### Now, onto the details:
  
  Before discussing "why", let's look at what is an ML framework:
  
  Simply put, an ML framework is a collection of abstractions that allow you to build ML models. If you are from a software engineering background, you may think of them as libraries with built - in modules to accomplish common ML related tasks, including but not limited to:
  
  1. Manipulating Tensors
  2. Loading and Processing Data
  3. Training a model
  4. Deploying your model
  
  For example, [Keras](https://keras.io/examples/) is a popular Deep learning library with many 'ready to use' components.
  
  ### The 'Why' Part
  
  To understand *why* you need a framework, you should have a basic understanding of what building an ML solution entails:
  
  1. Loading the data into your computer's memory
  2. Preprocessing data to some format that is suitable (for example you may have to preprocess images to have some fixed dimensions before feeding it to your model)
  3. Defining a basic model architecture. This architecture would vary depending on your task
  4. Training Your model
  5. Deploying it further
  
  Let's see how a framework helps at each stage:
  
  **Stages 1, 2 :** Major Deeplearning frameworks provide you with very easy to use interfaces for loading and processing data. These are generally as simple as making a few lines of core API calls.
  
  For example, in [PyTorch](https://pytorch.org/tutorials/beginner/basics/data_tutorial.html), loading the famous [MNIST](https://en.wikipedia.org/wiki/MNIST_database) database is as simple as
  
  ```
  
  training_data = datasets.FashionMNIST(
      root="data",
      train=True,
      download=True,
      transform=ToTensor()
  )
  
  ```
  
  Without such a simple API, you would be writing a ton of code to manually fetch, download, extract, copy / load your dataset.
  
  **Stage 3:** These DL frameworks also provide you easy interfaces to manipulate underlying Tensors in memory. (Essentially mostDL Models can be decomposed into Matrices/Tensors). This is by far the single largest reason, i.e. **Deeplearning Frameworks are capable of moving and manipulating these Tensors in GPUs**. And you my be well aware about how much Compute Power ML Models need. GPUs are especially good at handling parallel processing tasks. (For example, you may take say 10,000 dimension Tensor and add a single number to it. Conventional programming would involve running a `for` loop to iterate over each individual element, whereas GPUs can process the entire Tensor parallely)
  
  **Stage 4:** Finally, most DL frameworks also provide you means to deploy your trained model into production. (Although, this is a debatable topic as many people argue that it's better to re - write the model in performant languages like `C++` before deploying to production)
  
  ### Addressing your concern over what people used before these frameworks came into existence:
  
  Well, these popular DL *Frameworks* per se did not exist a decade ago, but we have had libraries and packages that attempt to perform some of the above tasks.
  
  For example, the popular python package [Numpy](https://en.wikipedia.org/wiki/NumPy) provided the capability of operating over nd arrays long before Tensorflow came into picture.
  
  As AI Expert [Andrej Karpathy](https://twitter.com/karpathy/status/1638996540214902784?lang=en) pointed out, during his academic journey, before GPU accelerated frameworks, he relied on tools like MATLAB to perform his experiments.
  
  ### Should you or should you not use a framework?
  
  For most cases, it's better to use a framework as it will save you a great deal of pain.
  
  If, however you want to learn the fundamentals of machine learning, down to the individual tensor and how it is manipulated, it might be useful to implement a Neural Network from scratch without any libraries (you might still need to use stuff like Numpy, or [go full on hardcore C/C++](https://www.youtube.com/watch?v=PGSba51aRYU) )
  
  
  ---
  
  