Answers.Body: <p>The most generic answer to this question is:</p>
  <p><strong>the same metrics you use to evaluate the quality of your model during training or in test phase</strong>. (Plus the timing of inference if you're referring to computational efficiency)</p>
  <p>And I'm not referring to any specific metric yet cause that's really task dependent. But in general if you have a model that perform a task and another algorithm that perform the same task, then you should be able to apply both to the same set of data, compute whatever metric is suitable to evaluate the performance on the task, and compare the two scores. Let me stress out that the test instances should be the same for a scientifically relevant comparison, and I mean literally the same.</p>
  <p>As an example of some metrics I would refer to the web since out there there's plenty of blog posts listing and comparing metrics. Just to link a few:</p>
  <ul>
  <li><a href="https://towardsdatascience.com/performance-metrics-for-classification-machine-learning-problems-97e7e774a007" rel="nofollow noreferrer">classification</a></li>
  <li><a href="https://machinelearningmastery.com/regression-metrics-for-machine-learning/" rel="nofollow noreferrer">regression</a></li>
  <li><a href="https://towardsdatascience.com/deep-image-quality-assessment-30ad71641fac" rel="nofollow noreferrer">image quality assessment</a></li>
  <li><a href="https://towardsdatascience.com/on-object-detection-metrics-with-worked-example-216f173ed31e" rel="nofollow noreferrer">object detection</a></li>
  <li><a href="https://medium.com/memsource-engineering/mt-metrics-explained-feeling-bleu-f62b3ab4500" rel="nofollow noreferrer">machine translation</a></li>
  </ul>
  <p>The list is not exhaustive but I think it illustrates the point.<br />
  Also, as a side note: almost all machine learning algorithms are optimization-based, if you want to refer to approaches that don't fall into machine learning I think a better term is <em>analytic methods/approaches</em>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-05T08:02:48.557Z
Answers.Text: This is a great answer, I just want to suggest another side note. If OP is writing a scientific article, it might be worth adding another point in the Results and Discussion which is "how can the ML-based methods mimic the performance or compete with the performance of 'non-ML' based methods"? does it rediscover a certain (well-known) method? For example, an RL agent for travelling salesman problem could rediscover the greedy heuristic as shown by likelihood of taking the furthest node in each step.
Answers.UserDisplayName: Sanyou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-05T07:44:19.367Z
Answers.LastActivityDate: 2021-10-05T07:44:19.367Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>If a certain task T is solved by a non-learning-based method A (let's say, an optimization-based approach). We now train a machine learning model B (let's say a neural network) on the same task.</p>
  <p>What are some metrics that we can use to compare their efficiency in terms of finding the solution (Assuming the quality of both solutions is comparable)?</p>
  
Tags: <neural-networks><optimization><metric><algorithm-request><efficiency>
Title: Compare the efficiency of a trained ML model with a non-learning-based method for solving the same problem
UserDisplayName: GSH
ThreadText: # Compare the efficiency of a trained ML model with a non-learning-based method for solving the same problem
  
  (Asked by: username_0 on 2021-10-02)
  
  If a certain task T is solved by a non-learning-based method A (let's say, an optimization-based approach). We now train a machine learning model B (let's say a neural network) on the same task.
  
  What are some metrics that we can use to compare their efficiency in terms of finding the solution (Assuming the quality of both solutions is comparable)?
  
  **Tags:** neural-networks, optimization, metric, algorithm-request, efficiency
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-10-05. Score: 1
  
  The most generic answer to this question is:
  
  **the same metrics you use to evaluate the quality of your model during training or in test phase**. (Plus the timing of inference if you're referring to computational efficiency)
  
  And I'm not referring to any specific metric yet cause that's really task dependent. But in general if you have a model that perform a task and another algorithm that perform the same task, then you should be able to apply both to the same set of data, compute whatever metric is suitable to evaluate the performance on the task, and compare the two scores. Let me stress out that the test instances should be the same for a scientifically relevant comparison, and I mean literally the same.
  
  As an example of some metrics I would refer to the web since out there there's plenty of blog posts listing and comparing metrics. Just to link a few:
  
  * [classification](https://towardsdatascience.com/performance-metrics-for-classification-machine-learning-problems-97e7e774a007)
  * [regression](https://machinelearningmastery.com/regression-metrics-for-machine-learning/)
  * [image quality assessment](https://towardsdatascience.com/deep-image-quality-assessment-30ad71641fac)
  * [object detection](https://towardsdatascience.com/on-object-detection-metrics-with-worked-example-216f173ed31e)
  * [machine translation](https://medium.com/memsource-engineering/mt-metrics-explained-feeling-bleu-f62b3ab4500)
  
  The list is not exhaustive but I think it illustrates the point.  
  Also, as a side note: almost all machine learning algorithms are optimization-based, if you want to refer to approaches that don't fall into machine learning I think a better term is *analytic methods/approaches*.
  
  > **Comments:**
  > **username_2**: This is a great answer, I just want to suggest another side note. If OP is writing a scientific article, it might be worth adding another point in the Results and Discussion which is "how can the ML-based methods mimic the performance or compete with the performance of 'non-ML' based methods"? does it rediscover a certain (well-known) method? For example, an RL agent for travelling salesman problem could rediscover the greedy heuristic as shown by likelihood of taking the furthest node in each step.  
  
  ---
  
  