Answers.Body: <p>As a more direct answer to your question as stated in the question title, finding a global minimum means we have found the optimal solution for <em>that model</em>.  This doesn't mean that the model necessarily works particularly well.  Hence the saying from George Box, &quot;All models are wrong, some models are useful.&quot;</p>
  <p>In other words, while you may have found an optimal state for that model, the model may not be optimal or even acceptable for the task.  (Note that that phrase can be interpreted in different ways, and this is not the only way to apply it.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-07T11:36:32.633Z
Answers.LastActivityDate: 2023-03-07T11:36:32.633Z
Answers.UserDisplayName: David Hoelzer
Answers.Body: <p>There is an existence of a loss/reward function for any task that can be evaluated, but this does not mean that function generalizes to any model.</p>
  <p>In your prompt you mention &quot;human-level performance&quot;, this assumes a metric such as accuracy, auc, precision, winning %, etc that humans were evaluated on for some task. Since it is possible to model any function with a boundless resource budget we can then model a function that reaches any form of theoretical optimum for that metric and some given domain.</p>
  <p>Ofcourse there is issues with what I wrote above in an applicable sense:</p>
  <ol>
  <li>Infinite resource budget doesn't exist.</li>
  <li>Not all metrics can be easily calculated.</li>
  <li>We usually subset a domain to &quot;train&quot;, but to find one that truly encompasses a domain is usually not feasible and requires clever data usage and handling.</li>
  </ol>
  <p>So in practice we make simplifying assumptions that handle the above issues. Examples include finite-parameter deep learning models designed for efficient learning, differential metrics such as distance, and data augmentation. In doing so, it is possible to achieve cases where the global min/max over its parameter space performs poorly. Its the researcher/engineers job though to make this not the case.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-11T03:52:11.230Z
Answers.Text: Thank you for your answer. I would like to clarify on a few points:
  
  1. `this does not mean that function generalizes to any model` - Do you mean that not all models may be suited for this function?
  
  2. `possible to model any function` - Can everything be expressed as a function? 
  
  To give an example, I'm working on a navigation task, the network is learning a function that maps from image to action, but can we be sure that such a function exist, one that generates the correct action for all images?
Answers.UserDisplayName: tangolin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-11T09:31:12.740Z
Answers.Text: adding to it outliers would be hard to determine as you club number of task together  which could affect all your performance metrics
Answers.UserDisplayName: saurabh kamble
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-11T03:23:36.947Z
Answers.LastActivityDate: 2022-01-11T03:23:36.947Z
Answers.UserDisplayName: mshlis
Body: <p>It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. However, with AI being applied to so many fields, can we be sure that there exists a loss function for every task, such that upon reaching its global optima, it guarantees reasonably good/human-level performance in the task?</p>
  
Tags: <deep-learning><applications><optimization>
Title: Does reaching the global optima guarantee good performance in a task?
UserDisplayName: tangolin
ThreadText: # Does reaching the global optima guarantee good performance in a task?
  
  (Asked by: username_0 on 2022-01-11)
  
  It is to my understanding that, in deep learning, we are essentially trying to minimize the loss function that we have defined and reach its global optima through some form of optimization technique. However, with AI being applied to so many fields, can we be sure that there exists a loss function for every task, such that upon reaching its global optima, it guarantees reasonably good/human-level performance in the task?
  
  **Tags:** deep-learning, applications, optimization
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2023-03-07. Score: 1
  
  As a more direct answer to your question as stated in the question title, finding a global minimum means we have found the optimal solution for *that model*. This doesn't mean that the model necessarily works particularly well. Hence the saying from George Box, "All models are wrong, some models are useful."
  
  In other words, while you may have found an optimal state for that model, the model may not be optimal or even acceptable for the task. (Note that that phrase can be interpreted in different ways, and this is not the only way to apply it.)
  
  
  ---
  
  ### Answer by username_2 on 2022-01-11. Score: 0
  
  There is an existence of a loss/reward function for any task that can be evaluated, but this does not mean that function generalizes to any model.
  
  In your prompt you mention "human-level performance", this assumes a metric such as accuracy, auc, precision, winning %, etc that humans were evaluated on for some task. Since it is possible to model any function with a boundless resource budget we can then model a function that reaches any form of theoretical optimum for that metric and some given domain.
  
  Ofcourse there is issues with what I wrote above in an applicable sense:
  
  1. Infinite resource budget doesn't exist.
  2. Not all metrics can be easily calculated.
  3. We usually subset a domain to "train", but to find one that truly encompasses a domain is usually not feasible and requires clever data usage and handling.
  
  So in practice we make simplifying assumptions that handle the above issues. Examples include finite-parameter deep learning models designed for efficient learning, differential metrics such as distance, and data augmentation. In doing so, it is possible to achieve cases where the global min/max over its parameter space performs poorly. Its the researcher/engineers job though to make this not the case.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. I would like to clarify on a few points:  1. `this does not mean that function generalizes to any model` - Do you mean that not all models may be suited for this function?  2. `possible to model any function` - Can everything be expressed as a function?   To give an example, I'm working on a navigation task, the network is learning a function that maps from image to action, but can we be sure that such a function exist, one that generates the correct action for all images?  
  > **username_3**: adding to it outliers would be hard to determine as you club number of task together  which could affect all your performance metrics  
  
  ---
  
  