Answers.Body: <p>It is somewhat common to measure the number of <em>operations</em> involved in the search. This is only really useful if you're doing scientific work, because the computational cost of measuring it accurately is quite high. For example, if you were using a GA that used tournament-based fitness selection, and wanted to compare it to one that used round-robin selection, counting the number of evaluations of individuals would be a good measure of total computational effort.</p>
  
  <p>In Genetic Programming, it is fairly common to build the measurement of the total op count into the interpreter you write for the programs. You can then compare that directly to the number of evals times the length of the genomes for something like a GA.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-24T02:06:04.840Z
Answers.LastActivityDate: 2019-09-24T02:06:04.840Z
Answers.UserDisplayName: John Doucette
Body: <p>Considering a black box optimization problem on non-linear, non-convex function where we want to minimize an objective function.</p>
  
  <p>One way to assess the quality of an optimizer is to look at the best solution it finds. However that doesn't give us any information on how much of the parameter space the optimizer had to explore to come up with these solutions. </p>
  
  <p>Therefore I was wondering if there are metrics quantifying how much of the parameter space is explored ?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-23T18:32:37.603Z
Comments.Text: Number of iterations is kind of measure of parameter space explored. If you want something more advanced you first have to specify strictly what do you mean by "explored". You have to specify type of coverage associated with set of parameter vectors.
Comments.UserDisplayName: mirror2image
Tags: <algorithm><genetic-algorithms><optimization>
Title: Metrics of quality of parameter space exploration
UserDisplayName: Phaune
ThreadText: # Metrics of quality of parameter space exploration
  
  (Asked by: username_0 on 2019-09-16)
  
  Considering a black box optimization problem on non-linear, non-convex function where we want to minimize an objective function.
  
  One way to assess the quality of an optimizer is to look at the best solution it finds. However that doesn't give us any information on how much of the parameter space the optimizer had to explore to come up with these solutions.
  
  Therefore I was wondering if there are metrics quantifying how much of the parameter space is explored ?
  
  **Tags:** algorithm, genetic-algorithms, optimization
  > **Comments:**
  > **username_1**: Number of iterations is kind of measure of parameter space explored. If you want something more advanced you first have to specify strictly what do you mean by "explored". You have to specify type of coverage associated with set of parameter vectors.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-09-24. Score: 1
  
  It is somewhat common to measure the number of *operations* involved in the search. This is only really useful if you're doing scientific work, because the computational cost of measuring it accurately is quite high. For example, if you were using a GA that used tournament-based fitness selection, and wanted to compare it to one that used round-robin selection, counting the number of evaluations of individuals would be a good measure of total computational effort.
  
  In Genetic Programming, it is fairly common to build the measurement of the total op count into the interpreter you write for the programs. You can then compare that directly to the number of evals times the length of the genomes for something like a GA.
  
  
  ---
  
  