Answers.Body: <p>The DEAP library (a Python library for EAs) contains some <a href="https://github.com/DEAP/deap/tree/master/deap/benchmarks" rel="nofollow noreferrer">benchmarks</a>. In particular, you may want to look at the following functions</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Function(s)</th>
  <th>Reference/paper</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L364" rel="nofollow noreferrer"><code>kursawe</code></a></td>
  <td>?</td>
  </tr>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L379" rel="nofollow noreferrer"><code>schaffer_mo</code></a></td>
  <td><a href="https://www.researchgate.net/publication/220885605_Multiple_Objective_Optimization_with_Vector_Evaluated_Genetic_Algorithms" rel="nofollow noreferrer">[1]</a></td>
  </tr>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L467" rel="nofollow noreferrer"><code>dtlz1</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L495" rel="nofollow noreferrer"><code>dtlz2</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L523" rel="nofollow noreferrer"><code>dtlz3</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L550" rel="nofollow noreferrer"><code>dtlz4</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L579" rel="nofollow noreferrer"><code>dtlz5</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L599" rel="nofollow noreferrer"><code>dtlz6</code></a>, <a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L619" rel="nofollow noreferrer"><code>dtlz7</code></a></td>
  <td><a href="https://www.researchgate.net/publication/3949503_Scalable_multi-objective_optimization_test_problems" rel="nofollow noreferrer">[2]</a></td>
  </tr>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L630" rel="nofollow noreferrer"><code>fonseca</code></a></td>
  <td><a href="https://eprints.whiterose.ac.uk/79978/1/acse%20research%20report%20565.pdf" rel="nofollow noreferrer">[3]</a></td>
  </tr>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L645" rel="nofollow noreferrer"><code>poloni</code></a></td>
  <td><a href="https://www.researchgate.net/publication/243686783_Hybrid_GA_for_multi_objective_aerodynamic_shape_optimization" rel="nofollow noreferrer">[4]</a></td>
  </tr>
  <tr>
  <td><a href="https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L670" rel="nofollow noreferrer"><code>dent</code></a></td>
  <td><a href="https://www.researchgate.net/publication/40906642_Computing_Gap_Free_Pareto_Front_Approximations_with_Stochastic_Search_Algorithms" rel="nofollow noreferrer">[5]</a></td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Besides the DEAP library repository, you might want to look at several well established MOEA frameworks such as <a href="https://pymoo.org/" rel="nofollow noreferrer">PyMoo</a> (Python) and <a href="https://github.com/BIMK/PlatEMO" rel="nofollow noreferrer">PlatEMO</a> (MatLab). Both have implementations of well known MOEAs and benchmark functions. You can look through their collection of benchmark functions for inspiration, and also implement your algorithm with those framework so that you can easily test your method's performance on their benchmark functions.</p>
  <p>PlatEMO even has a GUI for experimental study, where you can choose the algorithms and the benchmark functions to test, followed by a Wilcoxon’s rank sum test to see how your algorithm really perform compared with other algorithms. For PyMoo you can create a new test file by following the existing examples, but, as far as I know, it doesn't have an experimental study platform similar to PlatEMO yet.</p>
  <p>You may also be interested in the paper <a href="https://arxiv.org/abs/2001.11591" rel="nofollow noreferrer">Scalable and Customizable Benchmark Problems for Many-Objective Optimization</a> (2020).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-16T13:53:45.847Z
Answers.Text: +1 for mentioning the DEAP library, knowing that there are several well established Multi Objective Optimization libraries, I think the OP can start browsing benchmark function provided in the libraries, besides the obvious benefit of using the library as the platform to implement, evaluate, and compare the OP proposed algorithm with other algorithms.
Answers.UserDisplayName: Sanyou
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-10-16T13:23:09.720Z
Answers.LastActivityDate: 2021-10-16T23:40:26.467Z
Answers.LastEditDate: 2021-10-16T23:40:26.467Z
Answers.UserDisplayName: nbro
Body: <p>I'm working on a project for an evolutionary algorithms course, and the problem we're trying to solve is multi-objective. We'll use NSGA-II but we also wanted to compare with some other MOEAs, however, we haven't been able to find good comparisons/benchmarks of these algorithms, so we don't really know how to decide.</p>
  <p>Any insights will be appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-16T12:46:43.963Z
Comments.Text: There are like a whole lot of it, try ZDT, WFG, some also have "complicated" Pareto set (I think it's from MOEA/D-DE paper),  look at newer MOEA papers and see the references of the benchmark functions they use.
Comments.UserDisplayName: Sanyou
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-16T13:35:23.270Z
Comments.Text: @Sanyou If you have an answer to the question (which adds something to the existing answers), you should formally write it below.
Comments.UserDisplayName: nbro
Tags: <reference-request><evolutionary-algorithms><benchmarks><moea><nsga-2>
Title: Is there a benchmark for multi-objective evolutionary algorithms?
UserDisplayName: Matías Santurio
ThreadText: # Is there a benchmark for multi-objective evolutionary algorithms?
  
  (Asked by: username_0 on 2021-10-15)
  
  I'm working on a project for an evolutionary algorithms course, and the problem we're trying to solve is multi-objective. We'll use NSGA-II but we also wanted to compare with some other MOEAs, however, we haven't been able to find good comparisons/benchmarks of these algorithms, so we don't really know how to decide.
  
  Any insights will be appreciated.
  
  **Tags:** reference-request, evolutionary-algorithms, benchmarks, moea, nsga-2
  
  > **Comments:**
  > **username_1**: There are like a whole lot of it, try ZDT, WFG, some also have "complicated" Pareto set (I think it's from MOEA/D-DE paper),  look at newer MOEA papers and see the references of the benchmark functions they use.  
  > **username_2**: @username_1 If you have an answer to the question (which adds something to the existing answers), you should formally write it below.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-10-16. Score: 2 | Accepted answer
  
  The DEAP library (a Python library for EAs) contains some [benchmarks](https://github.com/DEAP/deap/tree/master/deap/benchmarks). In particular, you may want to look at the following functions
  
  | Function(s) | Reference/paper |
  | --- | --- |
  | [`kursawe`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L364) | ? |
  | [`schaffer_mo`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L379) | [[1]](https://www.researchgate.net/publication/220885605_Multiple_Objective_Optimization_with_Vector_Evaluated_Genetic_Algorithms) |
  | [`dtlz1`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L467), [`dtlz2`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L495), [`dtlz3`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L523), [`dtlz4`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L550), [`dtlz5`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L579), [`dtlz6`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L599), [`dtlz7`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L619) | [[2]](https://www.researchgate.net/publication/3949503_Scalable_multi-objective_optimization_test_problems) |
  | [`fonseca`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L630) | [[3]](https://eprints.whiterose.ac.uk/79978/1/acse%20research%20report%20565.pdf) |
  | [`poloni`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L645) | [[4]](https://www.researchgate.net/publication/243686783_Hybrid_GA_for_multi_objective_aerodynamic_shape_optimization) |
  | [`dent`](https://github.com/DEAP/deap/blob/f4b77759897d0322ab5a6551106b28f6f4401a4e/deap/benchmarks/__init__.py#L670) | [[5]](https://www.researchgate.net/publication/40906642_Computing_Gap_Free_Pareto_Front_Approximations_with_Stochastic_Search_Algorithms) |
  
  Besides the DEAP library repository, you might want to look at several well established MOEA frameworks such as [PyMoo](https://pymoo.org/) (Python) and [PlatEMO](https://github.com/BIMK/PlatEMO) (MatLab). Both have implementations of well known MOEAs and benchmark functions. You can look through their collection of benchmark functions for inspiration, and also implement your algorithm with those framework so that you can easily test your method's performance on their benchmark functions.
  
  PlatEMO even has a GUI for experimental study, where you can choose the algorithms and the benchmark functions to test, followed by a Wilcoxon’s rank sum test to see how your algorithm really perform compared with other algorithms. For PyMoo you can create a new test file by following the existing examples, but, as far as I know, it doesn't have an experimental study platform similar to PlatEMO yet.
  
  You may also be interested in the paper [Scalable and Customizable Benchmark Problems for Many-Objective Optimization](https://arxiv.org/abs/2001.11591) (2020).
  
  > **Comments:**
  > **username_1**: +1 for mentioning the DEAP library, knowing that there are several well established Multi Objective Optimization libraries, I think the OP can start browsing benchmark function provided in the libraries, besides the obvious benefit of using the library as the platform to implement, evaluate, and compare the OP proposed algorithm with other algorithms.  
  
  ---
  
  