Answers.Body: <p>The paper <a href="https://arxiv.org/pdf/1907.13508.pdf" rel="nofollow noreferrer">Evolutionary Dataset Optimisation: learning algorithm quality through evolution</a> (2019), by Henry Wilde et al., proposes a method to generate datasets with a genetic algorithm. Their goal is to generate data for which a particular algorithm performs well, in terms of a certain metric, so that to get more insights about this algorithm and why it performs well. The individuals of the population are <em>datasets</em> (so not features of the dataset!), which can be combined with a crossover operator or mutated. The details are explained in section 2 (page 4) and they also provide nice diagrams that summarise their descriptions.</p>
  <p>The authors evaluate their approach on k-means (section 3, page 12) and they use the k-means objective function as the fitness function of the genetic algorithm.</p>
  <p>They also developed a library <a href="https://github.com/daffidwilde/edo" rel="nofollow noreferrer">edo</a> that is freely available, so you can start to play with their approach.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-13T11:01:09.767Z
Answers.Text: I only read the paper up to page 12 (this is a long paper), so I don't know about their conclusion of their approach. I suggest that you read the paper yourself to know more about it.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-13T14:46:41.793Z
Answers.Text: Thanks. I will surely take a look at it what the paper is saying.
Answers.UserDisplayName: Souradip Roy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-13T10:59:18.390Z
Answers.LastActivityDate: 2020-09-13T11:07:03.410Z
Answers.LastEditDate: 2020-09-13T11:07:03.410Z
Answers.UserDisplayName: nbro
Answers.Body: <p>This question raises a lot more questions. It seems like a solution looking for a problem, instead of the other way round.</p>
  <ul>
  <li>How do you measure the fitness of a feature?</li>
  <li>What would one of the &quot;evolved datasets&quot; <em>mean</em>? What does it represent?</li>
  <li>What would your overal purpose be? If you just wish to generate simulated datasets, there are easier ways to do this, with more control over the various aspects of the resulting datasets.</li>
  </ul>
  <p>If you want to compute a new set of features to &quot;better&quot; describe a given dataset, there are many approaches to this, such as PCA, ISOMAP, self-organizing maps, ... If this is the kind of thing you're thinking about, I would recommend starting there.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-13T14:44:38.760Z
Answers.Text: I have edited the description with a specific example since the example is more closely related to what I am thinking. Please take a look once. I guess that will surely help to understand what I am trying to say.
Answers.UserDisplayName: Souradip Roy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-12T22:21:50.660Z
Answers.LastActivityDate: 2020-09-12T22:21:50.660Z
Answers.UserDisplayName: Robby Goetschalckx
Body: <p>Genetic algorithms are used to solve many optimization tasks.</p>
  <p>If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset?</p>
  <p>We could consider each feature of the initial dataset as a chromosome (or individual), which is then combined with other chromosomes (features) to find more features. Is this possible? Has this been done?</p>
  <p>I will like to edit the details with an example so that it is easier to understand.</p>
  <p>Example: In practice cyber-security attacks evolve over time since it finds a new way to breach a system. The main draw-back of intrusion detection model is that it needs to be trained every time attack evolves. So I was hoping if genetic algorithm can be used on the present benchmarked datasets (like NSL-KDD) to come up with a futuristic type dataset maybe after X-number of generations.
  And check if a model is able to classify that generated dataset as well.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-12T13:39:22.833Z
Comments.Text: I don't really understand the problem. Let's use the Iris dataset as an example. Your features are sepal length and width and petal length and width. So you make a bunch of four-chromosome individuals based on those features. Then what? What is the fitness of one of them and what makes it better or worse than another one?
Comments.UserDisplayName: deong
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-12T13:46:43.107Z
Comments.Text: @deong Initially, the OP used the term "parent" to refer to a feature, but I edited the post to use the term "chromosome", which is more general. I don't think I changed the meaning of the post, but I added the part "which is then combined with other chromosomes (features) to find more features", given that's what the OP seemed to suggest.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-09-13T14:46:06.683Z
Comments.Text: @deong I have edited the actual description with an example which may hep you understand my question much better. Because Iris-dataset is not the type which I am referring to.
Comments.UserDisplayName: Souradip Roy
Tags: <datasets><genetic-algorithms><genetic-operators>
Title: Can we use genetic algorithms to evolve datasets?
UserDisplayName: Souradip Roy
ThreadText: # Can we use genetic algorithms to evolve datasets?
  
  (Asked by: username_0 on 2020-09-12)
  
  Genetic algorithms are used to solve many optimization tasks.
  
  If I have a dataset, can I evolve it with a genetic algorithm to create an evolved version of the same dataset?
  
  We could consider each feature of the initial dataset as a chromosome (or individual), which is then combined with other chromosomes (features) to find more features. Is this possible? Has this been done?
  
  I will like to edit the details with an example so that it is easier to understand.
  
  Example: In practice cyber-security attacks evolve over time since it finds a new way to breach a system. The main draw-back of intrusion detection model is that it needs to be trained every time attack evolves. So I was hoping if genetic algorithm can be used on the present benchmarked datasets (like NSL-KDD) to come up with a futuristic type dataset maybe after X-number of generations.
  And check if a model is able to classify that generated dataset as well.
  
  **Tags:** datasets, genetic-algorithms, genetic-operators
  
  > **Comments:**
  > **username_1**: I don't really understand the problem. Let's use the Iris dataset as an example. Your features are sepal length and width and petal length and width. So you make a bunch of four-chromosome individuals based on those features. Then what? What is the fitness of one of them and what makes it better or worse than another one?  
  > **username_2**: @username_1 Initially, the OP used the term "parent" to refer to a feature, but I edited the post to use the term "chromosome", which is more general. I don't think I changed the meaning of the post, but I added the part "which is then combined with other chromosomes (features) to find more features", given that's what the OP seemed to suggest.  
  > **username_0**: @username_1 I have edited the actual description with an example which may hep you understand my question much better. Because Iris-dataset is not the type which I am referring to.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-09-13. Score: 2
  
  The paper [Evolutionary Dataset Optimisation: learning algorithm quality through evolution](https://arxiv.org/pdf/1907.13508.pdf) (2019), by Henry Wilde et al., proposes a method to generate datasets with a genetic algorithm. Their goal is to generate data for which a particular algorithm performs well, in terms of a certain metric, so that to get more insights about this algorithm and why it performs well. The individuals of the population are *datasets* (so not features of the dataset!), which can be combined with a crossover operator or mutated. The details are explained in section 2 (page 4) and they also provide nice diagrams that summarise their descriptions.
  
  The authors evaluate their approach on k-means (section 3, page 12) and they use the k-means objective function as the fitness function of the genetic algorithm.
  
  They also developed a library [edo](https://github.com/daffidwilde/edo) that is freely available, so you can start to play with their approach.
  
  > **Comments:**
  > **username_2**: I only read the paper up to page 12 (this is a long paper), so I don't know about their conclusion of their approach. I suggest that you read the paper yourself to know more about it.  
  > **username_0**: Thanks. I will surely take a look at it what the paper is saying.  
  
  ---
  
  ### Answer by username_3 on 2020-09-12. Score: 2
  
  This question raises a lot more questions. It seems like a solution looking for a problem, instead of the other way round.
  
  * How do you measure the fitness of a feature?
  * What would one of the "evolved datasets" *mean*? What does it represent?
  * What would your overal purpose be? If you just wish to generate simulated datasets, there are easier ways to do this, with more control over the various aspects of the resulting datasets.
  
  If you want to compute a new set of features to "better" describe a given dataset, there are many approaches to this, such as PCA, ISOMAP, self-organizing maps, ... If this is the kind of thing you're thinking about, I would recommend starting there.
  
  > **Comments:**
  > **username_0**: I have edited the description with a specific example since the example is more closely related to what I am thinking. Please take a look once. I guess that will surely help to understand what I am trying to say.  
  
  ---
  
  