Answers.Body: <p>I cannot tell exactly from the post or the linked project what type of data these scripts are designed for, but I am going to assume that "Scripts involved in our workflow for detecting CNVs from WGS data using read depth-based methods" refers to short-read Illumina sequencing. </p>
  
  <p>Simulations can be helpful, but they are usually imperfect. Nowadays we have long-read data available. I would suggest obtaining some long-read data for which short-read data is also available, doing CNV detection on both datasets independently, and comparing the results. </p>
  
  <p>The problem with using long-read data is that any differences you find will be conflated with the biases of the technology. So, if you find a difference, then this approach will not tell you if the differences are due to the biases of the technologies, or due to bugs in the scripts. If you use simulations, then this conflation of biases remains.</p>
  
  <p>Short of re-writing the pipeline, the only other suggestion I have is to borrow from software testing. Try constructing a few minimal examples that might also test some corner cases and see if the scripts behave exactly as expected. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-25T09:20:53.280Z
Answers.LastActivityDate: 2018-11-25T09:29:39.173Z
Answers.LastEditDate: 2018-11-25T09:29:39.173Z
Answers.UserDisplayName: winni2k
Body: <p>I'd like to evaluate a <a href="https://github.com/bjtrost/TCAG-WGS-CNV-workflow" rel="nofollow noreferrer">CNV project</a>.</p>
  
  <p>My aim is to evaluate if the scripts are sufficient for calling reasonable CNVs. I know they have a paper, but their scripts may be buggy... and all papers are biased...</p>
  
  <p>While there are CNV simulators on Google search, none seem popular in the community. How should I approach this problem?</p>
  
  <ul>
  <li>Simulate CNV variants from a genome</li>
  <li>Simple evaluation on the called variants against my simulation?</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-26T08:45:58.853Z
Comments.Text: Did you try to follow the structure of their code? And using other metrics/programs to evaluate the CNV?
Comments.UserDisplayName: llrs
Tags: <ngs><genome><cnv>
Title: Simulate and test CNV workflow?
ThreadText: # Simulate and test CNV workflow?
  
  (Asked by: username_0 on 2018-11-24)
  
  I'd like to evaluate a [CNV project](https://github.com/bjtrost/TCAG-WGS-CNV-workflow).
  
  My aim is to evaluate if the scripts are sufficient for calling reasonable CNVs. I know they have a paper, but their scripts may be buggy... and all papers are biased...
  
  While there are CNV simulators on Google search, none seem popular in the community. How should I approach this problem?
  
  * Simulate CNV variants from a genome
  * Simple evaluation on the called variants against my simulation?
  
  **Tags:** ngs, genome, cnv
  
  > **Comments:**
  > **username_1**: Did you try to follow the structure of their code? And using other metrics/programs to evaluate the CNV?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-11-25. Score: 1
  
  I cannot tell exactly from the post or the linked project what type of data these scripts are designed for, but I am going to assume that "Scripts involved in our workflow for detecting CNVs from WGS data using read depth-based methods" refers to short-read Illumina sequencing.
  
  Simulations can be helpful, but they are usually imperfect. Nowadays we have long-read data available. I would suggest obtaining some long-read data for which short-read data is also available, doing CNV detection on both datasets independently, and comparing the results.
  
  The problem with using long-read data is that any differences you find will be conflated with the biases of the technology. So, if you find a difference, then this approach will not tell you if the differences are due to the biases of the technologies, or due to bugs in the scripts. If you use simulations, then this conflation of biases remains.
  
  Short of re-writing the pipeline, the only other suggestion I have is to borrow from software testing. Try constructing a few minimal examples that might also test some corner cases and see if the scripts behave exactly as expected.
  
  
  ---
  
  