Answers.Body: <p>A generic way to start with under such circumstances is to try find an "oracle".</p>
  
  <p>Serial to parallel converters exist for quite some times, and some are open source (e.g. <a href="https://pips4u.org/" rel="nofollow noreferrer">PIPS</a>). The idea is to get serial code from step (1), use the "oracle" to produce parallel code, and that's it: Each conversion makes en entry in the dataset.</p>
  
  <p>Ensuring the quality of the dataset is critical here. A script generating the dataset shoud ensure (1) the serial code compiles and runs properlly, (2) the parallelized code compiles and runs properly too, (3) serial and parallel programs produce the same result, (4) some metrics state objectively how the parallel version does against the serial version, and (5) keep track of the actual hardware configuration.</p>
  
  <p>Point (4) is critical to the quality of such a dataset. Parallel programs are <em>not</em> always faster than a serial version: A 1000-iteration loop dumped over 1000 workers on a 8-core CPU may not do so well compared to 8 workers---we need to check what the converter is doing. And point (5) ensures we know under which conditions the data is valid.</p>
  
  <p>Using several "oracles" would be even better, to bring diversity, and to hopefully get the learning algorithm discover the best conversion tradeoffs---perhaps better than what the carefully hand-crafted converters are able to do on their own.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-26T15:32:47.360Z
Answers.Text: In addition to PIPS: [DawnCC](http://cuda.dcc.ufmg.br/dawn/index.php) and [bones](http://parse.ele.tue.nl/research/bones/).
Answers.UserDisplayName: Ilya Palachev
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-27T14:16:30.053Z
Answers.LastActivityDate: 2018-05-27T14:16:30.053Z
Answers.UserDisplayName: Eric Platon
Body: <p>I'm looking to perform two tasks:</p>
  
  <ul>
  <li><p>Train a classifier to classify code as serial or parallel</p></li>
  <li><p>Train a generative algorithm to generate parallel code from serial </p></li>
  </ul>
  
  <p>For the first task a simple scraper can scrape random C and C++ code from git, however for the second step I would need a decently large source of examples of serial to parallel code. Any ideas or pointers for existing or creating this type of dataset would be greatly appreciated.</p>
  
Tags: <classification><datasets><generative-model>
Title: Looking to build, compile, and/or find dataset for serial-parallelized code examples
UserDisplayName: JMed
ThreadText: # Looking to build, compile, and/or find dataset for serial-parallelized code examples
  
  (Asked by: username_0 on 2018-04-06)
  
  I'm looking to perform two tasks:
  
  * Train a classifier to classify code as serial or parallel
  * Train a generative algorithm to generate parallel code from serial
  
  For the first task a simple scraper can scrape random C and C++ code from git, however for the second step I would need a decently large source of examples of serial to parallel code. Any ideas or pointers for existing or creating this type of dataset would be greatly appreciated.
  
  **Tags:** classification, datasets, generative-model
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-05-27. Score: 1
  
  A generic way to start with under such circumstances is to try find an "oracle".
  
  Serial to parallel converters exist for quite some times, and some are open source (e.g. [PIPS](https://pips4u.org/)). The idea is to get serial code from step (1), use the "oracle" to produce parallel code, and that's it: Each conversion makes en entry in the dataset.
  
  Ensuring the quality of the dataset is critical here. A script generating the dataset shoud ensure (1) the serial code compiles and runs properlly, (2) the parallelized code compiles and runs properly too, (3) serial and parallel programs produce the same result, (4) some metrics state objectively how the parallel version does against the serial version, and (5) keep track of the actual hardware configuration.
  
  Point (4) is critical to the quality of such a dataset. Parallel programs are *not* always faster than a serial version: A 1000-iteration loop dumped over 1000 workers on a 8-core CPU may not do so well compared to 8 workers---we need to check what the converter is doing. And point (5) ensures we know under which conditions the data is valid.
  
  Using several "oracles" would be even better, to bring diversity, and to hopefully get the learning algorithm discover the best conversion tradeoffs---perhaps better than what the carefully hand-crafted converters are able to do on their own.
  
  > **Comments:**
  > **username_2**: In addition to PIPS: [DawnCC](http://cuda.dcc.ufmg.br/dawn/index.php) and [bones](http://parse.ele.tue.nl/research/bones/).  
  
  ---
  
  