Answers.Body: <p>I have trialed the free version of InsideDNA, and these were my notes:</p>
  
  <ul>
  <li>Cost: \$225/month for a team of 5 with 50TB storage or \$45/month with 10TB storage for individuals (assuming 6 month package: <a href="https://insidedna.me/pricing" rel="noreferrer">https://insidedna.me/pricing</a>).</li>
  <li>Software installed: Around 600 bioinformatic tools available and standard command line tools; some popular tools missing (like <a href="http://weizhongli-lab.org/cd-hit/" rel="noreferrer">CD-HIT</a>), but should be possible to install on request.</li>
  <li>Jobs: Maximum of 32 CPUs and 208 RAM per job submission. Test jobs generally worked, although a larger test job failed.</li>
  <li>Other points: Command line was sometimes slow, <code>wget</code> queries were slow, and <code>scp</code> was blocked. However, these may be resolvable issues.</li>
  </ul>
  
  <p>Overall, I felt InsideDNA could be useful for groups without their own computational infrastructure and could be used for easily sharing resources between groups. The packages on offer seem not expensive, but I had a few issues, and I don't know how good their sys admin support would be.</p>
  
  <p>I have not used the Amazon service, so can't comment beyond the details on their website. Also there are a few alternative companies, such as <a href="https://genestack.com/" rel="noreferrer">Genestack</a> and <a href="https://www.dnanexus.com/" rel="noreferrer">DNAnexus</a>, but I haven't directly tested them either.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T12:51:03.997Z
Answers.LastActivityDate: 2017-08-03T06:09:39.177Z
Answers.LastEditDate: 2017-08-03T06:09:39.177Z
Answers.UserDisplayName: Chris_Rands
Answers.Body: <p>I'm not sure what kinds of bioinformatics tasks you would like to perform, therefore it is difficult to give a good recommendation.</p>
  
  <p>If you're specifically working on statistical genetics, I can recommend <a href="https://hail.is" rel="nofollow noreferrer">Hail</a> [1]. Hail is an open-source tool for analyzing genetics data at the tens of terabyte scale. Most of Hail's users do their science in Jupyter notebooks that are backed by Google Cloud Platform Dataproc clusters. Hail permits you to perform a variety of statistical genetics tasks including:</p>
  
  <ul>
  <li>filtering and aggregation for quality control</li>
  <li>subsetting, linear regression, linear mixed model regression, and linear burden testing</li>
  <li>utilities for computing various measures of relatedness</li>
  <li>principal components analysis</li>
  <li>variant splitting</li>
  <li>import/export from a variety of formats including PLINK, VCF, and BGEN, and</li>
  <li>a python API which enables the use of libraries like matplotlib for plotting analysis results</li>
  </ul>
  
  <p>To learn specifically about using Hail with the Google Cloud Platform and Jupyter notebooks, I strongly recommend <a href="http://discuss.hail.is/t/using-hail-with-jupyter-notebooks-on-google-cloud/196/2" rel="nofollow noreferrer">Liam's Hail forum post about his cloud-tools repository</a>.</p>
  
  <p>Here's an example, from the <a href="https://hail.is/hail/tutorials/hail-overview.html#Quality-Control" rel="nofollow noreferrer">Hail tutorial</a>, of using Hail to perform some quality control and display a scatter plot of the first two principal components of the individuals:</p>
  
  <pre class="lang-py prettyprint-override"><code>from hail import *
  import matplotlib.pyplot as plt
  import matplotlib.patches as mpatches
  
  hc = HailContext()
  
  table = hc.import_table('data/1kg_annotations.txt', impute=True).key_by('Sample')
  common_vds = (hc.read('data/1kg.vds')
                .annotate_samples_table(table, root='sa')
                .sample_qc()
                .filter_samples_expr('sa.qc.dpMean &gt;= 4 &amp;&amp; sa.qc.callRate &gt;= 0.97')
                .filter_genotypes('''let ab = g.ad[1] / g.ad.sum() in
                           ((g.isHomRef &amp;&amp; ab &lt;= 0.1) ||
                            (g.isHet &amp;&amp; ab &gt;= 0.25 &amp;&amp; ab &lt;= 0.75) ||
                            (g.isHomVar &amp;&amp; ab &gt;= 0.9))''')
                .variant_qc()
                .filter_variants_expr('va.qc.AF &gt; 0.01')
                .ld_prune(memory_per_core=512, num_cores=4))
  
  pca = common_vds.pca('sa.pca', k=5, eigenvalues='global.eigen')
  pca_table = pca.samples_table().to_pandas()
  
  colors = {'AFR': 'green', 'AMR': 'red', 'EAS': 'black', 'EUR': 'blue', 'SAS': 'cyan'}
  plt.scatter(pca_table["sa.pca.PC1"], pca_table["sa.pca.PC2"],
              c = pca_table["sa.SuperPopulation"].map(colors),
              alpha = .5)
  plt.xlim(-0.6, 0.6)
  plt.xlabel("PC1")
  plt.ylabel("PC2")
  legend_entries = [mpatches.Patch(color=c, label=pheno) for pheno, c in colors.items()]
  plt.legend(handles=legend_entries, loc=2)
  plt.show()
  </code></pre>
  
  <p>[1] Disclaimer: I work on Hail</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-07-25T13:07:46.147Z
Answers.LastActivityDate: 2017-08-15T17:50:38.403Z
Answers.LastEditDate: 2017-08-15T17:50:38.403Z
Answers.UserDisplayName: Daniel King
Answers.Body: <p>Depending on your applications and uses, you might be interested in checking out <a href="http://www.cyverse.org" rel="nofollow noreferrer">CyVerse</a>. It is an NSF funded initiative that provides you with data storage, high performance computing resources, and easy access to commonly used tools. As far as I know, it is free to use once you have an account. I also usually encounter it being used with plant and microbial genomics, so not sure how it will work with something like human genomics projects. But might be worth checking out at least. :)</p>
  
  <p>More information: <a href="http://www.cyverse.org/about" rel="nofollow noreferrer">http://www.cyverse.org/about</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T13:56:06.117Z
Answers.LastActivityDate: 2017-06-12T13:56:06.117Z
Answers.UserDisplayName: G_Hannigan
Answers.Body: <p>DNAnexus -- <a href="http://dnanexus.com" rel="nofollow noreferrer">http://dnanexus.com</a></p>
  
  <p>BaseSpace -- <a href="http://basespace.illumina.com" rel="nofollow noreferrer">http://basespace.illumina.com</a></p>
  
  <p>Seven Bridges Genomics -- <a href="http://www.sbgenomics.com" rel="nofollow noreferrer">http://www.sbgenomics.com</a></p>
  
  <p>Curoverse <a href="http://curoverse.com" rel="nofollow noreferrer">http://curoverse.com</a></p>
  
  <p>InsideDNA <a href="http://insidedna.me/" rel="nofollow noreferrer">http://insidedna.me/</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-03T15:23:45.557Z
Answers.LastActivityDate: 2017-08-03T15:23:45.557Z
Answers.UserDisplayName: 719016
Answers.Body: <p>There are tonnes of them. On top of the excellent ones everybody mentioned</p>
  
  <ol>
  <li><p>iRods</p></li>
  <li><p>Arvados</p></li>
  <li><p>Galaxy</p></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-07T09:56:15.517Z
Answers.Text: Neither Galaxy nor iRods are cloud computing services. I suspect that Arvados isn't either, but I've never used it.
Answers.UserDisplayName: Devon Ryan
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-08-06T22:47:16.617Z
Answers.LastActivityDate: 2017-08-06T22:47:16.617Z
Answers.UserDisplayName: alpha_989
Answers.Body: <h3>Google Genomics</h3>
  <p>Google has an API called <a href="https://cloud.google.com/genomics" rel="nofollow noreferrer">Google Genomics</a>.</p>
  <h3>SNPedia</h3>
  <p>&quot;SNPedia is a wiki investigating human genetics.&quot;
  snpedia.com</p>
  <h3>Promethease</h3>
  <p>&quot;Promethease is a literature retrieval system that builds a personal DNA report&quot;
  promethease.com</p>
  <h3>DNA Land</h3>
  <p>&quot;Compare DNA with reference data from different populations&quot;
  dna.land</p>
  <h3>The CyDAS Project</h3>
  <p>And, there's the CyDAS project which has an API that can analyze ISCN formulae. Per their <a href="http://www.cydas.org" rel="nofollow noreferrer">web site</a>: their API &quot;lets you analyze a Karyotype for virtually all information which can be extracted from karyotypes and the rearrangements therein: gains and losses of chromosomal material, break points, junctions...&quot; It's a <strong>free</strong> service, but I don't know how up to date it is.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-12T20:17:58.167Z
Answers.LastActivityDate: 2017-06-13T19:07:49.597Z
Answers.LastEditDate: 2020-06-18T08:30:41.743Z
Answers.UserDisplayName: exaudio
Answers.Body: <p>It really depends on what you are trying to do, but here are a few services that I know of.</p>
  
  <ul>
  <li><a href="https://cloud.google.com/genomics/v1alpha2/gatk" rel="nofollow noreferrer">GATK on Google Genomics Cloud</a>: Google and the Broad offer a cloud instance tailored to GATK pipelines.</li>
  <li><a href="https://aws.amazon.com/health/genomics/" rel="nofollow noreferrer">Genomics on Amazon Web Services</a>: I don't think there is anything that makes this unique, but Amazon offers some resources to help get started with genomics/life sciences-centered cloud solutions.</li>
  <li><a href="https://www.illumina.com/informatics.html" rel="nofollow noreferrer">Illumina Bioinformatics</a>: Illumina is working on a whole suite of bioinformatics software for the cloud.</li>
  <li><a href="http://www.cancergenomicscloud.org/" rel="nofollow noreferrer">Cancer Genomics Cloud</a>: This is specific to cancer genomics, but I believe Seven Bridges allows you to push all sorts of data into the tool and analyze it.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-06-13T20:46:40.447Z
Answers.LastActivityDate: 2017-06-13T20:46:40.447Z
Answers.UserDisplayName: woemler
Body: <p>I'm looking for cloud computing services that can be used for doing bioinformatics. An example I found is <a href="https://insidedna.me" rel="noreferrer">InsideDNA</a> and there is <a href="https://aws.amazon.com/health/genomics/" rel="noreferrer">Amazon</a> of course.
  A little description of these would be appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T12:13:43.500Z
Comments.Text: Can you mention Google compute as well and further specify what exactly you're interested in knowing? As is I expect this will end up getting closed otherwise.
Comments.UserDisplayName: Devon Ryan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T12:38:37.127Z
Comments.Text: What do you want to use the computer services for? Do you have access to a university, research institute, or other academic facility? Is there a particular reason why your desktop or laptop is not appropriate?
Comments.UserDisplayName: gringer
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T15:47:34.163Z
Comments.Text: What kind of computation are you interested in? There are specialised companies for bioinformatics that are outsourcing the computations on google cloud or amazon. I think for different types of computations you should contact a different company...
Comments.UserDisplayName: Kamil S Jaron
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T17:00:42.047Z
Comments.Text: At the moment this is a purely theoretical question as the computing facilities at my university are more than adequate for RNA-seq data analysis. I'm just looking for a simple list of companies that provide this service, because a simple search does not return many. In my ideal environment I do not depend on pre-installed software (i.e. I can install software), command line (bash) interaction is fast and I can trust the company for backups etc -- so I'm also interested in the location of the provider, how long has it been around and the computing resources they have.
Comments.UserDisplayName: Peter
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T20:28:51.757Z
Comments.Text: I do not think that this is a good question since there is no one good answer (or at least the answer should a list, not a one provider), so I will just link the providers I know : [DNAnexus](https://www.dnanexus.com/) (private) and [Vital-it](http://www.vital-it.ch/) (academic)
Comments.UserDisplayName: Kamil S Jaron
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T20:49:09.600Z
Comments.Text: This post is too broad.
Comments.UserDisplayName: zx8754
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-06-12T20:57:45.660Z
Comments.Text: Related post: https://www.biostars.org/p/86463/
Comments.UserDisplayName: zx8754
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-26T06:30:01.217Z
Comments.Text: What kind of bioinformatics analysis you are planning to do?
Comments.UserDisplayName: arup
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-07-26T15:55:51.313Z
Comments.Text: @arupgsh See my comment above
Comments.UserDisplayName: Peter
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-11-02T00:43:02.817Z
Comments.Text: I put this on hold since asking for lists like this is really not a very good fit for the Q&A format and better suited to something like a forum where a discussion can be had. See [this old blog post](https://stackoverflow.blog/2010/11/23/qa-is-hard-lets-go-shopping/) for an explanation of the general rationale behind considering this type of question off topic.
Comments.UserDisplayName: terdon
Tags: <linux>
Title: What are the available cloud computing services for bioinformatics?
ThreadText: # What are the available cloud computing services for bioinformatics?
  
  (Asked by: username_0 on 2017-06-12)
  
  I'm looking for cloud computing services that can be used for doing bioinformatics. An example I found is [InsideDNA](https://insidedna.me) and there is [Amazon](https://aws.amazon.com/health/genomics/) of course.
  A little description of these would be appreciated.
  
  **Tags:** linux
  
  > **Comments:**
  > **username_1**: Can you mention Google compute as well and further specify what exactly you're interested in knowing? As is I expect this will end up getting closed otherwise.  
  > **username_2**: What do you want to use the computer services for? Do you have access to a university, research institute, or other academic facility? Is there a particular reason why your desktop or laptop is not appropriate?  
  > **username_3**: What kind of computation are you interested in? There are specialised companies for bioinformatics that are outsourcing the computations on google cloud or amazon. I think for different types of computations you should contact a different company...  
  > **username_0**: At the moment this is a purely theoretical question as the computing facilities at my university are more than adequate for RNA-seq data analysis. I'm just looking for a simple list of companies that provide this service, because a simple search does not return many. In my ideal environment I do not depend on pre-installed software (i.e. I can install software), command line (bash) interaction is fast and I can trust the company for backups etc -- so I'm also interested in the location of the provider, how long has it been around and the computing resources they have.  
  > **username_3**: I do not think that this is a good question since there is no one good answer (or at least the answer should a list, not a one provider), so I will just link the providers I know : [DNAnexus](https://www.dnanexus.com/) (private) and [Vital-it](http://www.vital-it.ch/) (academic)  
  > **username_4**: This post is too broad.  
  > **username_4**: Related post: https://www.biostars.org/p/86463/  
  > **username_5**: What kind of bioinformatics analysis you are planning to do?  
  > **username_0**: @arupgsh See my comment above  
  > **username_6**: I put this on hold since asking for lists like this is really not a very good fit for the Q&A format and better suited to something like a forum where a discussion can be had. See [this old blog post](https://stackoverflow.blog/2010/11/23/qa-is-hard-lets-go-shopping/) for an explanation of the general rationale behind considering this type of question off topic.  
  
  ---
  
  ## 7 answers
  
  ### Answer by username_13 on 2017-06-13. Score: 4 | Accepted answer
  
  It really depends on what you are trying to do, but here are a few services that I know of.
  
  * [GATK on Google Genomics Cloud](https://cloud.google.com/genomics/v1alpha2/gatk): Google and the Broad offer a cloud instance tailored to GATK pipelines.
  * [Genomics on Amazon Web Services](https://aws.amazon.com/health/genomics/): I don't think there is anything that makes this unique, but Amazon offers some resources to help get started with genomics/life sciences-centered cloud solutions.
  * [Illumina Bioinformatics](https://www.illumina.com/informatics.html): Illumina is working on a whole suite of bioinformatics software for the cloud.
  * [Cancer Genomics Cloud](http://www.cancergenomicscloud.org/): This is specific to cancer genomics, but I believe Seven Bridges allows you to push all sorts of data into the tool and analyze it.
  
  
  ---
  
  ### Answer by username_7 on 2017-06-12. Score: 9
  
  I have trialed the free version of InsideDNA, and these were my notes:
  
  * Cost: \$225/month for a team of 5 with 50TB storage or \$45/month with 10TB storage for individuals (assuming 6 month package: <https://insidedna.me/pricing>).
  * Software installed: Around 600 bioinformatic tools available and standard command line tools; some popular tools missing (like [CD-HIT](http://weizhongli-lab.org/cd-hit/)), but should be possible to install on request.
  * Jobs: Maximum of 32 CPUs and 208 RAM per job submission. Test jobs generally worked, although a larger test job failed.
  * Other points: Command line was sometimes slow, `wget` queries were slow, and `scp` was blocked. However, these may be resolvable issues.
  
  Overall, I felt InsideDNA could be useful for groups without their own computational infrastructure and could be used for easily sharing resources between groups. The packages on offer seem not expensive, but I had a few issues, and I don't know how good their sys admin support would be.
  
  I have not used the Amazon service, so can't comment beyond the details on their website. Also there are a few alternative companies, such as [Genestack](https://genestack.com/) and [DNAnexus](https://www.dnanexus.com/), but I haven't directly tested them either.
  
  
  ---
  
  ### Answer by username_8 on 2017-07-25. Score: 5
  
  I'm not sure what kinds of bioinformatics tasks you would like to perform, therefore it is difficult to give a good recommendation.
  
  If you're specifically working on statistical genetics, I can recommend [Hail](https://hail.is) [1]. Hail is an open-source tool for analyzing genetics data at the tens of terabyte scale. Most of Hail's users do their science in Jupyter notebooks that are backed by Google Cloud Platform Dataproc clusters. Hail permits you to perform a variety of statistical genetics tasks including:
  
  * filtering and aggregation for quality control
  * subsetting, linear regression, linear mixed model regression, and linear burden testing
  * utilities for computing various measures of relatedness
  * principal components analysis
  * variant splitting
  * import/export from a variety of formats including PLINK, VCF, and BGEN, and
  * a python API which enables the use of libraries like matplotlib for plotting analysis results
  
  To learn specifically about using Hail with the Google Cloud Platform and Jupyter notebooks, I strongly recommend [Liam's Hail forum post about his cloud-tools repository](http://discuss.hail.is/t/using-hail-with-jupyter-notebooks-on-google-cloud/196/2).
  
  Here's an example, from the [Hail tutorial](https://hail.is/hail/tutorials/hail-overview.html#Quality-Control), of using Hail to perform some quality control and display a scatter plot of the first two principal components of the individuals:
  
  ```py
  from hail import *
  import matplotlib.pyplot as plt
  import matplotlib.patches as mpatches
  
  hc = HailContext()
  
  table = hc.import_table('data/1kg_annotations.txt', impute=True).key_by('Sample')
  common_vds = (hc.read('data/1kg.vds')
                .annotate_samples_table(table, root='sa')
                .sample_qc()
                .filter_samples_expr('sa.qc.dpMean >= 4 && sa.qc.callRate >= 0.97')
                .filter_genotypes('''let ab = g.ad[1] / g.ad.sum() in
                           ((g.isHomRef && ab <= 0.1) ||
                            (g.isHet && ab >= 0.25 && ab <= 0.75) ||
                            (g.isHomVar && ab >= 0.9))''')
                .variant_qc()
                .filter_variants_expr('va.qc.AF > 0.01')
                .ld_prune(memory_per_core=512, num_cores=4))
  
  pca = common_vds.pca('sa.pca', k=5, eigenvalues='global.eigen')
  pca_table = pca.samples_table().to_pandas()
  
  colors = {'AFR': 'green', 'AMR': 'red', 'EAS': 'black', 'EUR': 'blue', 'SAS': 'cyan'}
  plt.scatter(pca_table["sa.pca.PC1"], pca_table["sa.pca.PC2"],
              c = pca_table["sa.SuperPopulation"].map(colors),
              alpha = .5)
  plt.xlim(-0.6, 0.6)
  plt.xlabel("PC1")
  plt.ylabel("PC2")
  legend_entries = [mpatches.Patch(color=c, label=pheno) for pheno, c in colors.items()]
  plt.legend(handles=legend_entries, loc=2)
  plt.show()
  
  ```
  
  [1] Disclaimer: I work on Hail
  
  
  ---
  
  ### Answer by username_9 on 2017-06-12. Score: 4
  
  Depending on your applications and uses, you might be interested in checking out [CyVerse](http://www.cyverse.org). It is an NSF funded initiative that provides you with data storage, high performance computing resources, and easy access to commonly used tools. As far as I know, it is free to use once you have an account. I also usually encounter it being used with plant and microbial genomics, so not sure how it will work with something like human genomics projects. But might be worth checking out at least. :)
  
  More information: <http://www.cyverse.org/about>
  
  
  ---
  
  ### Answer by username_10 on 2017-08-03. Score: 1
  
  DNAnexus -- <http://dnanexus.com>
  
  BaseSpace -- <http://basespace.illumina.com>
  
  Seven Bridges Genomics -- <http://www.sbgenomics.com>
  
  Curoverse <http://curoverse.com>
  
  InsideDNA <http://insidedna.me/>
  
  
  ---
  
  ### Answer by username_12 on 2017-06-12. Score: 1
  
  ### Google Genomics
  
  Google has an API called [Google Genomics](https://cloud.google.com/genomics).
  
  ### SNPedia
  
  "SNPedia is a wiki investigating human genetics."
  snpedia.com
  
  ### Promethease
  
  "Promethease is a literature retrieval system that builds a personal DNA report"
  promethease.com
  
  ### DNA Land
  
  "Compare DNA with reference data from different populations"
  dna.land
  
  ### The CyDAS Project
  
  And, there's the CyDAS project which has an API that can analyze ISCN formulae. Per their [web site](http://www.cydas.org): their API "lets you analyze a Karyotype for virtually all information which can be extracted from karyotypes and the rearrangements therein: gains and losses of chromosomal material, break points, junctions..." It's a **free** service, but I don't know how up to date it is.
  
  
  ---
  
  ### Answer by username_11 on 2017-08-06. Score: 0
  
  There are tonnes of them. On top of the excellent ones everybody mentioned
  
  1. iRods
  2. Arvados
  3. Galaxy
  
  > **Comments:**
  > **username_1**: Neither Galaxy nor iRods are cloud computing services. I suspect that Arvados isn't either, but I've never used it.  
  
  ---
  
  