<p>Astronomer is moving towards the big(ger) data era because of many sky survey technologies. The coming ones include, e.g., LSST, JWST, and WFIRST.</p>

<p>By the meaning of survey, it normally means observing the whole sky over a few days, and keep repeating over and over. Also, since most of the surveys are imaging technologies, every pixel in an image is important.</p>

<p>So, you can imagine how much data is incoming per day, and how much data science is important in many fields of astronomy (and cosmology). So, data science is, kind of, taking over when it comes to analyzing data from surveys.</p>

CC BY-SA 4.0
2018-12-05T16:57:00.990Z
2018-12-05T16:57:00.990Z
Kornpob Bhirombhakdi
<p>I'm an astronomer and I get lots of job offers to retrain as a data scientist, but it might be more tricky to go the other way.</p>

<p>Astronomy is definitely a field in which 'big data' is important, and the analysis and visualisation techniques we use every day are probably decades behind what is taught to computer scientists. However, most astronomy software is very niche, only used by a handful of people around the world, and usually edited to fit their particular purpose. I have seen attempts to make more 'generic' astronomical software that can be standardised and widely used, but it is difficult to find a task that can be automated and standardised because we are normally working with slightly different constraints each time.</p>

<p>A few years ago I met a PhD student who had studied computer science at undergraduate and then moved into astronomy, and he was amazed at how old the techniques we were just getting to grips with were, but he also had real trouble understanding what he was trying to analyse because he didn't have the astronomical knowledge to know all the sources of error and noise in his data.</p>

<p>I believe there are data scientists working on more abstract problems, like telescope pipelines, but unfortunately, I can't give much advice in this area as that is an observational problem and my work is theoretical.</p>

CC BY-SA 4.0
2018-12-17T13:38:24.010Z
It's simply fascinating, Man. I am definitely in. Can you give me any example of big data sets in theoretical work that you normally deal with; along with the current practices deployed in dealing with those data sets?
Zulkifl Agha
CC BY-SA 4.0
2018-12-17T14:19:07.860Z
On the theoretical side, any of the large scale simulations, such as [Illustris](http://www.illustris-project.org/) or [Millenium](https://wwwmpa.mpa-garching.mpg.de/galform/virgo/millennium/) simulations. Even my own relatively small scale simulations produce snapshots of about 2G each, and I have hundreds of snapshots as well as log files and star data.
Observationally, the largest data sets are probably from [ALMA](www.almaobservatory.org)
FJC
CC BY-SA 4.0
2018-12-06T12:00:07.050Z
2018-12-06T12:00:07.050Z
FJC
<p>I am a data scientist and curious about their roles in modern Astronomy, both now, and in the near future. What are type of roles and projects where data scientist are most active in Astronomy? What are the basic solutions provided by data scientists in astronomy. Are there data scientists as principle investigators, or are they at least team members in research efforts?</p>

CC BY-SA 4.0
2018-12-05T13:26:50.550Z
Some examples are provided in this coursera tutorial: https://www.coursera.org/learn/data-driven-astronomy/home/welcome
Nickpick
CC BY-SA 4.0
2018-12-05T23:54:57.870Z
As a random example of the intersection of data science and astronomy see http://jakevdp.github.io/pages/about.html as well as http://vanderplas.com/ and also https://www.youtube.com/watch?v=U51oPYSVU74 start at 23:30 for "big data flow"
uhoh
CC BY-SA 4.0
2018-12-18T12:09:32.070Z
I've made some edits to remove risk of "primarily opinion-based answers", hopefully can be reopened.
uhoh
<data-analysis><mathematics><future>
What are the present, and potentially the future roles of data scientists in Astronomy?
Zulkifl Agha
# What are the present, and potentially the future roles of data scientists in Astronomy?

(Asked by: username_0 on 2018-12-05)

I am a data scientist and curious about their roles in modern Astronomy, both now, and in the near future. What are type of roles and projects where data scientist are most active in Astronomy? What are the basic solutions provided by data scientists in astronomy. Are there data scientists as principle investigators, or are they at least team members in research efforts?

**Tags:** data-analysis, mathematics, future
> **Comments:**
> **username_1**: Some examples are provided in this coursera tutorial: https://www.coursera.org/learn/data-driven-astronomy/home/welcome  
> **username_2**: As a random example of the intersection of data science and astronomy see http://jakevdp.github.io/pages/about.html as well as http://vanderplas.com/ and also https://www.youtube.com/watch?v=U51oPYSVU74 start at 23:30 for "big data flow"  
> **username_2**: I've made some edits to remove risk of "primarily opinion-based answers", hopefully can be reopened.  

---

## 2 answers

### Answer by username_3 on 2018-12-05. Score: 5

Astronomer is moving towards the big(ger) data era because of many sky survey technologies. The coming ones include, e.g., LSST, JWST, and WFIRST.

By the meaning of survey, it normally means observing the whole sky over a few days, and keep repeating over and over. Also, since most of the surveys are imaging technologies, every pixel in an image is important.

So, you can imagine how much data is incoming per day, and how much data science is important in many fields of astronomy (and cosmology). So, data science is, kind of, taking over when it comes to analyzing data from surveys.


---

### Answer by username_4 on 2018-12-06. Score: 3

I'm an astronomer and I get lots of job offers to retrain as a data scientist, but it might be more tricky to go the other way.

Astronomy is definitely a field in which 'big data' is important, and the analysis and visualisation techniques we use every day are probably decades behind what is taught to computer scientists. However, most astronomy software is very niche, only used by a handful of people around the world, and usually edited to fit their particular purpose. I have seen attempts to make more 'generic' astronomical software that can be standardised and widely used, but it is difficult to find a task that can be automated and standardised because we are normally working with slightly different constraints each time.

A few years ago I met a PhD student who had studied computer science at undergraduate and then moved into astronomy, and he was amazed at how old the techniques we were just getting to grips with were, but he also had real trouble understanding what he was trying to analyse because he didn't have the astronomical knowledge to know all the sources of error and noise in his data.

I believe there are data scientists working on more abstract problems, like telescope pipelines, but unfortunately, I can't give much advice in this area as that is an observational problem and my work is theoretical.

> **Comments:**
> **username_0**: It's simply fascinating, Man. I am definitely in. Can you give me any example of big data sets in theoretical work that you normally deal with; along with the current practices deployed in dealing with those data sets?  
> **username_4**: On the theoretical side, any of the large scale simulations, such as [Illustris](http://www.illustris-project.org/) or [Millenium](https://wwwmpa.mpa-garching.mpg.de/galform/virgo/millennium/) simulations. Even my own relatively small scale simulations produce snapshots of about 2G each, and I have hundreds of snapshots as well as log files and star data. Observationally, the largest data sets are probably from [ALMA](www.almaobservatory.org)  

---

