<h2>AstroPy</h2>

<p>Jake Vanderplas (Director of Open Software at the University of Washington eScience Institute) has written a Pythonic Preambulations post <a href="https://jakevdp.github.io/blog/2015/06/13/lomb-scargle-in-python/" rel="nofollow noreferrer">Fast Lomb-Scargle Periodograms in Python</a> which <em>by definition appears to be</em> a good place to start to learn how to do Lomb-Scargle Periodograms... Fast... in Python. </p>

<p>I have not read it yet in full, but I've found other posts of his to be extremely useful. </p>

<p>He starts out by explaining the implementation in <a href="https://www.astropy.org/" rel="nofollow noreferrer">AstroPy</a> which if you are going to be doing Astronomy, is probably a good thing to become familliar with.</p>

<h2>SciPy</h2>

<p>Alternatively, there is <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lombscargle.html" rel="nofollow noreferrer"><code>scipy.signal.lombscargle</code></a> which you can try assuming you have a current version of SciPy installed but want to try something different than AstroPy or want to compare performance or results. </p>

<h2>StackOverflow</h2>

<p>I found <a href="https://stackoverflow.com/search?q=Lomb-Scargle">over 40 posts containing 'Lomb-Scargle' in Stack Overflow</a> which is a very good sign!</p>

<p>I notice that you've been active there and know how to ask well-received questions. Since your asking about speed and errors, StackOverflow is your go-to source! However, don't forget to be as clear as possible and quote all error messages and symptoms, and see if you can somehow add a short Minimum Complete and Verifiable Example or <a href="https://meta.stackoverflow.com/q/366988">MCVE</a>. That's hard when your problem results from a very large dataset I know.</p>

CC BY-SA 4.0
2019-04-16T13:52:54.097Z
I upload a test data set. See that data link please. The data is a series of photon arrival times. We need to bin the data  and  then do lomb-scargle or fft. The problem is our low computing efficiency now. We need a faster method.
questionhang
CC BY-SA 4.0
2019-04-16T13:56:56.673Z
@questionhang thanks, that's great! I recommend two things, 1) wait for someone with specific experience to add another answer here, and 2) consider asking a separate question in StackOverflow as I discuss above. It's really great that you are so responsive to comments and questions. As you know, in SO they can come very quickly! :-)
uhoh
CC BY-SA 4.0
2019-04-16T13:23:14.327Z
2019-04-16T13:29:24.877Z
2019-04-16T13:29:24.877Z
uhoh
<p>Our <a href="https://clbin.com/C7Jcz" rel="nofollow noreferrer">data set</a> has <span class="math-container">$10^4$</span> data points, but has a long baseline and many gaps.</p>

<p>If we bin the data, there would be <span class="math-container">$10^8$</span> data points (<span class="math-container">$[t,\rm {value}]$</span>), but only about 1% are non-zero values.</p>

<p>How to improve the detection efficiency? </p>

<p>Is a multi-threading way possible (especially for Lomb-Scargle)?</p>

<p>For example, if I use LombScargle in <em>astropy.stats</em>,</p>

<pre><code>freq, power = LombScargle.autopower(minimum_frequency=0.5, maximum_frequency=1.5, normalization='standard')
</code></pre>

<p>there are two problems:</p>

<ol>
<li>It is slow and easy to get <code>memoryError</code>.</li>
<li>It can't utilize the computing power as much as possible.</li>
</ol>

<p>So my question is about the efficiency. For the data set above, what is the best method?</p>

CC BY-SA 4.0
2019-04-16T08:45:04.970Z
Question might be a better fit for [Computational Science SE](https://scicomp.stackexchange.com/).
StephenG - Help Ukraine
CC BY-SA 4.0
2019-04-16T11:07:31.053Z
[Understanding the Lomb-Scargle Periodogram](https://arxiv.org/abs/1703.09824) explains that technique nicely. Would you simply like to know if it can be (or has already been) written in a function particularly conducive to multi-threading? First glance of my (non-expert eyes) suggests that would be really straightforward, but rather than multithreading, I think the calculation is easy vectorized. I think you could ask a more specific, practical question about the technique here (how to set it up with your data) and a more computational question in CompSci as mentioned above.
uhoh
CC BY-SA 4.0
2019-04-16T13:02:13.457Z
@uhoh could you suggest some method which can improve the computing efficiency?
questionhang
CC BY-SA 4.0
2019-04-16T13:08:04.490Z
okay that would definitely be a good, and separate question, asked separately, in [Computational Science SE](https://scicomp.stackexchange.com/). However, before you ask it, please find a way to explain *clearly* how you are doing it now. You can't ask how to improve something if you don't show your current method and state its speed. In the mean time, I will take a quick look at what you've added here :-)
uhoh
CC BY-SA 4.0
2019-09-28T22:49:12.093Z
@StephenG yep that makes more sense.
Inertial Ignorance
<binary-star><pulsar><algorithm>
How to improve efficiency in periodicity detection?
questionhang
# How to improve efficiency in periodicity detection?

(Asked by: username_0 on 2019-04-16)

Our data set has $10^4$ data points, but has a long baseline and many gaps.
If we bin the data, there would be $10^8$ data points ($[t,\rm {value}]$), but only about 1% are non-zero values.
How to improve the detection efficiency? 
Is a multi-threading way possible (especially for Lomb-Scargle)?
For example, if I use LombScargle in astropy.stats,
freq, power = LombScargle.autopower(minimum_frequency=0.5, maximum_frequency=1.5, normalization='standard')

there are two problems:

It is slow and easy to get memoryError.
It can't utilize the computing power as much as possible.

So my question is about the efficiency. For the data set above, what is the best method?

**Tags:** binary-star, pulsar, algorithm
> **Comments:**
> **username_1**: Question might be a better fit for [Computational Science SE](https://scicomp.stackexchange.com/).  
> **username_2**: [Understanding the Lomb-Scargle Periodogram](https://arxiv.org/abs/1703.09824) explains that technique nicely. Would you simply like to know if it can be (or has already been) written in a function particularly conducive to multi-threading? First glance of my (non-expert eyes) suggests that would be really straightforward, but rather than multithreading, I think the calculation is easy vectorized. I think you could ask a more specific, practical question about the technique here (how to set it up with your data) and a more computational question in CompSci as mentioned above.  
> **username_0**: @uhoh could you suggest some method which can improve the computing efficiency?  
> **username_2**: okay that would definitely be a good, and separate question, asked separately, in [Computational Science SE](https://scicomp.stackexchange.com/). However, before you ask it, please find a way to explain *clearly* how you are doing it now. You can't ask how to improve something if you don't show your current method and state its speed. In the mean time, I will take a quick look at what you've added here :-)  
> **username_3**: @StephenG yep that makes more sense.  

---

## 1 answers

### Answer by username_2 on 2019-04-16. Score: 1

## AstroPy

Jake Vanderplas (Director of Open Software at the University of Washington eScience Institute) has written a Pythonic Preambulations post [Fast Lomb-Scargle Periodograms in Python](https://jakevdp.github.io/blog/2015/06/13/lomb-scargle-in-python/) which *by definition appears to be* a good place to start to learn how to do Lomb-Scargle Periodograms... Fast... in Python.

I have not read it yet in full, but I've found other posts of his to be extremely useful.

He starts out by explaining the implementation in [AstroPy](https://www.astropy.org/) which if you are going to be doing Astronomy, is probably a good thing to become familliar with.

## SciPy

Alternatively, there is [`scipy.signal.lombscargle`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.lombscargle.html) which you can try assuming you have a current version of SciPy installed but want to try something different than AstroPy or want to compare performance or results.

## StackOverflow

I found [over 40 posts containing 'Lomb-Scargle' in Stack Overflow](https://stackoverflow.com/search?q=Lomb-Scargle) which is a very good sign!

I notice that you've been active there and know how to ask well-received questions. Since your asking about speed and errors, StackOverflow is your go-to source! However, don't forget to be as clear as possible and quote all error messages and symptoms, and see if you can somehow add a short Minimum Complete and Verifiable Example or [MCVE](https://meta.stackoverflow.com/q/366988). That's hard when your problem results from a very large dataset I know.

> **Comments:**
> **username_0**: I upload a test data set. See that data link please. The data is a series of photon arrival times. We need to bin the data  and  then do lomb-scargle or fft. The problem is our low computing efficiency now. We need a faster method.  
> **username_2**: @questionhang thanks, that's great! I recommend two things, 1) wait for someone with specific experience to add another answer here, and 2) consider asking a separate question in StackOverflow as I discuss above. It's really great that you are so responsive to comments and questions. As you know, in SO they can come very quickly! :-)  

---

