<p><strong>Folding introduction and example:</strong></p>
<p><a href="https://imagine.gsfc.nasa.gov/science/toolbox/timing2.html" rel="nofollow noreferrer">Folding</a> is a signal processing technique typically used to extract and characterize low SNR (signal to noise ratio) electromagnetic pulses from time domain series data.  The techniques were developed to support radar processing long before the first pulsar was discovered, but I don’t know if astronomers developed it independently.</p>
<p>Let’s simulate some pulses based on <a href="https://en.wikipedia.org/wiki/PSR_B1919%2B21" rel="nofollow noreferrer">the first pulsar found</a>, the PSR B1919+21.  This pulsar has a period of about 1.34 seconds, and a PW of 0.04 seconds. Let’s suppose our detector is not too great and we sample at 0.01 seconds from the absolute value of a uniform distribution with a mean of 400 photons, standard deviation of 200 photons, so a single simulated noiseless pulse in the time domain looks like this:</p>
<p><a href="https://i.sstatic.net/UWRCo.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UWRCo.jpg" alt="enter image description here" /></a></p>
<p>The x-axis is samples at 0.01 seconds. The y-axis is photon count. Then let’s produce some noise using the absolute value of a mean of 0 with 1000 standard deviation.</p>
<p><a href="https://i.sstatic.net/8BiX2.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8BiX2.jpg" alt="enter image description here" /></a></p>
<p>Then let's add the noise and the pulse together.  This is what our simulated detector sees for a faint pulsar:</p>
<p><a href="https://i.sstatic.net/WzAJp.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WzAJp.jpg" alt="enter image description here" /></a></p>
<p>The pulse barely clears the noise floor, and we wouldn't be able to determine that it's 0.04 seconds wide just from this data.  Here is the pulse and detection plotted together:</p>
<p><a href="https://i.sstatic.net/PwHLs.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/PwHLs.jpg" alt="enter image description here" /></a></p>
<p>We can capture 50 pulses by observing the signal for over a little more than a minute duration:</p>
<p><a href="https://i.sstatic.net/XCQrJ.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XCQrJ.jpg" alt="enter image description here" /></a></p>
<p>If we divide all the observational data into 50 separate windows (with a good guess for period) we can add all the windows together (element wise).  In the new window, the pulse is clear, since the noise tends to cancel out, and the signal compounds.</p>
<p><a href="https://i.sstatic.net/ZNmra.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZNmra.jpg" alt="enter image description here" /></a></p>
<p>If the pulse were more complicated, we could start to see subtle structure by this folding technique for longer observation times even in the presence of a lot of noise.</p>
<p><strong>Answer:</strong></p>
<p>Now that we've introduced folding, lets get to the actual answer.  There is no good reason to apply data folding to transit times of Mars, since the data set has no noise.  I believe that the differencing techniques applied by '@SE – stop firing the good guys' is the best technique.</p>
<p>However, I like pounding in nails with a screw driver and <em>we can pickle that</em>!  Let's cut out the leading zeros and divide the days by 10 and round so we don't get 'off by one' alignments when we fold.  Then let's create a square wave for each transit:</p>
<p><a href="https://i.sstatic.net/UXMIa.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/UXMIa.jpg" alt="enter image description here" /></a></p>
<p>We can chop it into 4 equally sized windows:</p>
<p><a href="https://i.sstatic.net/080Mg.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/080Mg.jpg" alt="enter image description here" /></a></p>
<p>And add all the windows to get:</p>
<p><a href="https://i.sstatic.net/Xr0no.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Xr0no.jpg" alt="enter image description here" /></a></p>
<p>Since our window was 103730 days, it reveals a periodicity of about 284 years.  To quote the other answer, &quot;there's not enough data to with any certainty detect any higher order periodicities that that.&quot;</p>
<p><strong>Notes:</strong></p>
<ol>
<li><p>Using folding on transit time data the way I have, is not an appropriate use of the technique, I think.  It can work, but it is overkill.</p>
</li>
<li><p>The other answer of a periodicity of 26+79+101=206 years, I respectfully believe is an error.  They should have added the first 4 gaps and subtracted 1 to get 26+79+101+79-1=284 years, which is my answer.  Their technique is still superior to the one I use here.</p>
</li>
<li><p>Picking the right window size for folding is key to get the technique to work. If you don't know the window size, you can try many window sizes and pick the one with the maximum response. Some folks also use <a href="https://en.wikipedia.org/wiki/Temporal_Raster_Plot" rel="nofollow noreferrer">raster</a> plots.</p>
</li>
<li><p>This technique is important for detecting and determining <a href="https://en.wikipedia.org/wiki/Glitch_(astronomy)" rel="nofollow noreferrer">glitches</a> for faint pulsars.</p>
</li>
<li><p>One could certainly use this technique for other periodic phenomena in the presence of noise, like finding planet transits across stars in other planetary systems.  This would be an especially interesting application since, instead of a spike, we would expect a 'dip' in the folded data where the detected energy of the star was lower when the planet passed in front.</p>
</li>
</ol>

CC BY-SA 4.0
2020-11-26T21:53:03.907Z
I am also happy to provide my code on request.
Connor Garcia
CC BY-SA 4.0
2020-11-26T23:13:30.810Z
Thank you for your thorough and systematic answer! "since the data set has no noise" There may not be noise in amplitude (signal is either 0 or 1) but there is certainly noise in time. Since Mars' orbit is substantially elliptical and there is no simple relationship between Earth and Mars orbital periods (e.g. it's not 3:2 for example) the transits will come "early" or "late" wrt circular orbits of the same period. I'm therefore a little surprised at the narrowness of your peaks, it's not what I expected! I haven't had my morning coffee yet so will back later today once all cylinders firing...
uhoh
CC BY-SA 4.0
2020-11-26T23:15:15.077Z
it's always possible that my input data doesn't reflect reality, or that the bin size here is sufficiently large that the effect is too small to be resolved.
uhoh
CC BY-SA 4.0
2020-11-26T23:35:01.373Z
@uhoh I don't think it would be wrong to use the normal distributions generated in your original question to 'fold' this data. There is more than one way to skin a cat! I kluged your given data set by dividing by 10 and rounding so the small offsets disappeared. And yes, I meant noise in amplitude rather than time.
Connor Garcia
CC BY-SA 4.0
2023-03-22T22:22:08.177Z
related in DSP SE: [Finding periodicity in discrete events II - missions to Mars](https://dsp.stackexchange.com/q/87178/25659)
uhoh
CC BY-SA 4.0
2020-11-26T17:55:07.100Z
2020-11-26T18:05:25.063Z
2020-11-26T18:05:25.063Z
Connor Garcia
<p>While this is not exactly how folding works, one-dimensional data encourages trying some simple statistical tallying first.</p>
<hr>
What very easily reveals those periodicities is just tallying up the deltas, with an appropriate fuzziness for what periodicities are relevant:
<ol>
<li>Deltas</li>
</ol>
<p><code>[0, 9308, 28857, 36709, 28857, 9308, 28858, 36708, 28858, 9308, 28857, 36708, 9309, 19549, 9309, 28857, 36707, 9310]</code></p>
<p>Already here, some patterns should be apparent. But we have some extra information that could be used to filter out the short period noise.</p>
<ol start="2">
<li>Rounding at an appropriate sampling frequency</li>
</ol>
<p>Earth transits aren't going to happen every day. If there was an Earth transit yesterday, it can't happen again today, or next day, or even the next month. In fact, we know they can only happen once for every synodic period of Mars-Earth (780 days).</p>
<p>To not miss out in any &quot;legitimate&quot; part of the signal, we should sample at <a href="https://en.wikipedia.org/wiki/Nyquist_rate" rel="nofollow noreferrer">at least</a> twice of that rate. &quot;one year&quot; happens to be pretty close to half of 780 days.</p>
<p>The periodicity is then readily visible:</p>
<p><code>[26, 79, 101, 79, 26, 79, 101, 79, 26, 79, 101, 26, 54, 26, 79, 101, 26]</code></p>
<p>Which also reveals an even higher extra periodicity of about 206 years.</p>
<p>With a strong 206 year periodicity found in a set of events only covering a little over 400 years, there's not enough data to with any certainty detect any higher order periodicities that that.</p>

CC BY-SA 4.0
2020-10-03T23:50:28.090Z
Is how astronomers implement *folding?*
uhoh
CC BY-SA 4.0
2020-10-04T00:02:36.920Z
@uhoh I will delete this is if the example data wasn't the point of your post. Sometimes simple methods are useful.
SE - stop firing the good guys
CC BY-SA 4.0
2020-10-04T00:09:10.813Z
It's up to you but I don't think it's necessary. My comment asks for clarification. If this is how folding works you can mention that this is "some insight into" or "the basic idea". If not, you can say "while this is not exactly how folding works, it may get you started" or anything else that offers guidance how this applies to folding. Since someone has added a bounty to the question I feel additional responsibility for encouraging clarity on how well an answer addresses the question as asked.
uhoh
CC BY-SA 4.0
2020-10-03T15:55:00.303Z
2020-10-04T06:08:58.310Z
2020-10-04T06:08:58.310Z
SE - stop firing the good guys
<p><a href="https://astronomy.stackexchange.com/a/26178/7982">@JamesK's answer</a> to <em>When will the next transit of Earth be visible from Mars? Was the last one really on May 11, 1984?</em> lists dates of 18 transits which I've converted to days since 1500-Jan-01 as</p>
<pre><code>[35011, 44319, 73176, 109885, 138742, 148050, 176908, 213616, 242474, 251782, 280639, 317347, 326656, 346205, 355514, 384371, 421078, 430388]
</code></pre>
<p>Answers to <a href="https://astronomy.stackexchange.com/q/32989/7982">Why do transits of Earth across the Sun seen from Mars follow a pattern of occurring after 26, 79 then 100 years?</a> explain why this happens and why if they happen at all, the happen in May and November.</p>
<p>As an exercise I'd like to use <em>folding</em> to see if I can detect those or similar periodicities in addition to the 6 month periodicity.</p>
<p>I could</p>
<ol>
<li>for each period of N days recalculate those modulo N</li>
<li>histogram the results</li>
</ol>
<p>but since there are so few event, they rarely occur when their time comes, and so many days between events, there won't be obvious peaks in the resulting histogram.</p>
<p><strong>Question:</strong> Is there a technique for folding sparse event data in order to detect an underlying periodicity?</p>
<hr />
<p>Events convoluted with σ=1000 day gaussians purely for illustration purposes. I thought about applying some Fourier transform to this but I don't think that that makes sense, and would be cheating rather than folding.</p>
<p><a href="https://i.sstatic.net/kUwnq.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kUwnq.png" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2023-03-22T22:10:50.693Z
working on an answer now [Finding periodicity in discrete events II - missions to Mars](https://dsp.stackexchange.com/q/87178/25659)
uhoh
<observational-astronomy><exoplanet><data-analysis><python>
Technique for folding sparse event data in order to detect an underlying periodicity?
uhoh
# Technique for folding sparse event data in order to detect an underlying periodicity?

(Asked by: username_0 on 2020-10-01)

@JamesK's answer to When will the next transit of Earth be visible from Mars? Was the last one really on May 11, 1984? lists dates of 18 transits which I've converted to days since 1500-Jan-01 as
[35011, 44319, 73176, 109885, 138742, 148050, 176908, 213616, 242474, 251782, 280639, 317347, 326656, 346205, 355514, 384371, 421078, 430388]

Answers to Why do transits of Earth across the Sun seen from Mars follow a pattern of occurring after 26, 79 then 100 years? explain why this happens and why if they happen at all, the happen in May and November.
As an exercise I'd like to use folding to see if I can detect those or similar periodicities in addition to the 6 month periodicity.
I could

for each period of N days recalculate those modulo N
histogram the results

but since there are so few event, they rarely occur when their time comes, and so many days between events, there won't be obvious peaks in the resulting histogram.
Question: Is there a technique for folding sparse event data in order to detect an underlying periodicity?

Events convoluted with σ=1000 day gaussians purely for illustration purposes. I thought about applying some Fourier transform to this but I don't think that that makes sense, and would be cheating rather than folding.

**Tags:** observational-astronomy, exoplanet, data-analysis, python
> **Comments:**
> **username_0**: working on an answer now [Finding periodicity in discrete events II - missions to Mars](https://dsp.stackexchange.com/q/87178/25659)  

---

## 2 answers

### Answer by username_1 on 2020-11-26. Score: 2 | Accepted answer

**Folding introduction and example:**

[Folding](https://imagine.gsfc.nasa.gov/science/toolbox/timing2.html) is a signal processing technique typically used to extract and characterize low SNR (signal to noise ratio) electromagnetic pulses from time domain series data. The techniques were developed to support radar processing long before the first pulsar was discovered, but I don’t know if astronomers developed it independently.

Let’s simulate some pulses based on [the first pulsar found](https://en.wikipedia.org/wiki/PSR_B1919%2B21), the PSR B1919+21. This pulsar has a period of about 1.34 seconds, and a PW of 0.04 seconds. Let’s suppose our detector is not too great and we sample at 0.01 seconds from the absolute value of a uniform distribution with a mean of 400 photons, standard deviation of 200 photons, so a single simulated noiseless pulse in the time domain looks like this:

[![enter image description here](https://i.sstatic.net/UWRCo.jpg)](https://i.sstatic.net/UWRCo.jpg)

The x-axis is samples at 0.01 seconds. The y-axis is photon count. Then let’s produce some noise using the absolute value of a mean of 0 with 1000 standard deviation.

[![enter image description here](https://i.sstatic.net/8BiX2.jpg)](https://i.sstatic.net/8BiX2.jpg)

Then let's add the noise and the pulse together. This is what our simulated detector sees for a faint pulsar:

[![enter image description here](https://i.sstatic.net/WzAJp.jpg)](https://i.sstatic.net/WzAJp.jpg)

The pulse barely clears the noise floor, and we wouldn't be able to determine that it's 0.04 seconds wide just from this data. Here is the pulse and detection plotted together:

[![enter image description here](https://i.sstatic.net/PwHLs.jpg)](https://i.sstatic.net/PwHLs.jpg)

We can capture 50 pulses by observing the signal for over a little more than a minute duration:

[![enter image description here](https://i.sstatic.net/XCQrJ.jpg)](https://i.sstatic.net/XCQrJ.jpg)

If we divide all the observational data into 50 separate windows (with a good guess for period) we can add all the windows together (element wise). In the new window, the pulse is clear, since the noise tends to cancel out, and the signal compounds.

[![enter image description here](https://i.sstatic.net/ZNmra.jpg)](https://i.sstatic.net/ZNmra.jpg)

If the pulse were more complicated, we could start to see subtle structure by this folding technique for longer observation times even in the presence of a lot of noise.

**Answer:**

Now that we've introduced folding, lets get to the actual answer. There is no good reason to apply data folding to transit times of Mars, since the data set has no noise. I believe that the differencing techniques applied by '@SE – stop firing the good guys' is the best technique.

However, I like pounding in nails with a screw driver and *we can pickle that*! Let's cut out the leading zeros and divide the days by 10 and round so we don't get 'off by one' alignments when we fold. Then let's create a square wave for each transit:

[![enter image description here](https://i.sstatic.net/UXMIa.jpg)](https://i.sstatic.net/UXMIa.jpg)

We can chop it into 4 equally sized windows:

[![enter image description here](https://i.sstatic.net/080Mg.jpg)](https://i.sstatic.net/080Mg.jpg)

And add all the windows to get:

[![enter image description here](https://i.sstatic.net/Xr0no.jpg)](https://i.sstatic.net/Xr0no.jpg)

Since our window was 103730 days, it reveals a periodicity of about 284 years. To quote the other answer, "there's not enough data to with any certainty detect any higher order periodicities that that."

**Notes:**

1. Using folding on transit time data the way I have, is not an appropriate use of the technique, I think. It can work, but it is overkill.
2. The other answer of a periodicity of 26+79+101=206 years, I respectfully believe is an error. They should have added the first 4 gaps and subtracted 1 to get 26+79+101+79-1=284 years, which is my answer. Their technique is still superior to the one I use here.
3. Picking the right window size for folding is key to get the technique to work. If you don't know the window size, you can try many window sizes and pick the one with the maximum response. Some folks also use [raster](https://en.wikipedia.org/wiki/Temporal_Raster_Plot) plots.
4. This technique is important for detecting and determining [glitches](https://en.wikipedia.org/wiki/Glitch_(astronomy)) for faint pulsars.
5. One could certainly use this technique for other periodic phenomena in the presence of noise, like finding planet transits across stars in other planetary systems. This would be an especially interesting application since, instead of a spike, we would expect a 'dip' in the folded data where the detected energy of the star was lower when the planet passed in front.

> **Comments:**
> **username_1**: I am also happy to provide my code on request.  
> **username_0**: Thank you for your thorough and systematic answer! "since the data set has no noise" There may not be noise in amplitude (signal is either 0 or 1) but there is certainly noise in time. Since Mars' orbit is substantially elliptical and there is no simple relationship between Earth and Mars orbital periods (e.g. it's not 3:2 for example) the transits will come "early" or "late" wrt circular orbits of the same period. I'm therefore a little surprised at the narrowness of your peaks, it's not what I expected! I haven't had my morning coffee yet so will back later today once all cylinders firing...  
> **username_0**: it's always possible that my input data doesn't reflect reality, or that the bin size here is sufficiently large that the effect is too small to be resolved.  
> **username_1**: @uhoh I don't think it would be wrong to use the normal distributions generated in your original question to 'fold' this data. There is more than one way to skin a cat! I kluged your given data set by dividing by 10 and rounding so the small offsets disappeared. And yes, I meant noise in amplitude rather than time.  
> **username_0**: related in DSP SE: [Finding periodicity in discrete events II - missions to Mars](https://dsp.stackexchange.com/q/87178/25659)  

---

### Answer by username_2 on 2020-10-03. Score: 3

While this is not exactly how folding works, one-dimensional data encourages trying some simple statistical tallying first.

---

What very easily reveals those periodicities is just tallying up the deltas, with an appropriate fuzziness for what periodicities are relevant:

1. Deltas

`[0, 9308, 28857, 36709, 28857, 9308, 28858, 36708, 28858, 9308, 28857, 36708, 9309, 19549, 9309, 28857, 36707, 9310]`

Already here, some patterns should be apparent. But we have some extra information that could be used to filter out the short period noise.

2. Rounding at an appropriate sampling frequency

Earth transits aren't going to happen every day. If there was an Earth transit yesterday, it can't happen again today, or next day, or even the next month. In fact, we know they can only happen once for every synodic period of Mars-Earth (780 days).

To not miss out in any "legitimate" part of the signal, we should sample at [at least](https://en.wikipedia.org/wiki/Nyquist_rate) twice of that rate. "one year" happens to be pretty close to half of 780 days.

The periodicity is then readily visible:

`[26, 79, 101, 79, 26, 79, 101, 79, 26, 79, 101, 26, 54, 26, 79, 101, 26]`

Which also reveals an even higher extra periodicity of about 206 years.

With a strong 206 year periodicity found in a set of events only covering a little over 400 years, there's not enough data to with any certainty detect any higher order periodicities that that.

> **Comments:**
> **username_0**: Is how astronomers implement *folding?*  
> **username_2**: @uhoh I will delete this is if the example data wasn't the point of your post. Sometimes simple methods are useful.  
> **username_0**: It's up to you but I don't think it's necessary. My comment asks for clarification. If this is how folding works you can mention that this is "some insight into" or "the basic idea". If not, you can say "while this is not exactly how folding works, it may get you started" or anything else that offers guidance how this applies to folding. Since someone has added a bounty to the question I feel additional responsibility for encouraging clarity on how well an answer addresses the question as asked.  

---

