<p>I'm not a spectroscopist, but, I know a fair bit about spacecraft data reduction.  CRISM is a spectrometer that's very sensitive, but it needs more light than it can get by just riding along with the spacecraft, so it slews in the reverse direction of motion (that's why CRISM images have that hourglass shape).  Despite that, it STILL has some light issues and so it can have noise problems (from the paper: &quot;Although radiance and raw data are noisy, robust mineral detections can be seen in all types of ratioed data.&quot;)</p>
<p>Among the noise issues are random spikes in the data.  The CRISM team designed, many years ago, code to try to remove those spikes, which can be both positive (bright) or negative (dark).  (from the paper: &quot;filtering is designed to remove sharp, narrow “spikes” in the radiance data set. Spikes can be positive or negative ...&quot;)</p>
<p>I'm guessing that, simply due to the way the detector was made, those spikes often occurred at 1.9 and 2.1 µm, and those wavelengths are particularly diagnostic for a few different minerals, including perchlorates (from the paper:  &quot;This artifact mimics real mineral absorptions in width and depth and occurs most often at 1.9 and 2.1 μm, thus interfering in the search for some mineral phases, including alunite, kieserite, serpentine, and perchlorate.&quot;).  This code that the team wrote to remove these artifacts/spikes is run automatically on their data, really before they look at it, and they look at a derived product from that data processing pipeline.</p>
<p>The problem appears to be that their code to remove the artifact wasn't perfect, and in some cases (&lt;0.05% of them), the spikes would remain, but since they thought their code was working, they thought those spikes - and hence the mineral detection - was real.</p>
<p>What this paper describes is the discovery that those spikes were NOT real in some cases.  And, they fixed the processing pipeline so it really does remove the false spikes.</p>
<p>Given that this paper was written in 2018, by people on the CRISM team, I think it's safe to say that anything published <em>since</em> then is reliable and run with the new code.</p>

CC BY-SA 4.0
2020-11-25T08:21:31.610Z
This is a pretty thorough analysis, I'll go back to the paper and dig in now, thanks!
uhoh
CC BY-SA 4.0
2020-11-25T05:24:41.717Z
2020-11-25T05:24:41.717Z
Stuart Robbins
<p>The Science News article <a href="https://www.sciencenews.org/article/mars-reconnaissance-orbiter-glitch-liquid-water" rel="nofollow noreferrer">An orbiter glitch may mean some signs of liquid water on Mars aren’t real</a> begins:</p>
<blockquote>
<p>Some signs of water on Mars may have just dried up.</p>
<p>Thanks to the way data from NASA’s Mars Reconnaissance Orbiter are handled, <strong>the spacecraft may be seeing <a href="https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018GL080077" rel="nofollow noreferrer">signs of hydrated salts that aren’t really there</a></strong>, planetary scientists report online November 9 in Geophysical Research Letters.</p>
<p>That lack of salts could mean that certain sites proposed as places where life could exist on Mars today, including purported streaks of liquid water on the walls of Martian craters, are probably dry and lifeless.</p>
<p>“People think these environments might be inhabitable by microbes,” says planetary scientist Ellen Leask of Caltech. But “there might not actually be any real evidence for it,” at least not from orbit.</p>
<p>Leask and her colleagues found the problem while searching for hydrated salts called perchlorates in maps of Mars taken by the orbiter’s <a href="http://crism.jhuapl.edu/" rel="nofollow noreferrer">Compact Reconnaissance Imaging Spectrometer for Mars</a>, or CRISM. Perchlorates can lower the freezing point of water by up to 80 degrees Celsius, which could be enough to melt ice in the frigid Martian climate.</p>
</blockquote>
<p>The report is <em>Open Access</em> in Geophysical Research Letters: <em>Challenges in the Search for Perchlorate and Other Hydrated Minerals with 2.1‐μm Absorptions on Mars</em>
Leask, E. K., Ehlmann, B. L., Dundar, M. Murchie, S. L. Seelos, F. P., <a href="https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018GL080077" rel="nofollow noreferrer">https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018GL080077</a></p>
<p>The data in question comes from Mars Reconnaissance Orbiter's CRISM (<a href="https://mars.nasa.gov/mro/mission/instruments/crismcompactreconnaissanceimagingspectrometerformars/" rel="nofollow noreferrer">Compact Reconnaissance Imaging Spectrometer for Mars</a>), a Vis-NIR push broom hyperspectral imager, where a 1D line image is cross-dispersed in wavelength and recorded on a 2D array as the spacecraft moves over Mars' surface.</p>
<p>If I understand correctly, during later image analysis and , some of the corrections to the data meant to compensate for &quot;spikes&quot; <em>in some observables</em> (<strong>despiking</strong>) occurring during large changes in brightness resulted in spectral features that suggested presence of hydrated minerals that didn't exist.</p>
<p><strong>Question:</strong> But I find the description of the step in the analysis responsible for the error a little bit complicated. Is it something simple like division by very small values leading to the spike in a derived quantity, followed by heavy-handed interpolation to smooth it over? What exactly does it mean when they talk about a few pixels, are there some physical locations on the detector that behave differently, or a spectral hot spot from artifacts in the grating? (e.g. Woods anlomalies)? “But for 0.05 percent of the pixels, it smooths out in a way that looks like perchlorate.”</p>
<p><strong>Potentially helpful resource:</strong> <a href="https://www.sciencedirect.com/science/article/pii/S0019103516301579" rel="nofollow noreferrer">Regularization of Mars Reconnaissance Orbiter CRISM along‐track oversampled hyperspectral imaging observations of Mars</a></p>
<p>Any help translating the &quot;processingese&quot; language to something easier to understand for a non-astronomer would be a great help!</p>
<p>More from the <a href="https://www.sciencenews.org/article/mars-reconnaissance-orbiter-glitch-liquid-water" rel="nofollow noreferrer">Science News article</a>:</p>
<blockquote>
<p>CRISM’s camera doesn’t work perfectly, though. It can be thrown off by boundaries between light and dark, like a shadowed region at the edge of a cliff. <strong>Some pixels in the orbiter’s camera take a fraction of a millisecond to realize the surface color changed, so they record an extra spot of light or dark where it shouldn’t be. Planetary scientists have software to correct for these “spikes” in the spectra and make the data more reliable and easier to read.</strong></p>
<p>But the correction sometimes introduces dips in the spectra at the same wavelengths as perchlorates, Leask and Ehlmann and their colleagues found. “We cleverly created a way to get rid of the spiky noise,” Ehlmann says. “But for 0.05 percent of the pixels, it smooths out in a way that looks like perchlorate.”</p>
<p>The researchers found the glitch while looking for small signs of the salts in CRISM images. Assuming that the orbiter would have already spotted large deposits of perchlorates if they existed, the team wrote an algorithm to find smaller traces that covered fewer than 10 pixels in a CRISM image. And the scientists started seeing perchlorates everywhere, including Jezero crater, which NASA announced was selected as the landing site for the Mars 2020 rover on November 19 (SN Online: 11/19/18).</p>
</blockquote>
<hr />
<p><a href="https://i.sstatic.net/F0ter.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/F0ter.jpg" alt="enter image description here" /></a></p>
<p>© 2018 American Geophysical Union. All rights reserved.</p>
<blockquote>
<p>Figure 1.Example of the (a) spatial and (b) spectral appearance of the artifact creating the spurious absorptions, shown in ratioed I/F and ratioed radiance. The threshold for an artifact is defined to be where ratioed radiance differs from ratioed I/F by more than 1% (c).  Laboratory spectra of minerals with a 2.1-μm absorption are shown (d) for comparison (from USGS spectral library (Clark et al. (2007)) and Hanley et al. (2015)).</p>
</blockquote>

CC BY-SA 4.0
2018-11-23T09:59:38.293Z
Mars perchlorate and water related: [What are the technologies that can identify, validate, and quantify water-rich locations on Mars?](https://space.stackexchange.com/q/27955/12102) as well as [Why doesn't the perchlorate on Mars' surface oxidize metallic meteorites?](https://space.stackexchange.com/a/24687/12102) and also [Sources for make-up breathable oxygen on Mars; of H20, CO2 and ClO4-, which is likely to be used first?](https://space.stackexchange.com/q/23694/12102)
uhoh
CC BY-SA 4.0
2020-11-24T02:35:49.320Z
I'm not a spectroscopist, but if no one else posts a technical answer in ~a day, I'll try to expand on this. Looking at the paper, it seems like there are known artifacts in the CRISM detector, the team has ways to deal with them. They didn't realize that the way to deal with them was occasionally (0.05% of the time) causing a spike in the spectrum which, if real (it's not real), could be interpreted as a perchlorate sign. They found the issue because they stressed the system which brought out the bug more. They have now fixed it. Paper was 2 years ago, so results since then should be correct.
Stuart Robbins
CC BY-SA 4.0
2020-11-24T03:13:11.077Z
@StuartRobbins the question has been around for two years so it seems nobody is jumping on it too quickly. It seems you have some insight into the specific issue here so I think that you are at least half way to an answer post already! Maybe include a link and quote a few sentences is all that's still needed.
uhoh
CC BY-SA 4.0
2020-11-24T18:28:25.927Z
Yeah, I seem to have issues these days observing when questions are actually posted, versus when they were last active.
Stuart Robbins
<mars><spectroscopy><artificial-satellite><water><spectrometry>
Help understanding the false MRO observations of hydrated minerals on Mars
uhoh
# Help understanding the false MRO observations of hydrated minerals on Mars

(Asked by: username_0 on 2018-11-23)

The Science News article [An orbiter glitch may mean some signs of liquid water on Mars aren’t real](https://www.sciencenews.org/article/mars-reconnaissance-orbiter-glitch-liquid-water) begins:

> Some signs of water on Mars may have just dried up.
>
> Thanks to the way data from NASA’s Mars Reconnaissance Orbiter are handled, **the spacecraft may be seeing [signs of hydrated salts that aren’t really there](https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018GL080077)**, planetary scientists report online November 9 in Geophysical Research Letters.
>
> That lack of salts could mean that certain sites proposed as places where life could exist on Mars today, including purported streaks of liquid water on the walls of Martian craters, are probably dry and lifeless.
>
> “People think these environments might be inhabitable by microbes,” says planetary scientist Ellen Leask of Caltech. But “there might not actually be any real evidence for it,” at least not from orbit.
>
> Leask and her colleagues found the problem while searching for hydrated salts called perchlorates in maps of Mars taken by the orbiter’s [Compact Reconnaissance Imaging Spectrometer for Mars](http://crism.jhuapl.edu/), or CRISM. Perchlorates can lower the freezing point of water by up to 80 degrees Celsius, which could be enough to melt ice in the frigid Martian climate.

The report is *Open Access* in Geophysical Research Letters: *Challenges in the Search for Perchlorate and Other Hydrated Minerals with 2.1‐μm Absorptions on Mars*
Leask, E. K., Ehlmann, B. L., Dundar, M. Murchie, S. L. Seelos, F. P., <https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2018GL080077>

The data in question comes from Mars Reconnaissance Orbiter's CRISM ([Compact Reconnaissance Imaging Spectrometer for Mars](https://mars.nasa.gov/mro/mission/instruments/crismcompactreconnaissanceimagingspectrometerformars/)), a Vis-NIR push broom hyperspectral imager, where a 1D line image is cross-dispersed in wavelength and recorded on a 2D array as the spacecraft moves over Mars' surface.

If I understand correctly, during later image analysis and , some of the corrections to the data meant to compensate for "spikes" *in some observables* (**despiking**) occurring during large changes in brightness resulted in spectral features that suggested presence of hydrated minerals that didn't exist.

**Question:** But I find the description of the step in the analysis responsible for the error a little bit complicated. Is it something simple like division by very small values leading to the spike in a derived quantity, followed by heavy-handed interpolation to smooth it over? What exactly does it mean when they talk about a few pixels, are there some physical locations on the detector that behave differently, or a spectral hot spot from artifacts in the grating? (e.g. Woods anlomalies)? “But for 0.05 percent of the pixels, it smooths out in a way that looks like perchlorate.”

**Potentially helpful resource:** [Regularization of Mars Reconnaissance Orbiter CRISM along‐track oversampled hyperspectral imaging observations of Mars](https://www.sciencedirect.com/science/article/pii/S0019103516301579)

Any help translating the "processingese" language to something easier to understand for a non-astronomer would be a great help!

More from the [Science News article](https://www.sciencenews.org/article/mars-reconnaissance-orbiter-glitch-liquid-water):

> CRISM’s camera doesn’t work perfectly, though. It can be thrown off by boundaries between light and dark, like a shadowed region at the edge of a cliff. **Some pixels in the orbiter’s camera take a fraction of a millisecond to realize the surface color changed, so they record an extra spot of light or dark where it shouldn’t be. Planetary scientists have software to correct for these “spikes” in the spectra and make the data more reliable and easier to read.**
>
> But the correction sometimes introduces dips in the spectra at the same wavelengths as perchlorates, Leask and Ehlmann and their colleagues found. “We cleverly created a way to get rid of the spiky noise,” Ehlmann says. “But for 0.05 percent of the pixels, it smooths out in a way that looks like perchlorate.”
>
> The researchers found the glitch while looking for small signs of the salts in CRISM images. Assuming that the orbiter would have already spotted large deposits of perchlorates if they existed, the team wrote an algorithm to find smaller traces that covered fewer than 10 pixels in a CRISM image. And the scientists started seeing perchlorates everywhere, including Jezero crater, which NASA announced was selected as the landing site for the Mars 2020 rover on November 19 (SN Online: 11/19/18).

---

[![enter image description here](https://i.sstatic.net/F0ter.jpg)](https://i.sstatic.net/F0ter.jpg)

© 2018 American Geophysical Union. All rights reserved.

> Figure 1.Example of the (a) spatial and (b) spectral appearance of the artifact creating the spurious absorptions, shown in ratioed I/F and ratioed radiance. The threshold for an artifact is defined to be where ratioed radiance differs from ratioed I/F by more than 1% (c). Laboratory spectra of minerals with a 2.1-μm absorption are shown (d) for comparison (from USGS spectral library (Clark et al. (2007)) and Hanley et al. (2015)).

**Tags:** mars, spectroscopy, artificial-satellite, water, spectrometry
> **Comments:**
> **username_0**: Mars perchlorate and water related: [What are the technologies that can identify, validate, and quantify water-rich locations on Mars?](https://space.stackexchange.com/q/27955/12102) as well as [Why doesn't the perchlorate on Mars' surface oxidize metallic meteorites?](https://space.stackexchange.com/a/24687/12102) and also [Sources for make-up breathable oxygen on Mars; of H20, CO2 and ClO4-, which is likely to be used first?](https://space.stackexchange.com/q/23694/12102)  
> **username_1**: I'm not a spectroscopist, but if no one else posts a technical answer in ~a day, I'll try to expand on this. Looking at the paper, it seems like there are known artifacts in the CRISM detector, the team has ways to deal with them. They didn't realize that the way to deal with them was occasionally (0.05% of the time) causing a spike in the spectrum which, if real (it's not real), could be interpreted as a perchlorate sign. They found the issue because they stressed the system which brought out the bug more. They have now fixed it. Paper was 2 years ago, so results since then should be correct.  
> **username_0**: @StuartRobbins the question has been around for two years so it seems nobody is jumping on it too quickly. It seems you have some insight into the specific issue here so I think that you are at least half way to an answer post already! Maybe include a link and quote a few sentences is all that's still needed.  
> **username_1**: Yeah, I seem to have issues these days observing when questions are actually posted, versus when they were last active.  

---

## 1 answers

### Answer by username_1 on 2020-11-25. Score: 3 | Accepted answer

I'm not a spectroscopist, but, I know a fair bit about spacecraft data reduction. CRISM is a spectrometer that's very sensitive, but it needs more light than it can get by just riding along with the spacecraft, so it slews in the reverse direction of motion (that's why CRISM images have that hourglass shape). Despite that, it STILL has some light issues and so it can have noise problems (from the paper: "Although radiance and raw data are noisy, robust mineral detections can be seen in all types of ratioed data.")

Among the noise issues are random spikes in the data. The CRISM team designed, many years ago, code to try to remove those spikes, which can be both positive (bright) or negative (dark). (from the paper: "filtering is designed to remove sharp, narrow “spikes” in the radiance data set. Spikes can be positive or negative ...")

I'm guessing that, simply due to the way the detector was made, those spikes often occurred at 1.9 and 2.1 µm, and those wavelengths are particularly diagnostic for a few different minerals, including perchlorates (from the paper: "This artifact mimics real mineral absorptions in width and depth and occurs most often at 1.9 and 2.1 μm, thus interfering in the search for some mineral phases, including alunite, kieserite, serpentine, and perchlorate."). This code that the team wrote to remove these artifacts/spikes is run automatically on their data, really before they look at it, and they look at a derived product from that data processing pipeline.

The problem appears to be that their code to remove the artifact wasn't perfect, and in some cases (<0.05% of them), the spikes would remain, but since they thought their code was working, they thought those spikes - and hence the mineral detection - was real.

What this paper describes is the discovery that those spikes were NOT real in some cases. And, they fixed the processing pipeline so it really does remove the false spikes.

Given that this paper was written in 2018, by people on the CRISM team, I think it's safe to say that anything published *since* then is reliable and run with the new code.

> **Comments:**
> **username_0**: This is a pretty thorough analysis, I'll go back to the paper and dig in now, thanks!  

---

