<p>Leaving aside the technicalities of different algorithms for speeding up the matches,[*] the basic idea is that you are trying to find e.g. sources in Catalog 2 that are plausible matches to sources in Catalog 1 (e.g., located within <span class="math-container">$d$</span> arc seconds). The simplest (albeit computationally inefficient) method is to step through every source in Catalog 1, checking (all of) Catalog 2 for possible matches. When you get to the end of Catalog 1, <em>you're done</em>. &quot;Looping around&quot; would mean repeating all of the same calculations and tests you just did, which would be a complete waste of time.</p>
<p>[*] A simplistic example: pre-sort and index Catalog 2 by one of the coordinates (e.g., Right Ascension). Then, for a given source in Catalog 1, you only need to test sources from Catalog 2 that have RA within <span class="math-container">$\pm d$</span> arc seconds of the Catalog 1 source. (Most of these sources will be too far away in the other coordinate to end up being matches, but you <em>know</em> all the sources with <span class="math-container">$\Delta$</span>RA <span class="math-container">$&gt; d$</span> can't possibly be matches, so there's no need to waste time testing them.)</p>

CC BY-SA 4.0
2022-09-26T13:22:16.247Z
It's going to be tougher than that.  Since the catalogs will have been prepared on different dates, possibly separated by decades, the sources will have to be adjusted for parallax and proper motion first.  And the catalogs will have different error margins for each of those.  Variations in luminosity, and the fact they're likely making observations at different wavelengths, make it even harder.
Greg Miller
CC BY-SA 4.0
2022-09-26T15:03:49.683Z
@GregMiller Sure, though in some cases (e.g., galaxy or AGN catalogs) things like parallax and proper motion do not apply. And matching catalogs made using different wavelengths is one of the main reasons for doing catalog matching! (E.g., "How many of the X-ray sources in Catalog 1 can be matched with the radio sources in Catalog 2?") I don't see how those last two factors make anything "harder", though.
Peter Erwin
CC BY-SA 4.0
2022-09-24T13:23:01.527Z
2022-09-26T07:48:36.563Z
2022-09-26T07:48:36.563Z
Peter Erwin
<p>Let's say I'm using a service like TOPCAT or Vizier's cross-matching service and I'm trying to match the RA and dec of the sources in Catalogue 1, with 100,000 sources, to those in Catalogue 2, with a million sources.</p>
<p>How does the algorithm deal with the difference in size when it gets to the end of the shortest catalogue? Does it loop around? Or am I making an incorrect assumption about how cross-matching works?</p>
<p>This question comes from my attempts at making my own algorithm using <code>astropy</code>'s <code>SkyCoord</code> function; it won't let me broadcast together two differently-sized arrays.</p>

CC BY-SA 4.0
2022-09-20T15:03:10.203Z
This sounds like a basic database command question.  Depending on what you really want to do, the answer could be as simple as a double loop, or a  matrix operation on an array of size  1E5 by 1E6 .
Carl Witthoft
<data-analysis><star-catalogues>
How do cross-matching algorithms deal with unequal-sized catalogues?
Jim421616
# How do cross-matching algorithms deal with unequal-sized catalogues?

(Asked by: username_0 on 2022-09-19)

Let's say I'm using a service like TOPCAT or Vizier's cross-matching service and I'm trying to match the RA and dec of the sources in Catalogue 1, with 100,000 sources, to those in Catalogue 2, with a million sources.

How does the algorithm deal with the difference in size when it gets to the end of the shortest catalogue? Does it loop around? Or am I making an incorrect assumption about how cross-matching works?

This question comes from my attempts at making my own algorithm using `astropy`'s `SkyCoord` function; it won't let me broadcast together two differently-sized arrays.

**Tags:** data-analysis, star-catalogues
> **Comments:**
> **username_1**: This sounds like a basic database command question.  Depending on what you really want to do, the answer could be as simple as a double loop, or a  matrix operation on an array of size  1E5 by 1E6 .  

---

## 1 answers

### Answer by username_2 on 2022-09-24. Score: 1

Leaving aside the technicalities of different algorithms for speeding up the matches,[\*] the basic idea is that you are trying to find e.g. sources in Catalog 2 that are plausible matches to sources in Catalog 1 (e.g., located within $d$ arc seconds). The simplest (albeit computationally inefficient) method is to step through every source in Catalog 1, checking (all of) Catalog 2 for possible matches. When you get to the end of Catalog 1, *you're done*. "Looping around" would mean repeating all of the same calculations and tests you just did, which would be a complete waste of time.

[\*] A simplistic example: pre-sort and index Catalog 2 by one of the coordinates (e.g., Right Ascension). Then, for a given source in Catalog 1, you only need to test sources from Catalog 2 that have RA within $\pm d$ arc seconds of the Catalog 1 source. (Most of these sources will be too far away in the other coordinate to end up being matches, but you *know* all the sources with $\Delta$RA $> d$ can't possibly be matches, so there's no need to waste time testing them.)

> **Comments:**
> **username_3**: It's going to be tougher than that.  Since the catalogs will have been prepared on different dates, possibly separated by decades, the sources will have to be adjusted for parallax and proper motion first.  And the catalogs will have different error margins for each of those.  Variations in luminosity, and the fact they're likely making observations at different wavelengths, make it even harder.  
> **username_2**: @GregMiller Sure, though in some cases (e.g., galaxy or AGN catalogs) things like parallax and proper motion do not apply. And matching catalogs made using different wavelengths is one of the main reasons for doing catalog matching! (E.g., "How many of the X-ray sources in Catalog 1 can be matched with the radio sources in Catalog 2?") I don't see how those last two factors make anything "harder", though.  

---

