The tap_layer files you pushed 40 minutes ago are a real result, but "layer 47 (shipped)" is not the shipped head. The bytes say so.
I read the zip directories of all three checkpoints over range requests rather than guessing from the table.
browser_head_118k.pt data/0 5,242,880 fp32 = 5120 x 1024
data/2 1,048,576 fp32 = 1024 x 1024
data/4 5,120 mu_img
data/5 1,024 mu_txt
tap_layer/head_full_L47.pt data/0 5,505,024 fp32 = 5376 x 1024
data/2 5,505,024 fp32 = 5376 x 1024
data/4 5,376 mu_img
data/5 5,376 mu_txt
tap_layer/head_full_L20.pt identical shapes to L47
The shipped head is asymmetric, 5120 in on the image side and 1024 in on the text side, which is exactly the architecture block: Qwen3.8-27B against Qwen3-0.6B. Both tap_layer heads are symmetric at 5376 on both sides. 5376 is neither backbone. A tap layer changes which block you read, never the hidden size, so no choice of tap turns one of these into the other.
Three more things point the same way, and none of them needs the bytes.
Your architecture block says the image side taps layer 52, not 47. Your text side taps 28, and you say 28 is the last block of Qwen3-0.6B, so 47 cannot be the text side either.
tap_layer_full.json says n_train 113,287. "How it was trained" says the shipped head used all 118,287 with all 5 captions. Different fit, 5,000 pairs smaller.
And the numbers do not line up with your own 5,000-pool rows. L47 scores r@1 0.2830 on a 5,000-pair holdout. The shipped head's 5,000-image val pool rows are 0.3985 text-to-image and 0.5348 image-to-text. 0.2830 is neither.
The comparison itself I have no argument with. Same 5376 backbone, same recipe, same 113,287/5,000 split, only the tap moves, and 0.3244 against 0.2830 is +0.0414, 14.6% relative, reproducing your +0.041 and 15% exactly. As an ablation it is clean.
What it does not support is the sentence after it. "The head shipped in this repo taps layer 47, so the shipped depth is not the best depth we have measured" reads as a finding about browser_head_118k.pt, and the file it rests on has different shapes, a different backbone, a different training-set size, and does not reproduce any published number for that head.
Two hundred lines above it your card already says the rule in bold: always read a retrieval number with its pool size, and with the head it was measured on. This is the one table that does not.
The fix is a label, not a retraction. Is L47 the shipped tap on the 5376 model this ablation actually ran on, with the browser head untouched at 52 and 28, or did the shipped tap move and the architecture block go stale?
Separately, the floor, since your last round asked for it.
Twenty derangements recomputed from your own file, and the derived block reproduces line for line: means 65,087.6 and 61,717.8, population SD 363.6 and 593.4, separation 3,369.8 over a pooled SE of 155.6, 21.65 SD. 59,071 is below all twenty. 64,481 is inside at -1.67.
One thing your file shows that your comment understates. The two sets of twenty do not overlap at all. Lowest as-is median 64,408, highest anchored median 62,719. There is no draw of either arm that could be mistaken for the other, which is a stronger statement than an SD distance, because it needs no distributional assumption.
The uniform draw is not a rival floor
For a fixed query your head induces a total order on 123,287 rows. Pick a row uniformly and its rank is uniform on 1..N. That is true for any head, any runtime, any quantization. So the median over 5,001 such draws sits at (N+1)/2 = 61,644 with SE N/(2*sqrt(5001)) = 871.7, which is your uniform_draw_se of 872 to within rounding.
That number is an identity in N and n. It carries zero bits about the head. So it was never a floor competing with yours, and the near-agreement at 74 cannot be a coincidence of the pool, because there is no pool in it.
Which makes your per-arm choice right, for a sharper reason than you gave
You justified it as the floor being a property of the arm. The stronger justification is exchangeability. The gold is a val2017 image, so the distractor has to be drawn from the same population as the gold, and yours is.
Your own training section says the gallery is 118,287 COCO train2017 images. 123,287 minus 118,287 is 5,000, which is val2017 exactly. So 95.9% of gallery rows are images the head was fitted on, and every deranged distractor is one of the 1,000 held-out rows at 122,287..123,286.
Analytic chance is not a weaker baseline than your floor. It is a baseline over a different population, one that is 96% training data.
So the +3,444 is a measurement, not a coincidence
Median rank of a held-out distractor, as-is arm: 65,087.6. Median rank of a uniformly drawn gallery row, any arm: 61,644. The gap is 3,443.6, against an SE of the mean of 363.6/sqrt(20) = 81.3. That is 42.4 SE.
The as-is head ranks images it has never seen 3,444 rows worse than it ranks the gallery average, and the gallery average is mostly images it trained on. That is a train/val generalization gap denominated in gallery rank. It has been sitting in the file the whole time as the thing you were dismissing.
The anchored arm's version of the same quantity is 73.8, which is 0.56 SE. Zero.
So the anchor does not only lift recall from 0.0000 to 0.0350. It closes the gap between seen and unseen images. Those are different claims and the second one is not on your card.
The test that would kill this
Rerun the twenty derangements with the distractor drawn from train2017 rows only, same 5,001 val queries, same gallery. Holds the query fixed and changes only the distractor population.
If the reading is right, the as-is floor falls to about 61,644 and the anchored floor barely moves, the two floors converge, and the 21.65 SD separation collapses to nothing. If both floors stay where they are, the gap reading is wrong and what you have found is something about the last block of the index instead.
One correction to my own prediction before you spend the compute on it. Ranks are a permutation of 1..123,287, so the average rank over the whole gallery is exactly 61,644 and displaced mass has to be paid for by somebody. If all 5,000 non-train rows carry the +3,444 that your 1,000 eval rows show, the 118,287 train rows have to sit at 61,644 - 3,444 x 5,000/118,287 = 61,498.
So a train-only floor should land near 61,498, not 61,644. That is 146 below, and 146 against an SE of the mean of 81 is 1.8 SE, so twenty derangements would see it as a lean rather than a result. The identity is exact for means and yours are medians, so treat 146 as the scale, not the value. It is still the number to aim at, because if a train-only floor comes back at 61,644 on the nose the conservation is not being paid and something in the pool construction is doing the work instead.
Either answer is worth more than the floor was.
And the exact number is cheaper than the estimate
You need the full score vector per query to compute a rank at all. Once you have it, the rank of all 999 non-gold pool images is 999 lookups into an array you have already built. The median over the whole 5,001 x 999 matrix is the exact quantity your twenty derangements estimate, and it has no error bar. Twenty draws pin it to plus or minus 81. One pass pins it.
That also tells you whether the 3,444 is a shift of the whole within-pool rank distribution or a tail of held-out images that rank catastrophically, which the median alone cannot separate and which would mean very different things about the head.
Does the fp16 reference arm show the same gap? It sits at median 36 on the gold, so it has the recall the Q4 arm lost, but nothing in what you have published says whether it also ranks held-out images below the gallery average. If fp16 shows +3,444 too, the gap is the head's and quantization is innocent of it. If only Q4 shows it, then Q4 is not just losing precision, it is losing generalization to images the head never saw, and that is the thing to say on the card.
