You’re absolutely right, the model does work correctly if model_target leaves aside the last pooling layer!
Apparently, it’s no mistake in the code either, as it’s documented in the paper (I_s is the search Image, and I_t is the target Image):
visualsearchzeroshot-layers980×270 103 KB
Here’s Supplementary Figure 13, where it shows this configuration yielded the best results:

Maybe the mistake is in resizing the target image to 28x28, instead of 224x224? Even though it’s clearly in the script, I couldn’t find anywhere in the paper where it stated that the target was resized to 28x28.
Anyway, I can’t thank you enough for all the help you’ve given me. I marked your latest post as the solution.