Fix refresh on the overall tab

#122
by tomaarsen HF staff - opened
Massive Text Embedding Benchmark org
edited Jun 5

Hello!

Pull Request overview

  • Fix refresh on the overall tabs

Details

This is a rather common bug:

functions = []
for i in range(5):
    functions.append(lambda: i)
print([func() for func in functions])

People tend to expect this to return

[0, 1, 2, 3, 4]

But it actually returns

[4, 4, 4, 4, 4]

The reason is that the value of iis not checked until the function is called, after which the iterator is already much further. We did the same here, meaning that every single Refresh button on an "Overall" tab resulted in the "other-sts" tab data being collected. This is not what the current tab needs, so the columns are all messed up, the averages get computed weirdly, and the table gets sorted incorrectly.

This PR fixes it by calling a function with i (or board_config["tasks"] in our case), much like:

def helper_func(i):
    return lambda: i

functions = []
for i in range(5):
    functions.append(helper_func(i))
print([func() for func in functions])

which returns

[0, 1, 2, 3, 4]
  • Tom Aarsen
tomaarsen changed pull request status to open
Massive Text Embedding Benchmark org

Amazing! This fixes https://github.com/embeddings-benchmark/mteb/issues/774 ; feel free to merge!

tomaarsen changed pull request status to merged

Sign up or log in to comment