Spaces:

mteb

/

arena

like 63

Running

Great work! I'm happy to see there is an arena for MTEB. Could you please add the UAE-Large-V1 model into the arena as well? It is also an open-source model and is licensed under MIT.

Thank you!

Thanks for this PR! Would be happy to add UAE-Large-V1 - it's really awesome work. Our issue at the moment is that we need more votes in order to make the leaderboard reliable. If we add more models, then we will have less votes per model and thus higher confidence intervals which will hurt the reliability of the Arena's leaderboard. We think that one key feature to increase the voting would be solving this: https://github.com/embeddings-benchmark/arena/issues/14 - If you have bandwidth to fix that issue that would be amazing and we'd be happy to add UAE 👍 The retrieval part also costs us a few hundred USD per model, but we're happy to cover that.