CQADupstackProgrammersRetrieval results are missing for a lot of models

[x-tabdeveloping]

A lot of models in the MTEB(eng, classic) tab are missing CQADupstackProgrammersRetrieval results and therefore get nan mean, and don't get displayed in the plot.
I checked and the files are missing here, in the results repo. Is this something we should run (@Muennighoff) or something we can fetch from the old results (@Samoed )?

[Samoed]

I think this should be runned, because curently when creating model card, then all CQADupstack datasets results are combined
https://github.com/embeddings-benchmark/mteb/blob/a6ce6f9b7050c1fad60e0c6e8985afa9356e2728/mteb/create_meta.py#L104

[KennethEnevoldsen]

Yep. I actually also think this will leave to a difference in the mean (which is a bit annoying). A solution is to allow custom aggregators for some benchmarks (might also fix the issue with displaying bright)