Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update benchmark results #2700

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

update benchmark results #2700

wants to merge 3 commits into from

Conversation

StefanBogdan
Copy link
Member

@StefanBogdan StefanBogdan commented Oct 15, 2024

What's being changed:

The benchmark results: new benchmark results to be published with latest Weaviate version and new infrastructure.

Type of change:

  • Documentation updates (non-breaking change to fix/update documentation)

How Has This Been Tested?

  • GitHub action – automated build completed without errors
  • Local build - the site works as expected when running yarn start

note, you can run yarn verify-links to test site links locally

@StefanBogdan StefanBogdan marked this pull request as draft October 15, 2024 12:59
@StefanBogdan
Copy link
Member Author

StefanBogdan commented Oct 15, 2024

For this PR I would need some help with the results we want to show on the documentation page. Currently it has ALL the results that were collected which is more than we would need to show. The results are stored in the new ann-{dataset}.mdx files. The new script collate-results.py from this PR can be used to generate/analyse the results.

Another thing that needs to be checked is the Recommended configuration for each dataset. I chose one that looked pretty balanced to me but let me know if there is a better one I can use.

And the last thing is the QPS vs Recall plots. Currently it is just PNG photo. I can create PNGs that have a transparent background with either black or white axes and text, the only thing I could not figure out is how can I switch between the black or white image depending on the light/dark mode used. Let me know if we want to further investigate how to achieve different plots depending on the mode used.

@weaviate-git-bot
Copy link

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Slack channel?

@StefanBogdan
Copy link
Member Author

@databyjp
Copy link
Contributor

Amazing. I will take a look later this week (was away last week), thanks so much @StefanBogdan !

@StefanBogdan
Copy link
Member Author

Here is the live lint to the website.

@StefanBogdan StefanBogdan marked this pull request as ready for review November 14, 2024 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants