Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add documentation for OSB query randomization #8990

Merged
merged 15 commits into from
Jan 9, 2025
Merged

Add documentation for OSB query randomization #8990

merged 15 commits into from
Jan 9, 2025

Conversation

peteralfonsi
Copy link
Contributor

Description

Adds a documentation page describing what OSB query randomization is and how to use it. As @IanHoang suggested in opensearch-project/opensearch-benchmark#712 I've put this under the Optimizing Benchmarks section.

Issues Resolved

Closes #8989

Version

The whole page describes how the feature will work once the pending opensearch-project/opensearch-benchmark#712 goes in, which will likely go into OSB 1.12. Most of the feature is present in OSB 1.3 and one flag was added in OSB 1.8. I can split up this PR into separate chunks to be backported separately if that's the right way to do things.

Frontend features

N/A

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions GitHub Actions
Copy link

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

Address linter issues
543b91f 
Signed-off-by: Peter Alfonsi <[email protected]>
@peteralfonsi
Copy link
Contributor Author

Addressed most of the style guide complaints, but it seems to be upset by the proper noun "Zipf" as in "Zipf distribution".

@Naarcha-AWS Naarcha-AWS added Blocked PR: Cannot move forward without assistance benchmark labels Jan 2, 2025
@Naarcha-AWS
Copy link
Collaborator

Adding a blocked label to this for now until the development PR is merged. I'll go ahead and edit the content in the meantime.

@Naarcha-AWS Naarcha-AWS added the 3 - Tech review PR: Tech review in progress label Jan 2, 2025
@Naarcha-AWS Naarcha-AWS removed the Blocked PR: Cannot move forward without assistance label Jan 7, 2025
@Naarcha-AWS
Add writer edits
dd67e64 
Signed-off-by: Naarcha-AWS <[email protected]>
@Naarcha-AWS
Copy link
Collaborator

@peteralfonsi and @IanHoang: I added my edits on top of this. Can one or both of y'all take a look and make sure my adjustments are still technically accurate?

@peteralfonsi
Copy link
Contributor Author

Looks good to me. I tweaked the new language in the Overview section slightly, to make it clearer why we need to reuse values, and also to make it clearer that the generation of saved value pairs is something that OSB does itself, rather than something the user has to do.

Tweak overview section slightly
883a25a 
Signed-off-by: Peter Alfonsi <[email protected]>
@peteralfonsi peteralfonsi force-pushed the osb-randomization-doc branch from 9fad08d to 883a25a Compare January 7, 2025 20:49
@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 3 - Tech review PR: Tech review in progress labels Jan 9, 2025
@Naarcha-AWS
Remove passive voice
578bf32 
Signed-off-by: Naarcha-AWS <[email protected]>
natebower
natebower requested changes Jan 9, 2025
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS @peteralfonsi Please see my comments and changes and tag me for approval when complete. Thanks!

_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated

Use the following CLI flags to customize randomization:

- `--randomization-enabled` turns randomization on and off. If randomization is not enabled, none of the randomization flags will have an effect.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"will have an effect" => "will be applied"?

_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
_benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md Outdated Show resolved Hide resolved
@Naarcha-AWS @natebower
Apply suggestions from code review
da62200 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Naarcha-AWS and others added 2 commits January 9, 2025 10:45
@Naarcha-AWS @natebower
Apply suggestions from code review
1d89075 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
@Naarcha-AWS
Apply suggestions from code review
7dc48e3 
Signed-off-by: Naarcha-AWS <[email protected]>
@Naarcha-AWS
Copy link
Collaborator

@natebower: Fanit's suggestion implemented, waiting for your approval.

natebower
natebower approved these changes Jan 9, 2025
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS LGTM

@Naarcha-AWS Naarcha-AWS merged commit 9f34935 into opensearch-project:main Jan 9, 2025
6 checks passed
@Naarcha-AWS Naarcha-AWS added backport 2.10 PR: Backport label for 2.10 backport 2.18 PR: Backport label for 2.18 and removed backport 2.10 PR: Backport label for 2.10 labels Jan 10, 2025
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 10, 2025
@github-actions @Naarcha-AWS @natebower
Add documentation for OSB query randomization (#8990)
752d1b8 
* Add documentation for OSB query randomization

Signed-off-by: Peter Alfonsi <[email protected]>

* Address linter issues

Signed-off-by: Peter Alfonsi <[email protected]>

* Add writer edits

Signed-off-by: Naarcha-AWS <[email protected]>

* Tweak overview section slightly

Signed-off-by: Peter Alfonsi <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Remove passive voice

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md

Signed-off-by: Nathan Bower <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Nathan Bower <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
(cherry picked from commit 9f34935)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 10, 2025
@github-actions @Naarcha-AWS @natebower
Add documentation for OSB query randomization (#8990)
4f2c9bd 
* Add documentation for OSB query randomization

Signed-off-by: Peter Alfonsi <[email protected]>

* Address linter issues

Signed-off-by: Peter Alfonsi <[email protected]>

* Add writer edits

Signed-off-by: Naarcha-AWS <[email protected]>

* Tweak overview section slightly

Signed-off-by: Peter Alfonsi <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Remove passive voice

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update _benchmark/user-guide/optimizing-benchmarks/randomizing-queries.md

Signed-off-by: Nathan Bower <[email protected]>

---------

Signed-off-by: Peter Alfonsi <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Signed-off-by: Nathan Bower <[email protected]>
Co-authored-by: Peter Alfonsi <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
(cherry picked from commit 9f34935)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This was referenced Jan 10, 2025
Merged
Merged
github-actions bot pushed a commit that referenced this pull request Jan 10, 2025
@opensearch-trigger-bot
Add documentation for OSB query randomization (#8990) (#9044)
d4a09be
github-actions bot pushed a commit that referenced this pull request Jan 10, 2025
@opensearch-trigger-bot
Add documentation for OSB query randomization (#8990) (#9045)
0742d56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Naarcha-AWS Naarcha-AWS Naarcha-AWS left review comments

@natebower natebower natebower approved these changes

@kolchfa-aws kolchfa-aws Awaiting requested review from kolchfa-aws kolchfa-aws is a code owner

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

@epugh epugh Awaiting requested review from epugh epugh is a code owner

Assignees

@Naarcha-AWS Naarcha-AWS

Labels
5 - Editorial review PR: Editorial review in progress backport 2.18 PR: Backport label for 2.18 benchmark
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Add documentation for OpenSearch Benchmarks query randomization
4 participants
@peteralfonsi @Naarcha-AWS @natebower @kolchfa-aws