Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add blog for sqfp16 #2971

Merged
merged 10 commits into from
Jun 27, 2024
Merged

Add blog for sqfp16 #2971

merged 10 commits into from
Jun 27, 2024

Conversation

naveentatikonda
Copy link
Member

Description

Blog for SQFP16 Quantization with k-NN Plugin

Issues Resolved

#2950

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

@naveentatikonda
Add blog for sqfp16
01a673d 
Signed-off-by: Naveen Tatikonda <[email protected]>
@kolchfa-aws kolchfa-aws self-assigned this Jun 18, 2024
@kolchfa-aws
Doc review
ce48905 
Signed-off-by: Fanit Kolchina <[email protected]>
@kolchfa-aws
Formatting
44ba0d4 
Signed-off-by: Fanit Kolchina <[email protected]>
@kolchfa-aws
Copy link
Collaborator

@natebower The blog is ready for your review. Thank you!

natebower
natebower reviewed Jun 26, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws @naveentatikonda Please see my comments and changes and let me know if you have any questions. Thanks!

@pajuric This will be ready to publish once my comments/changes have been addressed.

_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated

`1.1 * (2 * 256 + 8 * 16) * 1,000,000 ~= 0.656 GB`

For more information about memory estimation for scalar quantization with IVF, refer to [this documentation](https://opensearch.org/docs/latest/search-plugins/knn/knn-vector-quantization/#memory-estimation-1).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably define IVF here, given that it has a much more widely used definition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe HNSW will be widely used than IVF due to drop in recall with IVF. So, we intentionally added estimates for HNSW here

_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated Show resolved Hide resolved
kolchfa-aws and others added 2 commits June 26, 2024 07:54
@kolchfa-aws @natebower
Apply suggestions from code review
9aeb84b 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@kolchfa-aws @natebower
Update _posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md
d2268a2 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: kolchfa-aws <[email protected]>
@naveentatikonda naveentatikonda force-pushed the sqfp16_blog branch 3 times, most recently from ff2ce70 to 1df6805 Compare June 26, 2024 18:35
smacrakis
smacrakis approved these changes Jun 26, 2024
View reviewed changes
Copy link

@smacrakis smacrakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. There are a few small cleanups to be finished.

@naveentatikonda
Address review comments
9077afd 
Signed-off-by: Naveen Tatikonda <[email protected]>
@naveentatikonda
Copy link
Member Author

Thanks @smacrakis

@natebower Can you please do a final review and then we can ship it

@pajuric
Copy link

pajuric commented Jun 26, 2024

@naveentatikonda - I will have updated meta for you later today. It will need to be implemented before it can push live tomorrow.

pajuric
pajuric reviewed Jun 26, 2024
View reviewed changes
_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md Outdated
date: 2024-06-19 00:00:00 -0700
categories:
- technical-posts
meta_keywords: FP16 quantization, OpenSearch k-NN plugin, memory optimization, cost-effectiveness, performance, search latency, indexing throughput
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naveentatikonda - Please update your meta with the following:

meta_keywords: faiss scalar quantization, OpenSearch k-NN plugin, FP16 scalar quantization, vector embeddings

meta_description: Learn how FP16 scalar quantization in OpenSearch helps your generate vector embeddings while reducing memory requirements and minimizing quality loss at a lower cost.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pajuric in meta description, we are not generating vector embeddings with this feature

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naveentatikonda - I'm fine with your meta changes, but the phrase I pulled was from your blog. You may want to reevaluate that line. If you are good, I will push this to publish.

Copy link
Member Author

@naveentatikonda naveentatikonda Jun 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pajuric I fixed it. We are good to go, pls publish it.Thanks!

@natebower
Copy link
Collaborator

Thanks @smacrakis

@natebower Can you please do a final review and then we can ship it

If all of my comments have been addressed, then you should be good to go 😄.

@naveentatikonda
Copy link
Member Author

naveentatikonda commented Jun 27, 2024 •
edited
Loading

If all of my comments have been addressed, then you should be good to go 😄.

Sounds good. Thanks @natebower, they are all addressed

@naveentatikonda
Rephrase sentences
ce24fbb 
Signed-off-by: Naveen Tatikonda <[email protected]>
@pajuric
Copy link

pajuric commented Jun 27, 2024

@nateynateynate @krisfreedain - New blog ready to publish today.

nateynateynate
nateynateynate approved these changes Jun 27, 2024
View reviewed changes
Copy link
Member

@nateynateynate nateynateynate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good locally

@nateynateynate nateynateynate merged commit a347f82 into opensearch-project:main Jun 27, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pajuric pajuric pajuric left review comments

@kolchfa-aws kolchfa-aws kolchfa-aws left review comments

@smacrakis smacrakis smacrakis approved these changes

@nateynateynate nateynateynate nateynateynate approved these changes

@elfisher elfisher Awaiting requested review from elfisher elfisher is a code owner

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@nknize nknize Awaiting requested review from nknize nknize is a code owner

@krisfreedain krisfreedain Awaiting requested review from krisfreedain krisfreedain is a code owner

@peterzhuamazon peterzhuamazon Awaiting requested review from peterzhuamazon peterzhuamazon is a code owner

@CEHENKLE CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

@dtaivpp dtaivpp Awaiting requested review from dtaivpp dtaivpp is a code owner

@natebower natebower Awaiting requested review from natebower natebower is a code owner

Assignees

@kolchfa-aws kolchfa-aws

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@naveentatikonda @kolchfa-aws @pajuric @natebower @nateynateynate @smacrakis