-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial text and document embedders implementation #1
Conversation
Signed-off-by: Gad Markovits <[email protected]>
Signed-off-by: Gad Markovits <[email protected]>
Signed-off-by: Gad Markovits <[email protected]>
Signed-off-by: Gad Markovits <[email protected]>
* Adds an endpoint for image ingestion Signed-off-by: Melanie Buehler <[email protected]> * Combined image and video endpoint Signed-off-by: Melanie Buehler <[email protected]> * Add test and update README Signed-off-by: Melanie Buehler <[email protected]> * fixed variable name for embedding model (#1) Signed-off-by: okhleif-IL <[email protected]> * Fixed test script Signed-off-by: Melanie Buehler <[email protected]> * Remove redundant function Signed-off-by: Melanie Buehler <[email protected]> * get_videos, delete_videos --> get_files, delete_files (opea-project#3) Signed-off-by: okhleif-IL <[email protected]> * Updates test per review feedback Signed-off-by: Melanie Buehler <[email protected]> * Fixed test Signed-off-by: Melanie Buehler <[email protected]> * Add support for audio files multimodal data ingestion (opea-project#4) * Add support for audio files multimodal data ingestion Signed-off-by: dmsuehir <[email protected]> * Update function name Signed-off-by: dmsuehir <[email protected]> --------- Signed-off-by: dmsuehir <[email protected]> * Change videos_with_transcripts to ingest_with_text Signed-off-by: Melanie Buehler <[email protected]> * Add image support to video ingestion with transcript functionality Signed-off-by: Melanie Buehler <[email protected]> * Update test and README Signed-off-by: Melanie Buehler <[email protected]> * Updated for review suggestions Signed-off-by: Melanie Buehler <[email protected]> * Add two tests for ingest_with_text Signed-off-by: Melanie Buehler <[email protected]> * LVM TGI Gaudi update for prompts without images (opea-project#7) * LVM Gaudi TGI update for prompts without images Signed-off-by: dmsuehir <[email protected]> * Wording Signed-off-by: dmsuehir <[email protected]> * Add a test Signed-off-by: dmsuehir <[email protected]> --------- Signed-off-by: dmsuehir <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change dummy image to be b64 encoded instead of the url (opea-project#9) Signed-off-by: dmsuehir <[email protected]> * Updates based on review feedback (opea-project#10) Signed-off-by: dmsuehir <[email protected]> * Test fix (opea-project#11) Signed-off-by: dmsuehir <[email protected]> --------- Signed-off-by: Melanie Buehler <[email protected]> Signed-off-by: okhleif-IL <[email protected]> Signed-off-by: dmsuehir <[email protected]> Co-authored-by: dmsuehir <[email protected]> Co-authored-by: Omar Khleif <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abolfazl Shahbazi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks quite good to me already! 👍 Just left some minor comments about to_dict
and from_dict
.
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/generators/generator.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/generators/generator.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/generators/generator.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/document_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
…anges Signed-off-by: Gad Markovits <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Found one last typo in the output of OPEATextEmbedder
. Everything else looks good to me. Happy to approve right after the typo is fixed!
comps/integrations/haystack/src/opea_haystack/embedders/tei/text_embedder.py
Outdated
Show resolved
Hide resolved
Signed-off-by: Gad Markovits <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks very good to me! 👍
Description
Implemented a text embedder and a document embedder to integrate OPEA with Haystack.
Issues
RFC
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
Haystack
Tests
Ran e2e test on a local machine against a Xeon instance running the ChatQnA example.