Skip to content

Commit

Permalink
Clean up Getting started section
Browse files Browse the repository at this point in the history
  • Loading branch information
@kkraune
kkraune committed Dec 19, 2024
1 parent 1b5fcf6 commit 37f3036
Showing 1 changed file with 58 additions and 65 deletions.
123 changes: 58 additions & 65 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,94 +19,84 @@ also [PyVespa examples](https://pyvespa.readthedocs.io/en/latest/examples.html).



## Getting started - Basic Sample Applications
## Getting started


### Basic album-recommendation
The [album-recommendation](album-recommendation/) is the intro application to Vespa.
Learn how to configure the schema for simple recommendation and search use cases.

[Pyvespa: Hybrid Search - Quickstart](https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa.html) and
[Pyvespa: Hybrid Search - Quickstart on Vespa Cloud](https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html)
create a hybrid text search application combining traditional keyword matching with semantic vector search (dense retrieval).
They also demonstrate the Vespa native embedder functionality.
These are intro level applications for Python users using more advanced Vespa features.
Use [Pyvespa: Authenticating to Vespa Cloud](https://pyvespa.readthedocs.io/en/latest/authenticating-to-vespa-cloud.html) for Vespa Cloud credentials.

### Vector Search
There is a growing interest in AI-powered vector representations of unstructured multimodal data
and searching efficiently over these representations.
[vector-search](https://github.com/vespa-cloud/vector-search)
describes how your organization can unlock the full potential of multimodal AI-powered vector representations
using Vespa Cloud - the industry-leading managed Vector Search Service.

[Pyvespa: Querying Vespa](https://pyvespa.readthedocs.io/en/latest/query.html)
is a good start for Python users, exploring how to query Vespa using the Vespa Query Language (YQL).

### Simple hybrid semantic search
The [simple semantic search](simple-semantic-search/)
application demonstrates indexed vector search using `HNSW`,
creating embedding vectors from a transformer language model inside Vespa, and hybrid
text and semantic ranking. This app also demonstrates using native Vespa embedders.


### Retrieval Augmented Generation (RAG)
The [retrieval-augmented-generation](retrieval-augmented-generation) sample application
demonstrates how to build an end-to-end RAG pipeline with API-based and local LLMs.
[Pyvespa: Read and write operations](https://pyvespa.readthedocs.io/en/latest/reads-writes.html)
documents ways to feed, get, update and delete data;
Using context manager with with for efficiently managing resources
and feeding streams of data using `feed_iter` which can feed from streams, Iterables, Lists
and files by the use of generators.

[Pyvespa: Application packages](https://pyvespa.readthedocs.io/en/latest/application-packages.html)
is a good intro to the concept of application packages in Vespa.
Try [Advanced Configuration](https://pyvespa.readthedocs.io/en/latest/advanced-configuration.html) for Vespa Services configuration.

### Indexing multiple vectors per field
The [Vespa Multi-Vector Indexing with HNSW](multi-vector-indexing/) app demonstrates how to
index multiple vectors per document field for semantic search for longer documents.
[Pyvespa: Examples](https://pyvespa.readthedocs.io/en/latest/examples/pyvespa-examples.html)
is a repository of small snippets and examples, e.g. really simple vector distance search applications.


### Vespa streaming mode for naturally partitioned data
The [vector-streaming-search](vector-streaming-search) app
demonstrates how to use vector streaming search. See also [blog post](https://blog.vespa.ai/announcing-vector-streaming-search/).


### ColBERT token-level embeddings
The [colbert](colbert) application demonstrates how to
use the Vespa [colbert-embedder](https://docs.vespa.ai/en/embedding.html#colbert-embedder)
for explainable semantic search with better accuracy than regular
text embedding models.


### ColBERT token-level embeddings for long documents
The [colbert-long](colbert-long) application demonstrates how to
use the Vespa [colbert-embedder](https://docs.vespa.ai/en/embedding.html#colbert-embedder)
for explainable semantic search for longer documents.

## Ranking
* [Using Mixedbread.ai cross-encoder for reranking in Vespa.ai](https://pyvespa.readthedocs.io/en/latest/examples/cross-encoders-for-global-reranking.html)
* [Standalone ColBERT + Vespa for long-context ranking](https://pyvespa.readthedocs.io/en/latest/examples/colbert_standalone_long_context_Vespa-cloud.html)
* [Standalone ColBERT with Vespa for end-to-end retrieval and ranking](https://pyvespa.readthedocs.io/en/latest/examples/colbert_standalone_Vespa-cloud.html)
* [LightGBM: Training the model with Vespa features](https://pyvespa.readthedocs.io/en/latest/examples/lightgbm-with-categorical.html)
* [LightGBM: Mapping model features to Vespa features](https://pyvespa.readthedocs.io/en/latest/examples/lightgbm-with-categorical-mapping.html)

### SPLADE sparse learned weights for ranking
The [splade](splade) application demonstrates how to
use the Vespa [splade-embedder](https://docs.vespa.ai/en/embedding.html#splade-embedder) for
semantic search using sparse vector representations.


### Multilingual semantic search
The [multilingual](multilingual-search) sample application demonstrates multilingual semantic search
with multilingual text embedding models.
## Vector Search, Hybrid Search and Embeddings
There is a growing interest in AI-powered vector representations of unstructured multimodal data
and searching efficiently over these representations.

[vector-search](https://github.com/vespa-cloud/vector-search)
describes how your organization can unlock the full potential of multimodal AI-powered vector representations using Vespa Cloud -
the industry-leading managed Vector Search Service.

### Customizing embeddings
The [custom-embeddings](custom-embeddings) application demonstrates customizing frozen document embeddings for downstream tasks.
The [simple semantic search](simple-semantic-search/)
application demonstrates indexed vector search using `HNSW`,
creating embedding vectors from a transformer language model inside Vespa, and hybrid text and semantic ranking.
This app also demonstrates using native Vespa embedders.

The [Vespa Multi-Vector Indexing with HNSW](multi-vector-indexing/) app demonstrates how to
index multiple vectors per document field for semantic search for longer documents.

### Pyvespa intro notebooks
* [Querying Vespa](https://pyvespa.readthedocs.io/en/latest/query.html)
* [Read and write operations](https://pyvespa.readthedocs.io/en/latest/reads-writes.html)
* [Hybrid Search - Quickstart](https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa.html)
* [Hybrid Search - Quickstart on Vespa Cloud](https://pyvespa.readthedocs.io/en/latest/getting-started-pyvespa-cloud.html)
* [Application packages](https://pyvespa.readthedocs.io/en/latest/application-packages.html)
* [Advanced Configuration](https://pyvespa.readthedocs.io/en/latest/advanced-configuration.html)
* [Authenticating to Vespa Cloud](https://pyvespa.readthedocs.io/en/latest/authenticating-to-vespa-cloud.html)
* [Pyvespa examples](https://pyvespa.readthedocs.io/en/latest/examples/pyvespa-examples.html)
The [vector-streaming-search](vector-streaming-search) app
demonstrates how to use vector streaming search for naturally partitioned data.
See also [blog post](https://blog.vespa.ai/announcing-vector-streaming-search/).

The [colbert](colbert) application (simple hybrid search with ColBERT) demonstrates how to
use the Vespa [colbert-embedder](https://docs.vespa.ai/en/embedding.html#colbert-embedder)
for explainable semantic search with better accuracy than regular text embedding models.

The [multilingual](multilingual-search) sample application demonstrates multilingual semantic search
with multilingual text embedding models.

## Ranking
* [Using Mixedbread.ai cross-encoder for reranking in Vespa.ai](https://pyvespa.readthedocs.io/en/latest/examples/cross-encoders-for-global-reranking.html)
* [Standalone ColBERT + Vespa for long-context ranking](https://pyvespa.readthedocs.io/en/latest/examples/colbert_standalone_long_context_Vespa-cloud.html)
* [Standalone ColBERT with Vespa for end-to-end retrieval and ranking](https://pyvespa.readthedocs.io/en/latest/examples/colbert_standalone_Vespa-cloud.html)
* [LightGBM: Training the model with Vespa features](https://pyvespa.readthedocs.io/en/latest/examples/lightgbm-with-categorical.html)
* [LightGBM: Mapping model features to Vespa features](https://pyvespa.readthedocs.io/en/latest/examples/lightgbm-with-categorical-mapping.html)
ColBERT token-level embeddings for long documents;
The [colbert-long](colbert-long) application demonstrates how to
use the Vespa [colbert-embedder](https://docs.vespa.ai/en/embedding.html#colbert-embedder)
for explainable semantic search for longer documents.

SPLADE sparse learned weights for ranking;
The [splade](splade) application demonstrates how to
use the Vespa [splade-embedder](https://docs.vespa.ai/en/embedding.html#splade-embedder) for
semantic search using sparse vector representations.

The [custom-embeddings](custom-embeddings) application demonstrates customizing frozen document embeddings for downstream tasks.

## Vector Search, Hybrid Search and Embeddings
* [Billion-scale vector search with Cohere binary embeddings in Vespa](https://pyvespa.readthedocs.io/en/latest/examples/billion-scale-vector-search-with-cohere-embeddings-cloud.html)
* [Multi-vector indexing with HNSW](https://pyvespa.readthedocs.io/en/latest/examples/multi-vector-indexing.html)
* [BGE-M3 - The Mother of all embedding models](https://pyvespa.readthedocs.io/en/latest/examples/mother-of-all-embedding-models-cloud.html)
Expand All @@ -126,7 +116,10 @@ The [custom-embeddings](custom-embeddings) application demonstrates customizing



## RAG and Generative AI
## Retrieval Augmented Generation (RAG) and Generative AI
The [retrieval-augmented-generation](retrieval-augmented-generation) sample application
demonstrates how to build an end-to-end RAG pipeline with API-based and local LLMs.

* [Turbocharge RAG with LangChain and Vespa Streaming Mode for Partitioned Data](https://pyvespa.readthedocs.io/en/latest/examples/turbocharge-rag-with-langchain-and-vespa-streaming-mode-cloud.html)
* [Visual PDF RAG with Vespa - ColPali demo application](https://pyvespa.readthedocs.io/en/latest/examples/visual_pdf_rag_with_vespa_colpali_cloud.html)
* [Chat with your pdfs with ColBERT, langchain, and Vespa](https://pyvespa.readthedocs.io/en/latest/examples/chat_with_your_pdfs_using_colbert_langchain_and_Vespa-cloud.html)
Expand Down

0 comments on commit 37f3036

Please sign in to comment.