Skip to content

Latest commit

 

History

History
51 lines (32 loc) · 2.73 KB

example.md

File metadata and controls

51 lines (32 loc) · 2.73 KB
Raw

Example

Before running an example, MUST enter the specific example directory. For example, to run the map example, switch to the map directory, and then use python to run it.

git clone https://github.com/zilliztech/gpt-cache
cd gpt-cache

cd examples/map
python map_manager.py

If the running example includes a model or complex third-party library (like: faiss, towhee), the first run may take some time as it needs to download the model runtime environment, model data, dependencies, etc. However, the subsequent runs will be significantly faster.

Basic example

How to use the map to cache data.

Sqlite + Faiss manage cache data

How to use the sqlite to store the scale data and the faiss to query the vector data.

Sqlite + Faiss + Towhee

On the basis of the above example, use towhee for embedding operation

Note: the default embedding model only support the ENGLISH. If you want to use the Chinese, you can use the uer/albert-base-chinese-cluecorpussmall model. For other languages, you should use the corresponding model.

Sqlite + Milvus + Towhee

How to use the sqlite to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

Benchmark

The benchmark script about the Sqlite + Faiss + Towhee

Test data source: Randomly scrape some information from the webpage (origin), and then let chatgpt produce corresponding data (similar).

  • threshold: answer evaluation threshold, A smaller value means higher consistency with the content in the cache, a lower cache hit rate, and a lower cache miss hit; a larger value means higher tolerance, a higher cache hit rate, and at the same time also have higher cache misses.
  • positive: effective cache hit, which means entering similar to search and get the same result as origin
  • negative: cache hit but the result is wrong, which means entering similar to search and get the different result as origin
  • fail count: cache miss

data file: mock_data.json similarity evaluation func: pair_evaluation (search distance)

threshold average time positive negative fail count
20 0.04s 455 27 517
50 0.09s 871 86 42
100 0.12s 905 93 1