docs: refactor /docs

pingcap · Dec 16, 2024 · fa347f1 · fa347f1
1 parent 5612134
commit fa347f1
Show file tree

Hide file tree

Showing 8 changed files with 74 additions and 61 deletions.
diff --git a/README.md b/README.md
@@ -5,20 +5,20 @@
   <a href='https://www.pingcap.com/tidb-cloud-serverless/?utm_source=tidb.ai&utm_medium=community'>
     <img src="https://raw.githubusercontent.com/pingcap/tidb.ai/main/frontend/app/public/nextra/icon-dark.svg" alt="AutoFlow" width =100 height=100></img>
   </a>
-</div>
 
-[![Backend Docker Image Version](https://img.shields.io/docker/v/tidbai/backend?sort=semver&arch=amd64&label=tidbai%2Fbackend&color=blue&logo=fastapi)](https://hub.docker.com/r/tidbai/backend)
-[![Frontend Docker Image Version](https://img.shields.io/docker/v/tidbai/frontend?sort=semver&arch=amd64&label=tidbai%2Ffrontend&&color=blue&logo=next.js)](https://hub.docker.com/r/tidbai/frontend)
-[![E2E Status](https://img.shields.io/github/check-runs/pingcap/tidb.ai/main?nameFilter=E2E%20Test&label=e2e)](https://tidb-ai-playwright.vercel.app/)
+  <a href="https://trendshift.io/repositories/12294" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12294" alt="pingcap%2Fautoflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
 
-## Introduction
+  [![Backend Docker Image Version](https://img.shields.io/docker/v/tidbai/backend?sort=semver&arch=amd64&label=tidbai%2Fbackend&color=blue&logo=fastapi)](https://hub.docker.com/r/tidbai/backend)
+  [![Frontend Docker Image Version](https://img.shields.io/docker/v/tidbai/frontend?sort=semver&arch=amd64&label=tidbai%2Ffrontend&&color=blue&logo=next.js)](https://hub.docker.com/r/tidbai/frontend)
+  [![E2E Status](https://img.shields.io/github/check-runs/pingcap/tidb.ai/main?nameFilter=E2E%20Test&label=e2e)](https://tidb-ai-playwright.vercel.app/)
+</div>
 
-<a href="https://trendshift.io/repositories/12294" target="_blank"><img src="https://trendshift.io/api/badge/repositories/12294" alt="pingcap%2Fautoflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
+## Introduction
 
-An open source GraphRAG (Knowledge Graph) built on top of [TiDB Vector](https://www.pingcap.com/ai?utm_source=tidb.ai&utm_medium=community) and [LlamaIndex](https://github.com/run-llama/llama_index) and [DSPy](https://github.com/stanfordnlp/dspy).
+AutoFlow is an open source graph rag (graphrag: knowledge graph rag) based knowledge base tool built on top of [TiDB Vector](https://www.pingcap.com/ai?utm_source=tidb.ai&utm_medium=community) and [LlamaIndex](https://github.com/run-llama/llama_index) and [DSPy](https://github.com/stanfordnlp/dspy).
 
-- **Live Demo**: [TiDB.AI](https://tidb.ai)
-- **Documentation**: [Docs](https://tidb.ai/docs/?utm_source=github&utm_medium=tidb.ai)
+- **Live Demo**: [https://tidb.ai](https://tidb.ai?utm_source=tidb.ai&utm_medium=community)
+- **Deployment Docs**: [Deployment Docs](https://tidb.ai/docs/?utm_source=github&utm_medium=tidb.ai)
 
 ## Features
 
@@ -60,4 +60,4 @@ We welcome contributions from the community. If you are interested in contributi
 
 ## License
 
-TiDB.AI is open-source under the Apache License, Version 2.0. You can [find it here](/LICENSE.txt).
+AutoFlow is open-source under the Apache License, Version 2.0. You can [find it here](/LICENSE.txt).
diff --git a/frontend/app/src/pages/docs/_meta.ts b/frontend/app/src/pages/docs/_meta.ts
@@ -7,7 +7,7 @@ export default {
   },
   "index": "Introduction",
   "quick-start": "Quick Start",
-  "videos": "Videos",
+  "resources": "Resources",
   "faq": "FAQ",
   "-- Deploy This RAG": {
     "type": "separator",

diff --git a/frontend/app/src/pages/docs/deploy-with-docker.mdx b/frontend/app/src/pages/docs/deploy-with-docker.mdx
@@ -6,13 +6,11 @@ This document provides instructions for deploying the entire Autoflow applicatio
 
 ## Prerequisites
 
-- Set up a [TiDB Serverless cluster](https://docs.pingcap.com/tidbcloud/tidb-cloud-quickstart).  
-
+- Set up a TiDB cluster, you can use either:
+  - [TiDB Cloud Serverless](https://docs.pingcap.com/tidbcloud/tidb-cloud-quickstart)(recommended)
+  - [TiDB Self-Managed](https://docs.pingcap.com/tidb/stable/production-deployment-using-tiup)(>=v8.4).
 - Install [Docker Compose](https://docs.docker.com/compose/install/).
 
-> **Note:**
-> TiDB Vector Search is only available for TiDB Self-Managed (TiDB >= v8.4) and [TiDB Cloud Serverless](https://ai.pingcap.com/) for now.
-
 ## Deploy
 
 1. Clone the repository:
@@ -30,11 +28,13 @@ This document provides instructions for deploying the entire Autoflow applicatio
     ```
 
     Replace the following placeholders with your own values:
-    - `SECRET_KEY`: you can generate a random secret key using `python3 -c "import secrets; print(secrets.token_urlsafe(32))"`
-    - `TIDB_HOST`, `TIDB_USER`, `TIDB_PASSWORD` and `TIDB_DATABASE`: get them from your [TiDB Serverless cluster](https://tidbcloud.com/)
-      - Note: TiDB Serverless will provide a default database name called `test`, if you want to use another database name, you need to create a new database in the TiDB Serverless console.
-      - Note: Don't use '#' in the password, it will cause an error when connecting to the database. See [issue](https://github.com/pydantic/pydantic/issues/8061).
-    - `EMBEDDING_MAX_TOKENS`: set them according to the embedding model you choose before, it can not be changed after the deployment.
+    - `SECRET_KEY`: you can generate a random secret key using:
+      ```python
+      `python3 -c "import secrets; print(secrets.token_urlsafe(32))"`
+      ```
+    - `TIDB_HOST`, `TIDB_USER`, `TIDB_PASSWORD` and `TIDB_DATABASE`: you can get these values from the TiDB cluster you set up before.
+      - Note: if you are using a self-managed TiDB cluster, you need to set `TIDB_SSL=false`.
+    - `EMBEDDING_MAX_TOKENS`: set them according to the embedding model you choose before, it can not be changed after the deployment.(We will remove it in the future, and move it to the admin panel)
 
 3. Migrate the database schema:
 
@@ -64,20 +64,19 @@ This document provides instructions for deploying the entire Autoflow applicatio
 
 5. Start the services:
 
+    If you are using a SaaS embedding model, start the services with the following command:
+
     ```bash
     docker compose up
     ```
 
-    To use the local embedding model, start with the following command:
+    If you want to use the built-in local embedding reranker, start the services with the following command:
 
     ```bash
     docker compose --profile local-embedding-reranker up
     ```
 
-6. Open your browser and visit `http://localhost:3000` to access the web interface.
-
-That's it! You can now use pingcap/autoflow locally. You can also go to https://tidb.ai to experience the live demo.
-
+6. Done! Now you can open your browser and visit [http://localhost:3000](http://localhost:3000) locally
 
 ## Configuration
 
@@ -95,18 +94,18 @@ After you deploy the application, you need to initialize the application by foll
 
 This section will help you upgrade pingcap/autoflow to the new version.
 
-Suppose you want to upgrade pingcap/autoflow from 0.1.0 to version 0.2.0
+Suppose you want to upgrade pingcap/autoflow from 0.3.0 to 0.3.1. Follow these steps:
 
 1. Edit your docker-compose.yml file to use the new image version.
 
     ```yaml
     services:
       backend:
-        image: tidbai/backend:0.2.0
+        image: tidbai/backend:0.3.1
       frontend:
-        image: tidbai/frontend:0.2.0
+        image: tidbai/frontend:0.3.1
       background:
-        image: tidbai/backend:0.2.0
+        image: tidbai/backend:0.3.1
     ```
 
 2. Pull the new image:

diff --git a/frontend/app/src/pages/docs/javascript.mdx b/frontend/app/src/pages/docs/javascript.mdx
@@ -1,31 +1,21 @@
 # Embeddable JavaScript Snippet
 
-This is the **key feature** of our conversational search RAG (Retrieval-Augmented Generation) app.
-
-With little effort, you can integrate our conversational search window into your website by copying and embedding a simple JavaScript code snippet. This widget, typically placed at the bottom right corner of your site, facilitates instant responses to product-related queries.
-
+What if you could integrate a conversational search window into your website to facilitate instant responses to product-related queries? With the embeddable JavaScript snippet, you can do just that. This feature allows you to copy and embed a simple JavaScript code snippet into your website, typically placed at the bottom right corner of your site.
 
 ## How to Embed
 
 To embed the JavaScript snippet, follow these steps:
 
 1. Enter Settings > JavaScript, and copy the JavaScript code snippet like this:
 
-![embeddable-javascript-snippet](https://github.com/user-attachments/assets/e743af11-d19c-4219-a1ae-0e618c51abdb)
-
+![embeddable-javascript-snippet](https://github.com/user-attachments/assets/5322dc31-d4d5-492f-87dd-dd976c245ac3)
 
 2. Paste the JavaScript code snippet into your website's HTML file;
 
 3. Initialize the JavaScript snippet with input parameters like `Widget Title`, `Site Name`, `Search Titles`, `Example Questions`, `Footer Links`, and `Social Media Links`;
 
 4. Restart your website to see the conversational search window appear at the bottom right corner of your site.
 
-![embeddable-javascript-snippet](https://github.com/pingcap/autoflow/assets/1237528/5a445231-a27a-4ae6-8287-a4f8cf7b64d0 "Image Title")
-
-
-
-
-That's all you need to do to integrate our conversational search window into your website. Enjoy! 
-
+![embeddable-javascript-snippet](https://github.com/user-attachments/assets/f0dc82db-c14d-4863-a242-c7da3a719568 "Image Title")
 
-You can also go to https://tidb.ai to experience the live demo.
+That's all you need to do to integrate our conversational search window into your website. Enjoy! You can also go to https://tidb.ai and https://docs.pingcap.com to see the chat window in action.
diff --git a/frontend/app/src/pages/docs/llm.mdx b/frontend/app/src/pages/docs/llm.mdx
@@ -11,7 +11,7 @@ After logging in with an admin account, you can configure the LLM in the admin p
 1. Click on the `Models > LLMs` tab;
 2. Click on the `New LLM` button to add a new LLM;
 
-    ![llm-config](https://github.com/user-attachments/assets/118fe7ba-c0fd-459c-ad2c-12c5afba05ef "LLM Config")
+    ![llm-config](https://github.com/user-attachments/assets/993eec34-a99a-4acf-b4b7-a4ee8e28e3d5 "LLM Config")
 
 3. Input your LLM information and click `Create LLM` button;
 4. Done!
@@ -27,10 +27,17 @@ If you want to use the new LLM while answering user queries, you need switch to
 Currently Autoflow supports the following LLM providers:
 
 - [OpenAI](https://platform.openai.com/)
-- [Gemini](https://gemini.google.com/)
-- OpenAI Like
+- [Google Gemini](https://gemini.google.com/)
+- [Anthropic Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude)
+- [Amazon Bedrock](https://aws.amazon.com/bedrock/)
+- And all OpenAI-Like models:
     - [OpenRouter](https://openrouter.ai/)
-        - Default config: `{"api_base": "https://openrouter.ai/api/v1/"}`
+        - Default config:
+        ```json
+        {
+            "api_base": "https://openrouter.ai/api/v1/"
+        }
+        ```
     - [BigModel](https://open.bigmodel.cn/)
         - Default config:
         ```json
@@ -39,7 +46,10 @@ Currently Autoflow supports the following LLM providers:
             "is_chat_model": true
         }
         ```
-- [Bedrock](https://aws.amazon.com/bedrock/)
-- [Anthropic Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude)
-- [Ollama](https://ollama.com/)
-    - Default config: `{"base_url": "http://localhost:11434"}`
+    - [Ollama](https://ollama.com/)
+        - Default config:
+        ```
+        {
+            "base_url": "http://localhost:11434"
+        }
+        ```
diff --git a/frontend/app/src/pages/docs/requirements.mdx b/frontend/app/src/pages/docs/requirements.mdx
@@ -3,17 +3,20 @@ In this section, we will cover the requirements for deploying the project.
 
 
 ## LLM(Large Language Model) and Embedding Model
-
 * A saas LLM model like OpenAI API or self-hosted LLM model with requirements:
   * Smarter than GPT-3.5
   * Provide openai-like API
-* Embedding model. AutoFlow needs a llm module with openai-like API. You can use the [Jina AI API](https://jina.ai/) for this purpose. It is free for 1M tokens.
+* Embedding model: AutoFlow needs an embedding model to translate the text into vectors. You can use the following:
+  * OpenAI-like embedding model
+  * Cohere embedding model
+  * ZhipuAI embedding model
+  * You can also use the [Jina AI API](https://jina.ai/) for this purpose. It is free for 1M tokens.
 * (Optional) Reranker. You can use the [Jina AI API](https://jina.ai/) for this purpose. It is free for 1M tokens.
 
 
 ## TiDB
 * With [TiDB Serverless](https://pingcap.com/ai) account, you can setup a TiDB cluster with Vector Search enabled. Free quota is available for 1M RU per month.
-* You can also use a self-hosted TiDB cluster(>v8.4) with Vector Search enabled.
+* You can also use a self-hosted TiDB cluster(>v8.4) with Vector Search enabled, but please note it will require TiFlash enabled for Vector Search.
 
 
 ## Hardware

diff --git a/frontend/app/src/pages/docs/reranker-model.mdx b/frontend/app/src/pages/docs/reranker-model.mdx
@@ -1,6 +1,6 @@
 # Configure Reranker Model
 
-The Reranker is a machine learning model that is trained to optimize the order of results retrieved from the initial vector searches.
+In a RAG system, a reranker is a model that re-ranks the retrieved documents based on the user's query. The reranker model is used to improve the relevance of the retrieved documents.
 
 ## Configure Reranker Model
 
@@ -9,13 +9,13 @@ After logging in with an admin account, you can configure the Reranker Model in
 1. Click on the `Models > Reranker Models` tab;
 2. Click on the `New Reranker Model` button to add a new Reranker;
 
-![reranker-config](https://github.com/user-attachments/assets/eeacb71f-0e6e-4755-9b53-0cb6d234e92f "Reranker Config")
+![reranker-config](https://github.com/user-attachments/assets/f31bf557-16b3-4f7e-9e70-00c9fd7a5b1b "Reranker Config")
 
 3. Input your Reranker information and click `Create Reranker` button;
 
 ## Supported Reranker Providers
 
-Currently Autoflow supports the following Reranker providers:
+Currently AutoFlow supports the following reranker providers:
 
 * [Jina AI Reranker](https://jina.ai/reranker)
 * [Cohere](https://cohere.com/rerank)
diff --git a/frontend/app/src/pages/docs/videos.mdx → frontend/app/src/pages/docs/resources.mdx b/frontend/app/src/pages/docs/videos.mdx → frontend/app/src/pages/docs/resources.mdx
@@ -1,9 +1,20 @@
-# Videos
+# Resources
 
 Here are some videos tutorials help you to understand the basics of the platform and how to use it.
 
+## Presentation: Intro to TiDB (Graph) RAG
+> Agenda:
+> - Intro to TiDB RAG / Graph RAG
+> - How it works?
+> - Vector RAG + Graph RAG + Fulltext Search(WIP)
+> - Plan-able RAG engine - ngaut/StackVM
+> - Benefits of using TiDB for AI app storage
+> - How to copy one?
 
-## Intro to Post-Verification and Graph Search
+Google Slides: [Intro to TiDB (Graph) RAG](https://docs.google.com/presentation/d/1uK9oOW3QGHiNoqYUpFRqHPN4J_FvB2qeDsmAFMKASaY/edit?usp=sharing)
+
+## Video: Intro to PingCAP/AutoFlow Post-Verification and Graph Search
 > This video will show you how to use the platform to search(from graph) for information and verify the results.
 > We introduce a new feature called Post-Verification which allows you to verify the results and correct them if needed.
-<iframe width="560" height="315" src="https://www.youtube.com/embed/Hbj9TvRKTZw?si=cxc7lya28cQJWMlD" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+<iframe width="560" height="315" src="https://www.youtube.com/embed/Hbj9TvRKTZw?si=cxc7lya28cQJWMlD" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
+