Add Ragas integration docs (#275)

* Predibase documentation * Added Predibase integration to Github repo Readme * Updated Predibase integration * Add Ragas integration docs * Remove outputs from ragas docs
comet-ml · Sep 18, 2024 · 6f449a4 · 6f449a4
1 parent b99f7d1
commit 6f449a4
Show file tree

Hide file tree

Showing 8 changed files with 54 additions and 16 deletions.
diff --git a/README.md b/README.md
@@ -117,6 +117,7 @@ The easiest way to get started is to use one of our integrations. Opik supports:
 | LangChain | Log traces for all LangChain LLM calls | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/langchain/?utm_source=opik&utm_medium=github&utm_content=langchain_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb) |
 | LlamaIndex | Log traces for all LlamaIndex LLM calls | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/llama_index?utm_source=opik&utm_medium=github&utm_content=llama_index_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |
 | Predibase | Fine-tune and serve open-source Large Language Models | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/predibase?utm_source=opik&utm_medium=github&utm_content=predibase_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb) |
+| Ragas | Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines | [Documentation](https://www.comet.com/docs/opik/tracing/integrations/ragas?utm_source=opik&utm_medium=github&utm_content=ragas_link) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb) |
 
 > [!TIP]  
 > If the framework you are using is not listed above, feel free to [open an issue](https://github.com/comet-ml/opik/issues) or submit a PR with the integration.

diff --git a/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb b/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb
@@ -51,7 +51,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install opik ragas --quiet\n",
+    "%pip install --quiet --upgrade opik ragas\n",
     "\n",
     "import os\n",
     "import getpass\n",
@@ -129,16 +129,18 @@
     "import asyncio\n",
     "from ragas.integrations.opik import OpikTracer\n",
     "from ragas.dataset_schema import SingleTurnSample\n",
+    "import os\n",
     "\n",
+    "os.environ[\"OPIK_PROJECT_NAME\"] = \"ragas-integration\"\n",
     "\n",
     "# Define the scoring function\n",
     "def compute_metric(metric, row):\n",
     "    row = SingleTurnSample(**row)\n",
     "\n",
-    "    opik_tracer = OpikTracer()\n",
+    "    opik_tracer = OpikTracer(tags=[\"ragas\"])\n",
     "\n",
     "    async def get_score(opik_tracer, metric, row):\n",
-    "        score = await metric.single_turn_ascore(row, callbacks=[OpikTracer()])\n",
+    "        score = await metric.single_turn_ascore(row, callbacks=[opik_tracer])\n",
     "        return score\n",
     "\n",
     "    # Run the async function using the current event loop\n",

diff --git a/apps/opik-documentation/documentation/docs/cookbook/ragas.md b/apps/opik-documentation/documentation/docs/cookbook/ragas.md
@@ -26,7 +26,7 @@ First, we will install the necessary libraries and configure the OpenAI API key.
 
 
 ```python
-%pip install opik ragas --quiet
+%pip install --quiet --upgrade opik ragas
 
 import os
 import getpass
@@ -82,16 +82,18 @@ nest_asyncio.apply()
 import asyncio
 from ragas.integrations.opik import OpikTracer
 from ragas.dataset_schema import SingleTurnSample
+import os
 
+os.environ["OPIK_PROJECT_NAME"] = "ragas-integration"
 
 # Define the scoring function
 def compute_metric(metric, row):
     row = SingleTurnSample(**row)
 
-    opik_tracer = OpikTracer()
+    opik_tracer = OpikTracer(tags=["ragas"])
 
     async def get_score(opik_tracer, metric, row):
-        score = await metric.single_turn_ascore(row, callbacks=[OpikTracer()])
+        score = await metric.single_turn_ascore(row, callbacks=[opik_tracer])
         return score
 
     # Run the async function using the current event loop

diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/langchain.md b/apps/opik-documentation/documentation/docs/tracing/integrations/langchain.md
@@ -22,6 +22,12 @@ To use the `CometTracer` with LangChain, you'll need to have both the `opik` and
 pip install opik langchain langchain_openai
 ```
 
+In addition, you can configure Opik using the `opik configure` command which will prompt you for the correct local server address or if you are using the Cloud platfrom your API key:
+
+```bash
+opik configure
+```
+
 ## Using CometTracer
 
 Here's a basic example of how to use the `CometTracer` callback with a LangChain chain:

diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/llama_index.md b/apps/opik-documentation/documentation/docs/tracing/integrations/llama_index.md
@@ -29,6 +29,12 @@ To use the Opik integration with LlamaIndex, you'll need to have both the `opik`
 pip install opik llama-index llama-index-agent-openai llama-index-llms-openai
 ```
 
+In addition, you can configure Opik using the `opik configure` command which will prompt you for the correct local server address or if you are using the Cloud platfrom your API key:
+
+```bash
+opik configure
+```
+
 ## Using the Opik integration
 
 To use the Opik integration with LLamaIndex, you can set the Opik callback handler as the global callback handler:

diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/openai.md b/apps/opik-documentation/documentation/docs/tracing/integrations/openai.md
@@ -14,15 +14,21 @@ This guide explains how to integrate Comet Opik with the OpenAI Python SDK. By u
   </a>
 </div>
 
-## Integration Steps
+## Getting started
 
-1. First, ensure you have both `opik` and `openai` packages installed:
+First, ensure you have both `opik` and `openai` packages installed:
 
 ```bash
 pip install opik openai
 ```
 
-2. Import the necessary modules and wrap the OpenAI client:
+In addition, you can configure Opik using the `opik configure` command which will prompt you for the correct local server address or if you are using the Cloud platfrom your API key:
+
+```bash
+opik configure
+```
+
+## Tracking OpenAI API calls
 
 ```python
 from opik.integrations.openai import track_openai

diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/overview.md b/apps/opik-documentation/documentation/docs/tracing/integrations/overview.md
@@ -14,5 +14,6 @@ Opik aims to make it as easy as possible to log, view and evaluate your LLM trac
 | LangChain | Log traces for all LangChain LLM calls | [Documentation](/tracing/integrations/langchain.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb) |
 | LlamaIndex | Log traces for all LlamaIndex LLM calls | [Documentation](/tracing/integrations/llama_index.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |
 | Predibase | Fine-tune and serve open-source LLMs | [Documentation](/tracing/integrations/predibase.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/predibase.ipynb) |
+| Ragas | Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines | [Documentation](/tracing/integrations/ragas.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb) |
 
 If you would like to see more integrations, please open an issue on our [GitHub repository](https://github.com/comet-ml/opik).
diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/ragas.md b/apps/opik-documentation/documentation/docs/tracing/integrations/ragas.md
@@ -12,6 +12,27 @@ There are two main ways to use Ragas with Opik:
 1. Using Ragas to score traces or spans.
 2. Using Ragas to evaluate a RAG pipeline.
 
+<div style="display: flex; align-items: center; flex-wrap: wrap; margin: 20px 0;">
+  <span style="margin-right: 10px;">You can check out the Colab Notebook if you'd like to jump straight to the code:</span>
+  <a href="https://colab.research.google.com/github/comet-ml/opik/blob/main/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb" target="_blank" rel="noopener noreferrer">
+    <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" style="vertical-align: middle;"/>
+  </a>
+</div>
+
+## Getting started
+
+You will first need to install the `opik` and `ragas` packages:
+
+```bash
+pip install opik ragas
+```
+
+In addition, you can configure Opik using the `opik configure` command which will prompt you for the correct local server address or if you are using the Cloud platfrom your API key:
+
+```bash
+opik configure
+```
+
 ## Using Ragas to score traces or spans
 
 Ragas provides a set of metrics that can be used to evaluate the quality of a RAG pipeline, a full list of the supported metrics can be found in the [Ragas documentation](https://docs.ragas.io/en/latest/references/metrics.html#).
@@ -109,13 +130,6 @@ In the Opik UI, you will be able to see the full trace including the score calcu
 
 We recommend using the Opik [evaluation framework](/evaluation/evaluate_your_llm) to evaluate your RAG pipeline. It shares similar concepts with the Ragas `evaluate` functionality but has a tighter integration with Opik.
 
-<div style="display: flex; align-items: center; flex-wrap: wrap; margin: 20px 0;">
-  <span style="margin-right: 10px;">You can check out the Colab Notebook if you'd like to jump straight to the code:</span>
-  <a href="https://colab.research.google.com/github/comet-ml/opik/blob/main/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb" target="_blank" rel="noopener noreferrer">
-    <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" style="vertical-align: middle;"/>
-  </a>
-</div>
-
 :::
 
 If you are using the Ragas `evaluate` functionality, you can use the `OpikTracer` callback to keep track of the score calculation in Opik. This will track as traces the computation of each evaluation metric: