Jacques/docs v1.1 (#180)

* Updated frontend contribution guide * Fixed broken links * Fixed broken links * Updated PyPi readme file * Fix issue with SDK setup.py file * Fix linter issues --------- Co-authored-by: Nimrod Lahav <[email protected]>
comet-ml · Sep 5, 2024 · 576bf1c · 576bf1c
1 parent 6bf21e3
commit 576bf1c
Show file tree

Hide file tree

Showing 30 changed files with 90 additions and 129 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -239,6 +239,9 @@ The Opik frontend is a React application that is located in `apps/opik-frontend`
 In order to run the frontend locally, you need to have `npm` installed. Once installed, you can run the frontend locally using the following command:
 
 ```bash
+# Run the backend locally with the flag "--local-fe"
+./build_and_run.sh --local-fe
+
 cd apps/opik-frontend
 
 # Install dependencies - Only needs to be run once
@@ -248,20 +251,16 @@ npm install
 npm run start
 ```
 
-You can then access the development frontend at `http://localhost:5174/`. Any change you make to the frontend will be updated in real-time.
-
-The dev server is set up to work with Opik BE run on `http://localhost:8080`. All requests made to `http://localhost:5174/api` are proxied to the backend.
-The server port can be changed in `vite.config.ts` file section `proxy`.
+You can then access the development frontend at `http://localhost:5173/`. Any change you make to the frontend will be updated in real-time.
 
-> [!NOTE]
-> You will need to have the backend running locally in order for the frontend to work. For this, we recommend running a local instance of Opik using `opik server install`.
+> You will need to open the FE using `http://localhost:5173/` ignoring the output from the `npm run start` command which will recommend to open `http://localhost:5174/`. In case `http://localhost:5174/` is opened, the BE will not be accessible.
 
 Before submitting a PR, please ensure that your code passes the test suite, the linter and the type checker:
 
 ```bash
 cd apps/opik-frontend
 
-npm run test
+npm run e2e
 npm run lint
 npm run typecheck
 ```

diff --git a/README.md b/README.md
diff --git a/apps/opik-documentation/documentation/docs/cookbook/evaluate_hallucination_metric.ipynb b/apps/opik-documentation/documentation/docs/cookbook/evaluate_hallucination_metric.ipynb
@@ -17,7 +17,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/cookbook/evaluate_moderation_metric.ipynb b/apps/opik-documentation/documentation/docs/cookbook/evaluate_moderation_metric.ipynb
@@ -19,7 +19,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb b/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb
@@ -23,7 +23,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb b/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb
@@ -25,7 +25,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb b/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb
@@ -17,7 +17,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb b/apps/opik-documentation/documentation/docs/cookbook/ragas.ipynb
@@ -22,7 +22,7 @@
     "\n",
     "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
     "\n",
-    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+    "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
    ]
   },
   {

diff --git a/apps/opik-documentation/documentation/docs/evaluation/concepts.md b/apps/opik-documentation/documentation/docs/evaluation/concepts.md
diff --git a/apps/opik-documentation/documentation/docs/evaluation/overview.md b/apps/opik-documentation/documentation/docs/evaluation/overview.md
diff --git a/apps/opik-documentation/documentation/docs/home.md b/apps/opik-documentation/documentation/docs/home.md
@@ -18,23 +18,23 @@ During development, you can use the platform to log, view and debug your LLM tra
 
 1. Log traces using:
 
-    a. One of our [integrations](/tracing/integrations/overview).
+    a. One of our [integrations](/tracing/integrations/overview.md).
 
-    b. The `@track` decorator for Python, learn more in the [Logging Traces](/tracing/log_traces) guide.
+    b. The `@track` decorator for Python, learn more in the [Logging Traces](/tracing/log_traces.md) guide.
 
 3. [Annotate and label traces](/tracing/annotate_traces) through the SDK or the UI.
 
 ### Evaluation and Testing
 
 Evaluating the output of your LLM calls is critical to ensure that your application is working as expected and can be challenging. Using the Comet LLM Evaluation platformm, you can:
 
-1. Use one of our [LLM as a Judge evaluators](/evaluation/metrics/overview) or [Heuristic evaluators](/evaluation/metrics/heuristic_metrics) to score your traces and LLM calls
-2. [Store evaluation datasets](/evaluation/manage_datasets) in the platform and [run evaluations](/evaluation/evaluate_your_llm)
-3. Use our [pytest integration](/testing/pytest_integration) to track unit test results and compare results between runs
+1. Use one of our [LLM as a Judge evaluators](/evaluation/metrics/overview.md) or [Heuristic evaluators](/evaluation/metrics/heuristic_metrics.md) to score your traces and LLM calls
+2. [Store evaluation datasets](/evaluation/manage_datasets.md) in the platform and [run evaluations](/evaluation/evaluate_your_llm.md)
+3. Use our [pytest integration](/testing/pytest_integration.md) to track unit test results and compare results between runs
 
 
 ## Getting Started
 
 [Comet](https://www.comet.com/site) provides a managed Cloud offering for Opik, simply [create an account](https://www.comet.com/signup?from=llm) to get started.
 
-You can also run Opik locally using our [local installer](//self-host/self_hosting_opik#all-in-one-installation). If you are looking for a more production ready deployment, you can also use our [Kubernetes deployment option](/self-host/self_hosting_opik#kubernetes-installation).
+You can also run Opik locally using our [local installer](/self-host/self_hosting_opik.md#all-in-one-installation). If you are looking for a more production ready deployment, you can also use our [Kubernetes deployment option](/self-host/self_hosting_opik.md#kubernetes-installation).
diff --git a/apps/opik-documentation/documentation/docs/monitoring/_category_.json b/apps/opik-documentation/documentation/docs/monitoring/_category_.json
diff --git a/apps/opik-documentation/documentation/docs/monitoring/add_traces_to_dataset.md b/apps/opik-documentation/documentation/docs/monitoring/add_traces_to_dataset.md
diff --git a/apps/opik-documentation/documentation/docs/monitoring/annotate_traces.md b/apps/opik-documentation/documentation/docs/monitoring/annotate_traces.md
diff --git a/apps/opik-documentation/documentation/docs/monitoring/overview.md b/apps/opik-documentation/documentation/docs/monitoring/overview.md
diff --git a/apps/opik-documentation/documentation/docs/quickstart.md b/apps/opik-documentation/documentation/docs/quickstart.md
@@ -9,7 +9,7 @@ This guide helps you integrate the Opik platform with your existing LLM applicat
 
 ## Set up
 
-Getting started is as simple as creating an [account on Comet](https://www.comet.com/signup?from=llm) or [self-hosting the platform](/self-host/self_hosting_opik).
+Getting started is as simple as creating an [account on Comet](https://www.comet.com/signup?from=llm) or [self-hosting the platform](/self-host/self_hosting_opik.md).
 
 Once your account is created, you can start logging traces by installing the Opik Python SDK:
 
@@ -48,6 +48,6 @@ def your_llm_application(input):
     return output
 ```
 
-To learn more about the `track` decorator, see the [`track` documentation](./track). Once the traces are logged, you can view them in the OPIK UI:
+To learn more about the `track` decorator, see the [`track` documentation](/tracing/log_traces.md#log-using-function-annotators). Once the traces are logged, you can view them in the OPIK UI:
 
 ![Opik Traces](/img/home/traces_page_for_quickstart.png)
diff --git a/apps/opik-documentation/documentation/docs/testing/pytest_integration.md b/apps/opik-documentation/documentation/docs/testing/pytest_integration.md
@@ -33,7 +33,7 @@ When you run the tests, Opik will create a new experiment for each run and log e
 ![Test Experiments](/img/testing/test_experiments.png)
 
 :::tip
-If you are evaluating your LLM application during development, we recommend using the `evaluate` function as it will provide you with a more detailed report. You can learn more about the `evaluate` function in the [evaluation documentation](/evaluation/evaluate_your_llm).
+If you are evaluating your LLM application during development, we recommend using the `evaluate` function as it will provide you with a more detailed report. You can learn more about the `evaluate` function in the [evaluation documentation](/evaluation/evaluate_your_llm.md).
 :::
 
 ### Advanced Usage

diff --git a/apps/opik-documentation/documentation/docs/tracing/annotate_traces.md b/apps/opik-documentation/documentation/docs/tracing/annotate_traces.md
@@ -1,5 +1,5 @@
 ---
-sidebar_position: 5
+sidebar_position: 3
 sidebar_label: Annotate Traces
 ---
 
@@ -124,6 +124,5 @@ Comet supports many different LLM as a Judge metrics out of the box including:
 * `AnswerRelevanceMetric`
 * `ContextRecallMetric`
 * `ContextPrecisionMetric`
-* `ContextRelevancyMetric`
 
-You can find a full list of metrics in the [LLM as a Judge Metrics](/evaluation/metrics/llm_as_a_judge_metrics.md) section.
+You can find a full list of supported metrics in the [Metrics Overview](/evaluation/metrics/overview.md) section.
diff --git a/apps/opik-documentation/documentation/docs/tracing/concepts.md b/apps/opik-documentation/documentation/docs/tracing/concepts.md
diff --git a/apps/opik-documentation/documentation/docs/tracing/integrations/overview.md b/apps/opik-documentation/documentation/docs/tracing/integrations/overview.md
@@ -10,8 +10,8 @@ Opik aims to make it as easy as possible to log, view and evaluate your LLM trac
 
 | Integration | Description | Documentation | Try in Colab |
 | ----------- | ----------- | ------------- | ------------ |
-| OpenAI | Log traces for all OpenAI LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/openai) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb) |
-| LangChain | Log traces for all LangChain LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/langchain) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb) |
-| LlamaIndex | Log traces for all LlamaIndex LLM calls | [Documentation](https://www.comet.com/docs/opik/integrations/llama_index) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |
+| OpenAI | Log traces for all OpenAI LLM calls | [Documentation](/tracing/integrations/openai.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/openai.ipynb) |
+| LangChain | Log traces for all LangChain LLM calls | [Documentation](/tracing/integrations/langchain.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/langchain.ipynb) |
+| LlamaIndex | Log traces for all LlamaIndex LLM calls | [Documentation](/tracing/integrations/llama_index.md) | [![Open Quickstart In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/comet-ml/opik/blob/master/apps/opik-documentation/documentation/docs/cookbook/llama-index.ipynb) |
 
 If you would like to see more integrations, please open an issue on our [GitHub repository](https://github.com/comet-ml/opik).
diff --git a/apps/opik-documentation/documentation/docs/tracing/log_distributed_traces.md b/apps/opik-documentation/documentation/docs/tracing/log_distributed_traces.md
@@ -1,6 +1,6 @@
 ---
-sidebar_position: 4
-sidebar_label: Log DistributedTraces
+sidebar_position: 2
+sidebar_label: Log Distributed Traces
 ---
 
 # Log Distributed Traces

diff --git a/apps/opik-documentation/documentation/docs/tracing/log_traces.md b/apps/opik-documentation/documentation/docs/tracing/log_traces.md
@@ -1,5 +1,5 @@
 ---
-sidebar_position: 3
+sidebar_position: 1
 sidebar_label: Log Traces
 ---
 
@@ -18,7 +18,7 @@ pip install opik
 Once the SDK is installed, you can log traces to using one our Comet's integration, function annotations or manually.
 
 :::tip
-Opik has a number of integrations for popular LLM frameworks like LangChain or OpenAI, checkout a full list of integrations in the [integrations](/tracing/integrations/overview) section.
+Opik has a number of integrations for popular LLM frameworks like LangChain or OpenAI, checkout a full list of integrations in the [integrations](/tracing/integrations/overview.md) section.
 :::
 
 ## Log using function annotators
@@ -128,7 +128,7 @@ def llm_chain(input_text):
     span.update(name="llm_chain")
 ```
 
-You can learn more about the `Trace` object in the [Trace reference docs](/sdk-reference-docs/Objects/Trace.html) and the `Span` object in the [Span reference docs](/sdk-reference-docs/Objects/Span.html).
+You can learn more about the `Trace` object in the [Trace reference docs](/python-sdk-reference/Objects/Trace.html) and the `Span` object in the [Span reference docs](/python-sdk-reference/Objects/Span.html).
 
 ## Log scores to traces and spans
 

diff --git a/apps/opik-documentation/documentation/docs/tracing/overview.md b/apps/opik-documentation/documentation/docs/tracing/overview.md
diff --git a/apps/opik-documentation/documentation/docusaurus.config.ts b/apps/opik-documentation/documentation/docusaurus.config.ts
@@ -68,7 +68,7 @@ const config: Config = {
         {
           to: process.env.NODE_ENV === 'development' 
             ? 'http://localhost:8000' 
-            : '/sdk-reference-docs',
+            : '/python-sdk-reference',
           label: 'Python SDK reference docs',
           position: 'left',
           className: "header-external-link",

diff --git a/apps/opik-documentation/python-sdk-docs/source/Objects/Span.rst b/apps/opik-documentation/python-sdk-docs/source/Objects/Span.rst
@@ -1,6 +1,6 @@
 Span
 ====
 
-.. autoclass:: opik.Span
+.. autoclass:: opik.api_objects.span.Span
     :members:
-    :inherited-members:
+    :inherited-members:
diff --git a/apps/opik-documentation/python-sdk-docs/source/Objects/Trace.rst b/apps/opik-documentation/python-sdk-docs/source/Objects/Trace.rst
@@ -1,6 +1,6 @@
 Trace
 =====
 
-.. autoclass:: opik.Trace
+.. autoclass:: opik.api_objects.trace.Trace
     :members:
-    :inherited-members:
+    :inherited-members:
diff --git a/apps/opik-documentation/python-sdk-docs/source/index.rst b/apps/opik-documentation/python-sdk-docs/source/index.rst
@@ -155,6 +155,7 @@ You can learn more about the `opik` python SDK in the following sections:
 
    integrations/openai/index
    integrations/langchain/index
+   integrations/llama_index/index
 
 .. toctree::
    :caption: Objects
@@ -164,3 +165,9 @@ You can learn more about the `opik` python SDK in the following sections:
    Objects/Span.rst
    Objects/FeedbackScoreDict.rst
    Objects/UsageDict.rst
+
+.. toctree::
+   :caption: Documentation Guides
+   :maxdepth: 1
+
+   Opik Documentation <https://www.comet.com/docs/opik/>
diff --git a/...n/python-sdk-docs/source/integrations/llama_index/LlamaIndexCallbackHandler.rst b/...n/python-sdk-docs/source/integrations/llama_index/LlamaIndexCallbackHandler.rst
@@ -0,0 +1,4 @@
+LlamaIndexCallbackHandler
+=========================
+
+.. autofunction:: opik.integrations.llama_index.LlamaIndexCallbackHandler
diff --git a/apps/opik-documentation/python-sdk-docs/source/integrations/llama_index/index.rst b/apps/opik-documentation/python-sdk-docs/source/integrations/llama_index/index.rst
@@ -0,0 +1,19 @@
+llama_index
+===========
+
+Opik integrates with LlamaIndex to allow you to log your LlamaIndex calls to the Opik platform. To enable the logging to Opik, simply set::
+    
+    from llama_index.core import Settings
+    from llama_index.core.callbacks import CallbackManager
+    from opik.integrations.llama_index import LlamaIndexCallbackHandler
+
+    opik_callback_handler = LlamaIndexCallbackHandler()
+    Settings.callback_manager = CallbackManager([opik_callback_handler])
+
+You can learn more about the `LlamaIndexCallbackHandler` callback in the following section:
+
+.. toctree::
+   :maxdepth: 4
+   :titlesonly:
+
+   LlamaIndexCallbackHandler
diff --git a/sdks/python/setup.py b/sdks/python/setup.py
@@ -1,10 +1,7 @@
-from pathlib import Path
-
 from setuptools import find_packages, setup
 import os
 
 project_urls = {"Source code": "https://github.com/comet-ml/opik"}
-this_directory = Path(__file__).parent
 
 setup(
     author="Comet ML Inc.",
@@ -13,7 +10,7 @@
     classifiers=[
         "Development Status :: 2 - Pre-Alpha",
         "Intended Audience :: Developers",
-        "License :: OSI Approved :: MIT License",
+        "License :: OSI Approved :: Apache Software License",
         "Natural Language :: English",
         "Programming Language :: Python :: 3 :: Only",
         "Programming Language :: Python :: 3",
@@ -22,7 +19,7 @@
         "Programming Language :: Python :: 3.10",
     ],
     description="Comet tool for logging and evaluating LLM traces",
-    long_description=open("README.md", encoding="utf-8").read(),
+    long_description=open("./../../README.md", encoding="utf-8").read(),
     long_description_content_type="text/markdown",
     install_requires=[
         "httpx<1.0.0",
@@ -52,5 +49,5 @@
     project_urls=project_urls,
     version=os.environ.get("VERSION", "0.0.1"),
     zip_safe=False,
-    license="MIT",
+    license="Apache 2.0 License",
 )
-Original file line number
+Diff line change
@@ Expand Up / @@ -17,7 +17,7 @@ @@
         "\n",
         "[Comet](https://www.comet.com/site) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm) and grab you API Key.\n",
         "\n",
-        "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik) for more information."
+        "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/self_hosting_opik/) for more information."
        ]
       },
       {
@@ Expand Down @@