Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPIK-48] e2e tests #192

Merged
merged 10 commits into from
Sep 6, 2024
Merged

[OPIK-48] e2e tests #192

merged 10 commits into from
Sep 6, 2024

Conversation

alexkuzmik
Copy link
Collaborator

@alexkuzmik alexkuzmik commented Sep 6, 2024

This PR adds the first batch of e2e tests between the SDK and the backend.

Currently covered happy flows:

  1. Creating and populating a dataset
  2. Manual trace and span creation via Opik class.
  3. Automatic functions tracing via @track decorator
  4. Feedback logging for spans and traces via Span, Trace and Opik classes.
  5. Experiment creation via evaluate function call.
  6. Added get_trace_content() and get_span_content() methods to Opik class, they return dataclass instances with the corresponding trace/span data retrieved from the backend.

To run tests:

pytest tests/e2e
If you need to specify some env vars, you can do it like that:
OPIK_WORKSPACE=xxx OPIK_API_KEY=yyy pytest tests/e2e

Notes:

  1. All e2e tests are stored in tests/e2e
  2. Currently, the tests are running against the backend configured. Tests are NOT a part of the CI yet.
  3. Tests log traces to e2e-tests project
  4. Nothing is deleted after the test run except for the dataset (we'll need to discuss whether we need this deletion or no and then proceed with the rest of the data depending on what we decide)

@alexkuzmik alexkuzmik requested a review from a team as a code owner September 6, 2024 15:29
@alexkuzmik alexkuzmik self-assigned this Sep 6, 2024
@alexkuzmik alexkuzmik merged commit 81dee5e into main Sep 6, 2024
6 checks passed
@alexkuzmik alexkuzmik deleted the OPIK-48-e2e-tests branch September 6, 2024 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants