Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement multi tenancy in Flow Framework #980

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from

Conversation

dbwiddis
Copy link
Member

@dbwiddis dbwiddis commented Dec 13, 2024

Description

Uses the multitenant remote metadata client developed in ML Commons feature/multi_tenancy branch to migrate Flow Framework system indices to use the same pattern.

Draft - WIP

Submitting in logical steps to ease maintainer review. Suggestions:

  1. To start, review each of the first several commits to understand how the tenant id is implemented and passed around
  2. Then visit the tenant aware integ test classes to understand what it is we're trying to allow/prohibit

To use this locally, you will need to clone https://github.com/dbwiddis/opensearch-remote-metadata-sdk and run ./gradlew publishToMavenLocal. That is in the process of getting its own repo with publication to maven; this is a temporary workaround to keep the work unblocked.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added the backport 2.x backport PRs to 2.x branch label Dec 13, 2024
@dbwiddis dbwiddis force-pushed the multi-tenancy branch 2 times, most recently from 60e881d to f611e41 Compare December 13, 2024 21:55
@dbwiddis dbwiddis force-pushed the multi-tenancy branch 10 times, most recently from 090727f to 8cd5b8f Compare December 27, 2024 21:39
Copy link

codecov bot commented Dec 27, 2024

Codecov Report

Attention: Patch coverage is 60.91772% with 247 lines in your changes missing coverage. Please review.

Project coverage is 76.45%. Comparing base (0284554) to head (1a9f856).

Files with missing lines Patch % Lines
...framework/indices/FlowFrameworkIndicesHandler.java 51.65% 68 Missing and 5 partials ⚠️
...ework/transport/CreateWorkflowTransportAction.java 66.36% 28 Missing and 9 partials ⚠️
.../opensearch/flowframework/util/EncryptorUtils.java 64.64% 26 Missing and 9 partials ⚠️
...ework/transport/DeleteWorkflowTransportAction.java 55.17% 10 Missing and 3 partials ⚠️
...h/flowframework/rest/RestCreateWorkflowAction.java 60.00% 6 Missing and 4 partials ⚠️
...flowframework/transport/handler/SearchHandler.java 62.50% 8 Missing and 1 partial ⚠️
.../opensearch/flowframework/FlowFrameworkPlugin.java 33.33% 7 Missing and 1 partial ⚠️
.../opensearch/flowframework/model/WorkflowState.java 57.89% 4 Missing and 4 partials ⚠️
.../org/opensearch/flowframework/util/ParseUtils.java 60.00% 5 Missing and 3 partials ⚠️
...owframework/rest/AbstractSearchWorkflowAction.java 50.00% 7 Missing ⚠️
... and 13 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #980      +/-   ##
============================================
- Coverage     77.57%   76.45%   -1.12%     
- Complexity      993     1011      +18     
============================================
  Files            99      101       +2     
  Lines          4714     4983     +269     
  Branches        431      472      +41     
============================================
+ Hits           3657     3810     +153     
- Misses          877      943      +66     
- Partials        180      230      +50     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Daniel Widdis <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
@@ -34,6 +34,22 @@ jobs:
options: --user root

steps:
# TEMPORARY until this is on Maven
- name: Set up JDK 21

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious why do we need to setup JDK again when are doing this at line 56?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport PRs to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants