Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Intra-Vendor Model Routing for AI APIs #2681

Open
5 tasks
CrowleyRajapakse opened this issue Jan 9, 2025 · 0 comments
Open
5 tasks

Implement Intra-Vendor Model Routing for AI APIs #2681

CrowleyRajapakse opened this issue Jan 9, 2025 · 0 comments
Assignees
@CrowleyRajapakse
Labels
Type/New Feature

Comments

@CrowleyRajapakse
Copy link
Contributor

CrowleyRajapakse commented Jan 9, 2025 •
edited
Loading

Problem

Need to implement Intra-Vendor Model Routing for AI APIs.

Solution

Tasks:

  • Implement Intra-Vendor Model Routing CRD related changes to provide Native CR experience. #2682
  • Implement Round Robin and Failover based on AI model logic using go enforcer in request flow.
  • Implement Intra-Vendor Model Routing APK Conf related changes to provide REST API experience in config-deployer.
  • Implement Intra-Vendor Model Routing APIM-APK-Agent related changes to retrieve changes relevant to new policies, multi-endpoints and AI Provider new changes to deploy updated CRs.
  • Adding Intra-Vendor Model Routing related integration and cucumber tests.

Affected Component

None

Version

1.3.0

Implementation

No response

Related Issues

No response

Suggested Labels

No response
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CrowleyRajapakse CrowleyRajapakse

Labels
Type/New Feature
Projects
APK Development
Status: In-Progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@CrowleyRajapakse