Skip to content

Latest commit

 

History

History
136 lines (98 loc) · 2.84 KB

api_server.md

File metadata and controls

136 lines (98 loc) · 2.84 KB
Raw

API Server

Manage API server

Start api server locally:

llm-serve start apiserver

Stop api server locally:

llm-serve stop apiserver

API list

Start models serving

Start Serving

POST /api/start_serving HTTP/1.1

Content-Type: application/json
Accept: application/json
user-name: string

Body parameter

[
  {
    "model_id": "string",
    "model_task": "string",
    "model_revision": "string",
    "is_oob": true,
    "scaling_config": {
      "num_workers": 0,
      "num_gpus_per_worker": 1,
      "num_cpus_per_worker": 1
    }
  }
]

Parameters

Name In Type Required Description
body body Body_start_serving_start_serving_post true none

Responses

Status Meaning Description Schema
200 OK Successful Response Inline
422 Unprocessable Entity Validation Error HTTPValidationError

List serving status and predict URL

Serving URL

GET /api/list_serving HTTP/1.1

Content-Type: application/json
Accept: application/json
user-name: string

Body parameter

[
  {
    "model_id": "string",
    "model_revision": "main"
  }
]

Parameters

Name In Type Required Description
body body Body_serving_url_list_serving_get true none

Responses

Status Meaning Description Schema
200 OK Successful Response Inline
422 Unprocessable Entity Validation Error HTTPValidationError

Stop models serving

Delete Serving

POST /api/stop_serving HTTP/1.1

Content-Type: application/json
Accept: application/json
user-name: string

Body parameter

[
  {
    "model_id": "string",
    "model_revision": "main"
  }
]

Parameters

Name In Type Required Description
body body Body_delete_serving_delete_serving_post true none

Responses

Status Meaning Description Schema
200 OK Successful Response Inline
422 Unprocessable Entity Validation Error HTTPValidationError

Detailed API Documents

See more API documents and OpenAPI Specification.