OPIK-546: Implement create chat completions endpoint #890
Conversation
|
|
||
| @Positive private Double backoffExp; | ||
|
|
||
| @MinDuration(value = 1, unit = TimeUnit.MILLISECONDS) |
There was a problem hiding this comment.
maybe the minimum value in milliseconds for timeouts(this and the next ones) could be higher, like at least 100? This way we protect users that dont know/understand if its seconds or milliseconds.
it can be very frustating for users setting a value like '10' imagining seconds and only getting timeouts.
There was a problem hiding this comment.
I wanted to make it as flexible as possible to not to restrict the user to any value. Also, I took some inspiration from the values in some of the core Dropwizard configs such as DataSourceFactory.
But your comment makes sense. I might update the values in a follow-up PR.
| openApiClient: | ||
| # See demo endpoint Langchain4j documentation: https://docs.langchain4j.dev/get-started | ||
| # Not https but only used for testing purposes. It's fine as long as not sensitive data is sent. | ||
| url: http://langchain4j.dev/demo/openai/v1 |
There was a problem hiding this comment.
is this a public endpoint provided by the library developers to support testing?
There was a problem hiding this comment.
Correct, you can find it in the documentation linked in the description of this PR.
A potential downside is that tests might fail or be flaky, if that endpoint is no longer supported or if they throttle us or if they make any backwards incompatible change etc.
I had zero issues while developing and also we are already hitting other external endpoints from other non-backend tests. Specially for the SDK integrations, including the overall end to end tests.
Considering all this, I think it's ok to follow this approach and we'll change it if any issue appears.
andrescrz
deleted the
andrescrz/OPIK-546-implement-create-chat-completions-endpoint
branch
Details
Implemented create chat completions endpoint:
A PR will follow to update the Open API spec.
Issues
Resolves OPIK-546
Testing
Documentation