⬆️✅ Support 0.6.5+ vllm

[evaline-ju]

vllm APIs including OpenAIServingChat that the chat detection base class is built on underwent some breaking changes. The decision was made in this PR to just update the lower bound of vllm instead of maintaining conditional support in the tests for 0.6.2-pre0.6.5, since vllm APIs move quickly. At time of writing, there are at least two patch versions 0.6.5-0.6.6 with these supported changes. Some post-0.6.6 but in main branch changes have been noted as inline comments.

Key changes

• https://github.com/vllm-project/vllm/pull/9919 building on https://github.com/vllm-project/vllm/pull/9358 added the non-optional chat_template_content_format field to OpenAIServingChat
• https://github.com/vllm-project/vllm/pull/10463 that allows extra fields now in the vllm API since the OpenAI API now allows extra fields, impacting request/response fields like ChatCompletionRequest [used to make the request to chat completions]
• https://github.com/vllm-project/vllm/pull/11164 added get_diff_sampling_params to model configs

Closes: #6

[gkumbhat]

hmm. this will change the general API behavior from our side. Does orchestrator expects bad request in such scenario or passthrough?

[evaline-ju]

This will just cause a passthrough of the variable from my testing. My worry is that adding additional validation when vllm and openAI allow passthrough is then we're even more tied to small API changes (like tracking all expected fields)