[Feature] JSON format output in ChatQnA

[weichengintel]

Priority

P3-Medium

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

Add a feature that allows the LLM to output in JSON format, similar to what OpenAI provides, by including a parameter in the HTTP request.

REF:

Structured Outputs - OpenAI API

[xiguiw]

@weichengintel

The example output format is of low priority.
Could you give some more background so that we can evaluate the priority of this feature?

[weichengintel]

@weichengintel

The example output format is of low priority. Could you give some more background so that we can evaluate the priority of this feature?

I also consider this low priority. My intention for adding it is mainly to check the model I am using through the output logs when interacting with the ChatQnA UI interface.