local agents are producing only 100-150 tokens response

[DuskoPre]

hello,
i'm using AGiXT with docker and have a local LLM Mistral-7B-OpenOrca for inference. i've tried several LLM and each just produces only 100-150 tokens (~460 characters) response. the max_tokens setting is changed to 16384 in the Defaults.py and config.json (in agents/local agent folder). the gpt4all agent gives >350 tokens (~1500 characters) response. how can i set up the max_tokens of response for a local agent?

[DuskoPre]

for me it is like catching the clouds while dancing in the dark- how to set-up the max_tokens settings inside the local-llm module?

[Josh-XT]

Local-LLM has gone through a lot of changes. It is multimodal now, so the name LLM made no more sense. It has been rebranded to ezlocalai. The local provider will be removed from AGiXT in a coming update. ezlocalai can be used with the OpenAI provider in AGiXT. You will just need to modify your agent settings, change the API_URL to your ezlocalai URL, your OPENAI_API_KEY to your ezlocalai API key if you set one, and the AI_MODEL to the model that you are running in ezlocalai.

[Josh-XT]

Local provider has been replaced by ezLocalai in the last release.