Configuring LibreChat to connect to Ollama appears to use the openai.js client instead of the ollama.js client.

[kirys]

What happened?

Configuring LibreChat to connect to Ollama appears to use the openai.js client instead of the ollama.js client.
This results in minor incompatibilities due to the behavior of the OpenAI client, as described in issue #3330.

This assumption is based primarily on observations in the log files (e.g., chat completion reports) and the encountered behavior, which aligns with that of the OpenAI client.

Best regards
K.

Steps to Reproduce

Configure custom provider as in the doc

endpoints:
  custom:
    - name: "Ollama"
      apiKey: "ollama"
      timeoutMs: 180000
      # use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
      baseURL: "http://---:11434/v1/"
      models:
        default: [
          "llama3.2-vision:11b-instruct-q8_0"
          ]
        # fetching list of models is supported but the `name` field must start
        # with `ollama` (case-insensitive), as it does in this example.
        fetch: true
      titleConvo: true
      titleModel: "current_model"
      summarize: false
      summaryModel: "current_model"
      forcePrompt: false
      modelDisplayLabel: "Ollama"

Look in the debug log i see that is using openaiclient

What browsers are you seeing the problem on?

Firefox

Relevant log output

2024-12-20T09:56:34.835Z debug: Making request to http://---:11434/v1/chat/completions
2024-12-20T09:56:34.836Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-12-20T09:56:46.602Z debug: Making request to http://---:11434/v1/chat/completions
2024-12-20T09:56:58.820Z debug: Making request to http://---:11434/v1/chat/completions
2024-12-20T09:57:10.123Z warn: [OpenAIClient.chatCompletion][stream] API error
2024-12-20T09:57:10.124Z error: [handleAbortError] AI response error; aborting request: Connection error.
2024-12-20T09:57:10.126Z debug: [AskController] Request closed

...

2024-12-21T09:52:10.123Z debug: [OpenAIClient] chatCompletion

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

This is expected, the ollama client is only used when attaching images or using agents.

I use ollama and do not experience the issue, but I can see about always using the client as suggested.

[kirys]

Got it but the problem is that the OpenAI timeout logic create problems with low end hardware.
An alternative would be to accept the timeoutMs and pass it to the openai client.

[gshockbu]

Same challenge. getting the AI response error; aborting request: connection error.
Using docker with ollama and one model. Localhost for all.
Able to use curl to validate the api endpoint for ollama.

Debug log:
{
baseURL: "http://host.docker.internal:11434/v1",
modelOptions.model: "llama3.2:latest",
modelOptions.user: "677d4a682dfc2759f0a5a47e",
modelOptions.stream: true,
// 5 message(s)
modelOptions.messages: [{"role":"user","content":"tet"},{"role":"assistant","content":"An error occurred while processing your request. Please contact the A... [truncated],{"role":"user","content":"test"},{"role":"assistant","content":"An error occurred while processing your request. Please contact the A... [truncated],{"role":"user","content":"test"}],
}
2025-01-15T20:04:41.658Z debug: Making request to http://host.docker.internal:11434/v1/chat/completions
2025-01-15T20:04:41.664Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2025-01-15T20:04:42.158Z debug: Making request to http://host.docker.internal:11434/v1/chat/completions
2025-01-15T20:04:43.124Z debug: Making request to http://host.docker.internal:11434/v1/chat/completions
2025-01-15T20:04:43.127Z warn: [OpenAIClient.chatCompletion][stream] API error
2025-01-15T20:04:43.128Z error: [handleAbortError] AI response error; aborting request: Connection error.
2025-01-15T20:04:43.141Z debug: [AskController] Request closed
2025-01-15T20:05:43.738Z debug: [indexSync] Clearing sync timeouts before exiting...

[danny-avila]

This is a common issue for ollama and most apps. Please see: continuedev/continue#2358