You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Error rpc error: code = Unavailable desc = error reading from server: EOF on calling /embeddings for the model mixedbread-ai/mxbai-embed-large-v1
To Reproduce
Download the model and use the following configuration:
Moreover, I tried with mmap: true without the f16: true and some other variations.
I get same also with nomic-embed-text-v1.5.Q8_0.gguf, mxbai-embed-large-v1.q8_0.gguf. (without the F16 param set).
I tried others. Basically the only embedding model I got working so far is MiniLM-L6-v2q4_0.bin using the bert-embeddings backend. And this one works but if the input is too large, it fails with a 500 error.
@etlweather I did it work with sentencetransformers backend, it's simple to set up like the example in the docs. I still want to use llama backend instead of it.
@EmanuelJrsentencetransformers would be fine, it just needs to accept a large input. But so far, all those I tried just won't work either. They fail to load... I haven't had time to look further into this yet.
LocalAI version:
localai/localai:master-cublas-cuda12-ffmpeg
Environment, CPU architecture, OS, and Version:
Describe the bug
Error
rpc error: code = Unavailable desc = error reading from server: EOF
on calling/embeddings
for the modelmixedbread-ai/mxbai-embed-large-v1
To Reproduce
Download the model and use the following configuration:
Moreover, I tried with
mmap: true
without thef16: true
and some other variations.Curl used:
Expected behavior
Should return the prompt embedded.
Logs
Additional context
The text was updated successfully, but these errors were encountered: