What should I expect?

[silvestron]

I'd only use this with 13b 4bit quantized local LLMs. Would it get anything done with these kind of models?

[Josh-XT]

It depends on what you're trying to do. Basic instructions with only the appropriate available commands is most ideal.

[silvestron]

Thank you, I think I'd put it in a docker container and let it do whatever it wants. I haven't tried it yet only because I've done enough troubleshooting trying other LLM tools. I was hoping to find some video demonstration on youtube but I haven't had any luck.