Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Locally ran models support #71

Open
appatalks opened this issue Dec 24, 2024 · 3 comments
Open

Locally ran models support #71

appatalks opened this issue Dec 24, 2024 · 3 comments

Comments

@appatalks
Copy link
Owner

Going to start playing with locally ran model support.

LMStudio seems like a good beginners path forward - https://lmstudio.ai/

@appatalks
Copy link
Owner Author

OpenAI API like ability

Sweeet, should make this easy to drop in place

@appatalks
Copy link
Owner Author

Needs some polish
See #72

@appatalks
Copy link
Owner Author

  • Tested a bunch of models
  • Liking granite-3.1-8b-instruct as a good overall local model.
    • But a little happy to serve up images - need to play with the prompt, and rework Google Images

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant