Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request]: 如果使用CPU的话，要求的CPU核数和内存大概是多大，运行4B的模型。 #254

Open
zaojiahua opened this issue Oct 14, 2024 · 1 comment
Open

[Feature Request]: 如果使用CPU的话，要求的CPU核数和内存大概是多大，运行4B的模型。 #254

zaojiahua opened this issue Oct 14, 2024 · 1 comment
Labels
feature New features

Comments

@zaojiahua
Copy link

Feature request / 功能建议

如果使用CPU的话，要求的CPU核数和内存大概是多大，运行4B的模型。
@zaojiahua zaojiahua added the feature New features label Oct 14, 2024
@LDLINGLINGLING
Copy link
Collaborator

你好，这个对cpu的核心数量没什么具体要求，算力高就时间短，否则就时间长。对内存来说，4B的模型至少8G内存才能加载，加上一些上下文占用内存，推荐12g内存。当然如果你使用量化后的模型，对内存的要求会大幅度减少

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature New features
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@zaojiahua @LDLINGLINGLING