update/fix maximum token count, support 'chatgpt-4o-latest' #565
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
其实可以将最大令牌数量这个部分删去，不规定最大令牌数量是最好的，否则每出一个模型都要更新 |
|
可是，确实有限制诶，不按模型上限打请求会报错的。我也是有用户报错了我才开始维护这个列表的 |
我的意思是，其实不传max_tokens这个参数就行了。maxtoken限制的输出token数，不传就默认模型最大窗口。 |
|
没maxtoken的话没法算最多能带多少上下文进去吧？ |
上下文长度与128k的输入窗口挂钩，maxtoken是限制输出窗口，所以上下文与maxtoken没有关系的。只有输入超128k了才会报错，输出不可能超，因此不传maxtoken不会有任何问题。所以只需要options.maxModelTokens = 128000这是上下文，不需要options.maxResponseTokens = 16384这是指输出限制（max_tokens＝16384） |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
and remove some deprecated models