EOS token triggered!

[HarryK4673]

Hi everyone!

I met a problem when generating texts. I set the flags to --contextsize 98304 and when I test it, I input a text with about 30k tokens and set the maximum output tokens to 600. EOS token triggered when it output 8/400 tokens.

(EOS token triggered! ID:128009)
CtxLimit:30989/98304, Amt:8/600, Init:0.12s, Process:268.39s (8.7ms/T = 115.43T/s), Generate:10.71s (1338.2ms/T = 0.75T/s), Total:279.10s (0.03T/s)

Is there any ways to modify the EOS token?

I met this problem only on Google Colab. It's fine on my local computer.

[LostRuins]

EOS means the model wishes to stop generating the current response as it believes it is complete

To override this, set EOS token ban to "Ban"

[HarryK4673]

EOS means the model wishes to stop generating the current response as it believes it is complete

To override this, set EOS token ban to "Ban"

Thanks for that, but I'm using Google Colab (using the Linux version and command to execute). I did not find any flag related to EOS when ./koboldcpp

[LostRuins]

EOS banning is set over the API by the frontend. Are you using the KoboldAI Lite frontend?