[jinja] GPT4All-Chat 3.6.1 google/gemma-2-2b-it

[ThiloteE]

Meta-issue: #3340

Bug Report

Model does not work out of the box

Steps to Reproduce

1. Download the gguf, sideload it in GPT4All-Chat, start chatting

Expected Behavior

Model works out of the box.

Your Environment

• GPT4All version: 3.6.1
• Operating System: Windows 10
• Chat model used (if applicable): google/gemma-2-2b-it.
• Link to chat model: https://huggingface.co/google/gemma-2-2b-it
• Link to gguf of chat model: https://huggingface.co/mradermacher/gemma-2-2b-it-i1-GGUF

Original Jinja template in tokenizer_config.json (taken from unsloth/gemma-2-2b-it)

{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}

Jinja template out of the box in GPT4All-Chat 3.6.1, but still has EOS issues:

{{- bos_token }}
{%- if messages[0]['role'] == 'system' %}
    {{- raise_exception('System role not supported') }}
{%- endif %}
{%- for message in messages %}
    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
        {{- raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
    {%- endif %}
    {%- if message['role'] == 'assistant' %}
        {%- set role = 'model' %}
    {%- else %}
        {%- set role = message['role'] %}
    {%- endif %}
    {{- '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<start_of_turn>model\n' }}
{%- endif %}