[jinja] GPT4All 3.6.1 GPT4All-Community/Llama3-DiscoLeo-Instruct-8B-32k-v0.1 chat template issues (ready for substitution)

[ThiloteE]

Meta-issue: #3340

Bug Report

Model does not work out of the box

Steps to Reproduce

1. Download the gguf
2. sideload it in GPT4All-Chat
3. start chatting

Expected Behavior

Model works out of the box.

Your Environment

• GPT4All version: 3.6.1
• Operating System: Windows 10
• Link to gguf of model: https://huggingface.co/GPT4All-Community/Llama3-DiscoLeo-Instruct-8B-32k-v0.1-GGUF

Jinja template out of the box in GPT4All-Chat 3.6.1 (does not work)

{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>

'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>

' }}{% endif %}

Jinja template made by ThiloteE (works)

{%- set loop_messages = messages %}
{%- for message in loop_messages %}
    {%- set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}
        {%- if loop.index0 == 0 %}
            {%- set content = bos_token + content %}
        {%- endif %}
    {{- content }}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
{%- endif %}

Btw. this is a good system prompt for German, but I don't know if the model was finetuned on it:

"Die folgende Anweisung gibt einen Text vor und fordert Sie auf, eine angemessene Erwiderung zu formulieren. Bitte geben Sie Ihre Antwort."