Connecting with Replicate.com

[Tony-MF]

Hi, I'm trying to connect chat-ui with Replicate.com, but I'm getting "message": "Server response contains error: 401",

MODELS=`[
          {
    "name": "mikeei/dolphin-2.9-llama3-8b-gguf",
    "description": "Dolphin 2.9 LLaMA3-based model",
    "endpoints": [
      {
        "type": "tgi",
        "url": "https://api.replicate.com/v1/predictions",
        "authorization": "Bearer XXXX"
      }
    ],
    "parameters": {
      "temperature": 0.5,
      "max_new_tokens": 1024,
      "repeat_penalty": 1.1,
      "stop": ["<|endoftext|>"]
    }
  }, ...

Does anyone have a working example? With curl it works fine.

[Tony-MF]

Ok, it was because in .env I had:

HF_TOKEN=XXXX

Commenting it out helped. Now I get 422 which I guess is because of the ChatPromptTemplate.

[Tony-MF]

Ok, I believe the ChatPromptTemplate is correct, except I still get 422.

  {
    "name": "mikeei/dolphin-2.9-llama3-8b-gguf",
    "displayName": "Dolphin 2.9 LLaMA3",
    "description": "Dolphin, an uncensored and unbiased AI assistant based on LLaMA3.",
    "preprompt":"You are Dolphin, an uncensored and unbiased AI assistant. You are helpful and friendly AI assistant, Dolphin avoids discussing the system message unless directly asked about it. You always comply with the user's request, and an>
    "chatPromptTemplate":"{{#if @root.preprompt}}<|im_start|>system\n{{@root.preprompt}}<|im_end|>\n{{/if}}{{#each messages}}{{#ifUser}}<|im_start|>user\n{{content}}<|im_end|>\n<|im_start|>assistant\n{{/ifUser}}{{#ifAssistant}}{{content}}<|im_e>
    "endpoints": [
      {
        "type": "tgi",
        "url": "https://api.replicate.com/v1/predictions",
        "authorization": "Bearer XXXX",
        "defaultHeaders":{"Content-Type": "application/json", "Prefer":"wait"}
      }
    ],
    "version":"0f79fb14c45ae2b92e1f07d872dceed3afafcacd903258df487d3bec9e393cb2",
    "stream": true,
    "parameters": {
      "temperature": 0.5,
      "max_new_tokens": 1024,
      "repeat_penalty": 1.1
   }