System message editor

[igo]

Hi, how can I show system message and allow editing it? (while preserving first user message in a branch)

[Yonom]

Hey,

you don't want to use a system message, but rather a system prompt. The system prompt will be sent to the server on every request.

Here's a input component that adds its contents to your system prompt:

const SystemPromptInput: React.FC = () => {
  const [prompt, setPrompt] = useState();
  const runtime = useAssistantRuntime();
  useEffect(() => {
    return runtime.registerModelConfigProvider(() => { system: prompt });
  }, [runtime, prompt]);

  const handlePromptChange = (e) => {
    setPrompt(e.target.value);
  };
  
  return (
    <input value={prompt} onChange={handlePromptChange} />
  );
};

Let me know if this works as expected

[igo]

I use useEdgeRuntime. Anyway what is registerModelConfigProvider good for? I cannot find it in documentation (btw that trieve search doesn't seem to work properly). I was thinking if it would be possible to:

• either always show some message at the beginning that would server as a system message
• or I create my custom input with system message and when I change it, runtime would allow me to get all current messages, I take first user message and create a new branch with my new system message and first user message and then send this new conversation in branch to the backend

[Yonom]

Thanks for flagging the issue with the docs search. The model config provider is still missing docs. Model config is used to set model configuration parameters such as temperature, maxTokens, system prompt, etc.

You have the choice between the system model config, and a system message.

The approach above is for the model config approach. It does not have a branching support; whatever system value is currently provided will be sent to the backend for the next request. If you want the system prompt to behave like a message (including the ability to switch to previous versions of the message, use a system message)

You also can add a system message e.g. to initialMessages. By default, system messages do not have a corresponding UI; you have to specify one to <Thread.Messages> (see decomposition guide) or <ThreadPrimitive.Messages />. In there, you can add a <ActionBarPrimitive.Edit>Edit</ActionBarPrimitive.Edit> button to enable edit mode.

<Thread.Messages components={{ SystemMessage: MySystemMessage }} />

// ....

const MySystemMessage = () => {
  return <div>
    <MessagePrimitive.Content />
    <ActionBarPrimitive.Root>
      <ActionBarPrimitive.Edit>Edit</ActionBarPrimitive.Edit>
    </ActionBarPrimitive.Root>
  </div>;
}

In your EditComposer, you can add a custom send button which basically does:

const composerRuntime = useComposerRuntime();
const threadRuntime = useThreadRuntime();

const handleSend = () => {
  const firstUserMessage = threadRuntime.getState().messages[1];
  composerRuntime.send(); // submit the system message edit (note: system message appends do not trigger a backend request by default)
  if (!firstUserMessage) return;
  threadRuntime.append({ // append the previous first user message again, note: user message appends trigger a backend request)
    content: firstUserMessage.content,
    attachments: firstUserMessage.attachments,
  });
}