-
Notifications
You must be signed in to change notification settings - Fork 29
/
Copy pathprompts.toml
46 lines (34 loc) · 2.33 KB
/
prompts.toml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
[eval]
prompt = """
Given an instruction and two responses—one generated by a human and the other by a language model—I'm seeking to evaluate how closely the language model's response mirrors the human-generated one. Additionally, I want to assess the accuracy and relevance of the language model's response to the original instruction.
Instruction:
```
$instruction
```
Human Response:
```
$human_response
```
Language Model Response:
```
$lm_response
```
You are quality assessor who analyzes the similarity between the Human Response and the Language Model Response on a scale of 1 to 100, where 1 indicates no similarity and 100 indicates identical responses.
Also you analyze the Language Model Response how it accurately answers the given Instruction on a scale of 1 to 100. Analysis MUST be rigorous and thorough.
Provide the assessment in the following JSON format:
{
"similarity_assessment": {"score": [Insert similarity score here],"reason": [Insert how the similarity score is determined]},
"precision_assessment": {"score": [Insert precision score here],"reason": [Insert how the precision score is determined]}
}
"""
[synth_data_gen]
prompt = """
Generate a series of (instruction, response) pairs that are similar in context and structure to the example provided below. Each pair should consist of a concise instruction followed by an appropriate, detailed response. The instruction should pose a clear task or question, while the response should provide a comprehensive answer or solution that could be understood by someone with a basic understanding of the subject.
Example pair:
Instruction: $instruction
Response: $response
Your task is to generate more pairs that maintain this level of clarity and detail. The topic is $topic. Ensure that the responses are informative and accurate, suitable for an educational context.
Store the generated pairs in JSON format, with each pair as an object within an array. Each object should have two key-value pairs: "instruction" and "response". For instance:
{"contents": [{"instruction": "text", "response": "text"}, {"instruction": "text", "response": "text"}, ...]}
Remember to maintain consistency in the format and ensure the generated pairs are diverse and cover a broad range of subjects. You must return the response in the asked format and you must not add any additional text in your response.
"""