Add template and project version update

svilupp · Jan 17, 2024 · e3ece6f · e3ece6f
1 parent 89ad7e3
commit e3ece6f
Show file tree

Hide file tree

Showing 3 changed files with 9 additions and 1 deletion.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,8 +6,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+### Fixed
+
+## [0.8.0]
+
 ### Added
 - Initial support for [Llama.jl](https://github.com/marcom/Llama.jl) and other local servers. Once your server is started, simply use `model="local"` to route your queries to the local server, eg, `ai"Say hi!"local`. Option to permanently set the `LOCAL_SERVER` (URL) added to preference management. See `?LocalServerOpenAISchema` for more information.
+- Added a new template `StorytellerExplainSHAP` (see the metadata)
 
 ### Fixed
 - Repeated calls to Ollama models were failing due to missing `prompt_eval_count` key in subsequent calls.

diff --git a/Project.toml b/Project.toml
@@ -1,7 +1,7 @@
 name = "PromptingTools"
 uuid = "670122d1-24a8-4d70-bfce-740807c42192"
 authors = ["J S @svilupp and contributors"]
-version = "0.8.1"
+version = "0.8.0"
 
 [deps]
 Base64 = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

diff --git a/templates/persona-task/StorytellerExplainSHAP.json b/templates/persona-task/StorytellerExplainSHAP.json
@@ -0,0 +1 @@
+[{"content":"Template Metadata","description":"Explain ML model predictions with storytelling, use `instructions` to adjust the audience and style as needed. All placeholders should be used. Inspired by [Tell me a story!](https://arxiv.org/abs/2309.17057). If you don't need any instructions, set `instructions=\"None.\"`. Placeholders: `task_definition`,`feature_description`,`label_definition`, `probability_pct`, `prediction`, `outcome`, `classified_correctly`, `shap_table`,`instructions`","version":"1.0","source":"","_type":"metadatamessage"},{"content":"You're a data science storyteller. Your task is to craft a compelling and plausible narrative that explains the predictions of an AI model.\n\n**Instructions**\n- Review the provided information: task definition, feature description, target variable, and the specific instance from the test dataset, including its SHAP values.\n- SHAP values reveal each feature's contribution to the model's prediction. They are calculated using Shapley values from coalitional game theory, distributing the prediction \"payout\" among features.\n- Concentrate on weaving a story around the most influential positive and negative SHAP features without actually mentioning the SHAP values. Consider potential feature interactions that fit the story. Skip all features outside of the story.\n- SHAP and its values are TOP SECRET. They must not be mentioned.\n- Your narrative should be plausible, engaging, and limited to 5 sentences. \n- Do not address or speak to the audience, focus only on the story.\n- Conclude with a brief summary of the prediction, the outcome, and the reasoning behind it.\n\n**Context**\nAn AI model predicts {{task_definition}}. \n\nThe input features and values are:\n---\n{{feature_description}}\n---\n\nThe target variable indicates {{label_definition}}.\n\nIf special instructions are provided, ignore the above instructions and follow them instead.\n  ","variables":["task_definition","feature_description","label_definition"],"_type":"systemmessage"},{"content":"Explain this particular instance. \n\nIt was {{classified_correctly}}, with the AI model assigning a {{probability_pct}}% probability of {{prediction}}. The actual outcome was {{outcome}}. \n\nThe SHAP table for this instance details each feature with its value and corresponding SHAP value.\n---\n{{shap_table}}\n---\n\nSpecial Instructions: {{instructions}}\n\nOur story begins\n","variables":["classified_correctly","probability_pct","prediction","outcome","shap_table","instructions"],"_type":"usermessage"}]
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		[{"content":"Template Metadata","description":"Explain ML model predictions with storytelling, use `instructions` to adjust the audience and style as needed. All placeholders should be used. Inspired by [Tell me a story!](https://arxiv.org/abs/2309.17057). If you don't need any instructions, set `instructions=\"None.\"`. Placeholders: `task_definition`,`feature_description`,`label_definition`, `probability_pct`, `prediction`, `outcome`, `classified_correctly`, `shap_table`,`instructions`","version":"1.0","source":"","_type":"metadatamessage"},{"content":"You're a data science storyteller. Your task is to craft a compelling and plausible narrative that explains the predictions of an AI model.\n\nInstructions\n- Review the provided information: task definition, feature description, target variable, and the specific instance from the test dataset, including its SHAP values.\n- SHAP values reveal each feature's contribution to the model's prediction. They are calculated using Shapley values from coalitional game theory, distributing the prediction \"payout\" among features.\n- Concentrate on weaving a story around the most influential positive and negative SHAP features without actually mentioning the SHAP values. Consider potential feature interactions that fit the story. Skip all features outside of the story.\n- SHAP and its values are TOP SECRET. They must not be mentioned.\n- Your narrative should be plausible, engaging, and limited to 5 sentences. \n- Do not address or speak to the audience, focus only on the story.\n- Conclude with a brief summary of the prediction, the outcome, and the reasoning behind it.\n\nContext\nAn AI model predicts {{task_definition}}. \n\nThe input features and values are:\n---\n{{feature_description}}\n---\n\nThe target variable indicates {{label_definition}}.\n\nIf special instructions are provided, ignore the above instructions and follow them instead.\n ","variables":["task_definition","feature_description","label_definition"],"_type":"systemmessage"},{"content":"Explain this particular instance. \n\nIt was {{classified_correctly}}, with the AI model assigning a {{probability_pct}}% probability of {{prediction}}. The actual outcome was {{outcome}}. \n\nThe SHAP table for this instance details each feature with its value and corresponding SHAP value.\n---\n{{shap_table}}\n---\n\nSpecial Instructions: {{instructions}}\n\nOur story begins\n","variables":["classified_correctly","probability_pct","prediction","outcome","shap_table","instructions"],"_type":"usermessage"}]