Skip to content

Commit

Permalink
Improved prompt
Browse files Browse the repository at this point in the history
  • Loading branch information
Kvadratni committed Sep 24, 2024
1 parent 874eba8 commit fa67c0c
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 3 deletions.
5 changes: 2 additions & 3 deletions src/goose/toolkit/io.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from goose.toolkit.base import Toolkit, tool
from exchange import Message


class IO(Toolkit):
Expand Down Expand Up @@ -157,6 +158,4 @@ def locate_all_on_screen(self, image: str) -> str:
# Provide any system instructions for the model
# This can be generated dynamically, and is run at startup time
def system(self) -> str:
return """**You can move the mouse, click, right-click, type text, send hotkeys, scroll,
and locate images on the screen using the tools provided by the IO toolkit.
Please narrate all the actions taken back to user.**"""
return Message.load("prompts/io.jinja").text
16 changes: 16 additions & 0 deletions src/goose/toolkit/prompts/io.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
You can move the mouse, click, right-click, type text, send keypresses, scroll,
and locate images on the screen using the tools provided by the IO toolkit.
Utilize these tools to perform actions on the screen and interact with the GUI of any application
Please use screenshot to check every step of the way.
Also tell the user every action you are going to take including the mouse coordinates you are going to move to
and keys you are planning to press
Make sure that the application you are interacting with is visible on the screen and is the focused one.
On MacOS the name of the application in the top left corner should be the name of the application you are interacting with.
On Windows the name of the application in the title bar should be the name of the application you are interacting with.
On Linux the name of the application in the title bar should be the name of the application you are interacting with.
If the application is not visible on the screen, please move it to the center of the screen.
If the application is not the focused one, please click on the application to make it the focused one.
If the application is not running, please start the application.
On macOs use Spotlight to search for the application and open it.
On Windows use the search bar to search for the application and open it.
On Linux use the application menu to search for the application and open it.

0 comments on commit fa67c0c

Please sign in to comment.