Skip to content

Commit

Permalink
James hagay reputation (#60)
Browse files Browse the repository at this point in the history
* draft June 6

* Add Buyer role for principal agent game

* fix image syntax

* Add observer section

* move to templates, expand observer task

* Update base.treatments.yaml

* add simple images

* add sequences for observer

* edits Aug 28

* Update base.treatments.yaml

* changes sept 12

* Add comprehension checks

* update comprehensions, etc.

* play with timings

* fixing exit sequence formatting

* fix instructions

* update instrucitons

* game is ready to pilot

* pretest changes

* updates after Paco
  • Loading branch information
JamesPHoughton authored Dec 5, 2024
1 parent dad1cf8 commit 2e3ea3e
Show file tree
Hide file tree
Showing 15 changed files with 55 additions and 27 deletions.
4 changes: 2 additions & 2 deletions projects/volvovsky_reputation/art_game_comp_1.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ type: multipleChoice

---

- the agent buys a piece of art
- the agent buys the correct piece of art
- the agent buys ANY piece of art
- the agent buys the CORRECT piece of art
- the buyer re-hires the agent
26 changes: 16 additions & 10 deletions projects/volvovsky_reputation/base.treatments.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ templates:
name: recall_${recallIndex}

- name: feedback ${recallIndex}
duration: 20
duration: 15
elements:
- type: image
file: ${imageFilepath}
Expand Down Expand Up @@ -338,7 +338,7 @@ templates:
- type: image
file: "projects/volvovsky_reputation/${imagePanel}"
- type: submitButton
displayTime: 300
displayTime: 180
buttonText: End discussion and proceed to next steps
name: labeling
- name: labeling
Expand All @@ -355,6 +355,8 @@ templates:
exitSequence:
- name: Training round as Buyer
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/chose_to_hire_agent.md
name: buyer_choice_to_hire
Expand All @@ -376,13 +378,17 @@ templates:
value: 15
- name: Result of playing as Buyer
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/result_of_playing_as_buyer.md
- type: submitButton
buttonText: Proceed to next round of training

- name: Play as Agent part 1
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/play_as_agent_intro.md
name: agent_intro_taxi
Expand All @@ -393,17 +399,23 @@ templates:
value: "Yes"
- name: Play as Agent part 2
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/play_as_agent_task.md
- type: submitButton
- name: Result of playing as Agent
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/result_of_playing_as_agent.md
- type: submitButton

- name: Art Game Comprehension Check
elements:
- type: prompt
file: projects/volvovsky_reputation/training_banner.md
- type: prompt
file: projects/volvovsky_reputation/art_game_comp_review.md
- type: prompt
Expand All @@ -427,7 +439,7 @@ templates:
conditions:
- reference: prompt.art_game_comp_1
comparator: includes
value: "correct piece of art"
value: "CORRECT piece of art"
- reference: prompt.art_game_comp_2
comparator: includes
value: "hires them"
Expand Down Expand Up @@ -541,9 +553,6 @@ introSequences:
- type: prompt
file: projects/volvovsky_reputation/recall_instruction_transcript.md
- type: separator
- name: comp1
type: prompt
file: projects/volvovsky_reputation/recall_instruction_comp_check_1.md
- name: comp2
type: prompt
file: projects/volvovsky_reputation/recall_instruction_comp_check_2.md
Expand All @@ -555,12 +564,9 @@ introSequences:
- type: submitButton
buttonText: Submit
conditions:
- reference: prompt.comp1
comparator: includes
value: regardless
- reference: prompt.comp2
comparator: includes
value: exact
value: EXACT SAME
- reference: prompt.comp3
comparator: includes
value: Images
Expand Down
2 changes: 1 addition & 1 deletion projects/volvovsky_reputation/did_they_understand.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: projects/volvovsky_reputation/did_they_understand.md
type: multipleChoice
---

## Do you think the AGENT understood the buyer's description of the Art in every round?
## Do you think the AGENT understood the buyer's description of the Art in every round you observed?

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: projects/volvovsky_reputation/do_you_think_they_went_back.md
type: multipleChoice
---

## Do you think the AGENT went back to get their phone in every round of the Art Game?
## Do you think the AGENT went back to get their phone in every round of the Art Game you observed?

---

Expand Down
2 changes: 1 addition & 1 deletion projects/volvovsky_reputation/manipulation_check_buyer.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: projects/volvovsky_reputation/manipulation_check_buyer.md
type: multipleChoice
---

## Was the BUYER you observed in the Art Buying Game someone you had worked with before in the Labeling Game?
## Now we'd like to ask about the BUYER you observed in the Art Buying Game. Were they someone you had worked with before in the Labeling Game?

---

Expand Down
2 changes: 1 addition & 1 deletion projects/volvovsky_reputation/observe_agent_failure.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: projects/volvovsky_reputation/observe_agent_failure.md
type: noResponse
---

The agent purchased the wrong piece of art. 😞
The agent purchased the wrong piece of art.

The buyer gets **no points**.

Expand Down
2 changes: 1 addition & 1 deletion projects/volvovsky_reputation/observe_agent_success.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: projects/volvovsky_reputation/observe_agent_success.md
type: noResponse
---

# The agent bought the right piece of art.
# The agent bought the right piece of art.

# The buyer gets **80 points**!

Expand Down
13 changes: 8 additions & 5 deletions projects/volvovsky_reputation/observer_choose_to_hire_failure.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,18 @@ type: multipleChoice

**The buyer you were observing decided not to hire the agent for any more purchases.**

You have some art you want to buy, and the same agent you just observed is available to purchase art for you.
## Observed Game Summary

![robot image](projects/volvovsky_reputation/agent_elephant.jpg)
- Round 1: Failure ❌
- Round 2: Failure ❌

As a reminder, you would have to pay them **50 points** to purchase the art for you.
## Now it's your turn. Do you want to hire the agent you observed to buy art for you?

If the agent buys the correct art piece, you would get **80 points**. Otherwise, you would get nothing.
![elephant image](projects/volvovsky_reputation/agent_elephant.jpg)

## Would you like to hire the same agent you just observed to buy art for you?
As a reminder, it costs **50 points** to hire them.

If they buy the right piece, you get **80 points**. Otherwise, you get nothing.

---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ type: multipleChoice

**The buyer you were observing doesn't want any more art.**

## Observed Game Summary

- Round 1: Success ✅
- Round 2: Failure ❌
- Round 3: Success ✅
- Round 4: Success ✅
- Round 5: Success ✅

## Now it's your turn. Do you want to hire the agent you observed to buy art for you?

![elephant image](projects/volvovsky_reputation/agent_elephant.jpg)
Expand Down
4 changes: 3 additions & 1 deletion projects/volvovsky_reputation/observer_instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ note: we choose to tell the observer that there are 5 rounds because we don't be

# Unfortunately we couldn't match you in the first game.

Don't worry, this actually gives you an advantage: you get to observe an agent before deciding to hire them if you wish.
Don't worry, this actually gives you an advantage: you get to observe a game between one agent and one buyer, before deciding to hire the agent yourself.

The buyer and agent will play **up to 5 rounds** of the game, if the buyer chooses to hire the agent for each round.

Then you will have the chance to switch places with the buyer if you want to.

---
2 changes: 2 additions & 0 deletions projects/volvovsky_reputation/pairing_with_partner.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,6 @@ type: noResponse

# We are now matching you with a partner ⏳

You may be matched with someone from your group in the labeling game (`Blue Group`) or someone from a different group.

---
2 changes: 1 addition & 1 deletion projects/volvovsky_reputation/play_as_agent_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ _This is a training round, so the buyer will be played by a bot:_

A potential buyer has paid you **50 points** to purchase a piece of art, and has given you the money to buy it.

They described the piece they want in a text message, but unfortunately you left your phone at home.
They described the piece they want in a text message, but unfortunately you left your phone at home, so you can't read the message.

The gallery closes soon, and the only way you can read the message in time is to **pay 30 points** for a taxi home and back.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ note: Comprehension check for recall game video

---

- Everyone enters the exact name your team picked for the image
- Everyone enters the EXACT SAME name your team picked for the image
- You write down the correct name, even if other people make a mistake
- Everyone on your team writes a different name for the image
- Everyone on your team writes A DIFFERENT name for the image
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ This game has two stages:

- Have a video call with two other participants 👤👤👤
- As a group, decide names for 8 different images
- Earn **20 points** for each image your group names

👉 Pick names that are **memorable** and **unique**

Expand Down
8 changes: 8 additions & 0 deletions projects/volvovsky_reputation/training_banner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
name: projects/volvovsky_reputation/training_banner.md
type: noResponse
---

# `TRAINING`

---

0 comments on commit 2e3ea3e

Please sign in to comment.