ERROR dspy.evaluate.evaluate: Error for example in dev set:

[fdzr]

Hi,

I'm getting this error when I use MIPROv2 optimizer:

ERROR dspy.evaluate.evaluate: Error for example in dev set: 'list' object has no attribute 'items'. Set provide_traceback=True to see the stack trace.

I created my training set with the following code:

...

for _, row in new_data.iterrows():
        training_set.append(
            dspy.Example(
                sentence1=row["context_x"],
                sentence2=row["context_y"],
                target_word=row["lemma"],
                answer=1 if row["judgment"] >= 3 else 0,
            ).with_inputs("sentence1", "sentence2", "target_word")
        )

and my module was created the following way:"

class Score(dspy.Signature):
    """Classify sentence pairs using WiC style evaluation,
    where 1 means same meaning and 0 meaning completely different
    considering a common target word in both sentences"""

    sentence1: str = dspy.InputField()
    sentence2: str = dspy.InputField()
    target_word: str = dspy.InputField()
    answer: Union[Literal["1", "0"], Literal[1, 0]] = dspy.OutputField()
    

class Wrapper(dspy.Module):
    def __init__(self):
        super().__init__()

        self.generate_answer = dspy.ChainOfThought(Score)

    def forward(self, sentence1, sentence2, target_word):
        pred = self.generate_answer(
            sentence1=sentence1,
            sentence2=sentence2,
            target_word=target_word,
        )
        return pred

I need help community! Thanks!

[Liu-Eroteme]

are you using assertions in the module you're optimizing? mipro doesn't like those.

try putting your compile call in a with dspy.context(bypass_suggest=True, bypass_assert=True) block

(haven't tested it yet, thought of it on my way home, but should work actually)