Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifying arguments on the fly is not supported #431

Open
FloridSleeves opened this issue Jan 23, 2024 · 0 comments
Open

Modifying arguments on the fly is not supported #431

FloridSleeves opened this issue Jan 23, 2024 · 0 comments

Comments

@FloridSleeves
Copy link

FloridSleeves commented Jan 23, 2024

Hi,

I'm a new user and I'm really impressed with your library. I recently encountered an issue while using your library and attempting to modify function arguments on the fly using the after function. Here's the problem I encountered:

In my code, I'm trying to change the argument messages if its length exceeds the maximum allowed length. Here's a snippet of the code I'm using:

def my_after():
    def change_messages(retry_state) -> None:
        if retry_state.outcome.failed:
            messages = retry_state.args[1]
            model = retry_state.args[0]
            messages = messages[-3:]
            new_args = list(retry_state.args)
            new_args[1] = messages
            retry_state.args = new_args

    return change_messages

@retry(wait=wait_random_exponential(min=1, max=180), stop=stop_after_attempt(6), after=my_after())
def gpt_chat(
    model: str,
    messages: List[Message],
) -> Union[List[str], str]:
    try:
        print("MESSAGE LEN:", len(messages))
        response = client.chat.completions.create(
            model=model,
            messages=[dataclasses.asdict(message) for message in messages],
        )
    except Exception as e:
        if "context_length_exceeded" in str(e):
            raise Exception
        else:
            assert False, "GPT API error: " + str(e)

    if num_comps == 1:
        return response.choices[0].message.content
    return [choice.message.content for choice in response.choices]

However, it appears that this modification is not working as expected. After debugging into the library, I discovered that the problem lies in this line of code within the library itself:

result = fn(*args, **kwargs)
.

It seems that the library does not use the args and kwargs from retry_state, but instead continues to use the original arguments, which causes any modifications made to the retry state arguments to have no effect. This could be fixed with result = fn(*retry_state.args, **retry_state.kwargs).

I'm reaching out to clarify whether this is a usage error on my part or if it's a bug within the library itself. I'd be more than willing to help with a fix if it's indeed a bug.

Thank you for your assistance and for creating such a valuable library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant