Add type hints for args and kwargs

[gentlegiantJGC]

Description

The stubs generated by pybind11 do not contain any type hints for *args or **kwargs which makes static type checkers like mypy sad.

This pull request adds templated classes py::Args and py::KWArgs which can be used in place of py::args and py::kwargs to add custom type hints for *args and **kwargs as described in PEP 484

Examples

#include <pybind11/pybind11.h>

void init_module(py::module m){
    m.def("test", [](py::args args, py::kwargs kwargs){});
}

This will generate stub files that look like this

def test(*args, **kwargs) -> None: ...

If this is used with mypy it will generate this error.
error: Function is missing a type annotation for one or more arguments

The py::Args and py::KWArgs classes can be used instead to add type hints in the same vein as pybind11 does typing.

#include <pybind11/pybind11.h>

void init_module(py::module m){
    m.def("test", [](py::Args<int> args, py::KWArgs<float> kwargs){});
}

This generates the following stub.

def test(*args: int, **kwargs: float) -> None: ...

This can also be used with anything pybind11 supports including custom classes added through pybind11.

Alternatives

The only alternative solution I am currently aware of is to manually write the definition in a doc like this.

#include <pybind11/pybind11.h>

void init_module(py::module m){
    py::options options;
    options.disable_function_signatures();
    m.def("test", [](py::args args, py::kwargs kwargs){},
    py::doc("test(*args: int, **kwargs: float) -> None")
    );
    options.enable_function_signatures();
}

Suggested changelog entry:

Added `py::Args` and `py::KWArgs` to enable custom type hinting of `*args` and `**kwargs` (see PEP 484).

[gentlegiantJGC]

The tests are failing because I haven't updated them.
If this is something you would consider merging I can update them.

[rwgk]

which makes static type checkers like mypy sad.

Do you have "before" and "after" this PR examples? Could you please add to the PR description?

Could you describe in more detail what's better after this PR?

[gentlegiantJGC]

which makes static type checkers like mypy sad.

Do you have "before" and "after" this PR examples? Could you please add to the PR description?

Could you describe in more detail what's better after this PR?

I have updated the post.

[Skylion007]

typing Any isn't the right typing for this anyway, typing_extensions has specific typing args for kwargs and args params, doesn't it?

[rwgk]

Sorry I somehow missed your reply last week.

Quoting from the updated PR description:

---

Currently this will generate stub files that look like this

def test(*args, **kwargs) -> None: ...

If this is used with mypy it will generate this error.

error: Function is missing a type annotation for one or more arguments

---

The generated stub looks good to me, but I agree the mypy error is annoying.

What's the ideal desired behavior, assuming we can change both pybind11 and mypy?

My mind is going to: The generated stub is exactly what I'd want to see, we just need a way to express that that's intentional here, so that mypy does not complain.

[gentlegiantJGC]

typing_extensions has specific typing args for kwargs and args params, doesn't it?

From what I can see it has ParamSpecArgs and ParamSpecKwargs but from the typing docs They are intended for runtime introspection and have no special meaning to static type checkers. I can't see anything else.

typing Any isn't the right typing for this anyway

What type hint would you suggest? I tried typing it as object but that lead to a compile error on your CI so I assumed that was wrong.

Here is the section in PEP 484 about typing hinting *args and **kwargs
https://peps.python.org/pep-0484/#arbitrary-argument-lists-and-default-argument-values

def test(*args, **kwargs) -> None: ...

The type hints here are implicitly Any but mypy doesn't like implicit type hints hence the error. The form I converted it to is the explicit form.

A workaround could be to add # type: ignore to the end of the function definition to suppress the mypy error but this is kind of janky and I don't know what side effects it may have if you use *args or **kwargs as part of a larger function definition.

[gentlegiantJGC]

I have created #5381 which includes only the part allowing subclasses of args and kwargs.

Hopefully this will be easier to merge and allow me to implement a workaround until you can find a solution for this.

[gentlegiantJGC]

A less intrusive workaround would be to leave the type hints for py::args and py::kwargs as they were and add type hints for the subclasses. That way type hints can be opt-in.
This does mean that mypy would error for the normal variants but that issue can be worked out at a later date.
Once #5396 is solved I will look into this.

[gentlegiantJGC]

@rwgk this is ready for review again.
I have removed the type hint from py::args and py::kwargs so that type hints are opt-in rather than forcing it onto existing codebases.

[rwgk]

I finally found a few minutes to look.

Could you please help me understand?

From the PR description:

The changes proposed will generate stub files that look like this which resolves the mypy error.

def test(*args: typing.Any, **kwargs: typing.Any) -> None: ...

Is this still relevant? (If not, could you please update the PR description?)

Also from the PR description:

def test(*args: int, **kwargs: float) -> None: ...

What do the int and float type hints mean here? What are type checkers meant to do here? Enforce that all positional arguments have type int, and all keyword arguments have type float?

[gentlegiantJGC]

Is this still relevant?

I have removed this.

What do the int and float type hints mean here? What are type checkers meant to do here? Enforce that all positional arguments have type int, and all keyword arguments have type float?

Yes. I have edited the OP to include a link to the PEP where this is defined.

[rwgk]

Yes. I have edited the OP to include a link to the PEP where this is defined.

I see you also added this as a comment, perfect, thanks!