Segfaults in ctypes _as_parameter_ handling when called with MagicMock

[devdanzin]

Crash report

What happened?

It's possible to segfault the interpreter by calling any of the 3 functions below with MagicMock as argument. It takes a long time to trigger the crash (up to 3 minutes in my slow machine).

from unittest.mock import MagicMock
import _pyrepl._minimal_curses

obj = _pyrepl._minimal_curses.tparm(MagicMock(), 0, 0, 0, 0, 0, 0, 0)
obj = _pyrepl._minimal_curses.setupterm(MagicMock(), 0)
obj = _pyrepl._minimal_curses.tigetstr(MagicMock())

The backtrace is very long, with over 87k entries in one case. Here's part of it:

#0  0x00005555556b04e6 in compare_unicode_unicode (mp=0x0, dk=0x55556ce6f460, ep0=0x55556ce6f500, ix=38, key=0x7ffff73ae860, hash=5657245593745306375)
    at Objects/dictobject.c:1136
#1  0x00005555556af1cb in do_lookup (mp=mp@entry=0x0, dk=dk@entry=0x55556ce6f460, key=key@entry=0x7ffff73ae860, hash=hash@entry=5657245593745306375, 
    check_lookup=check_lookup@entry=0x5555556b04da <compare_unicode_unicode>) at Objects/dictobject.c:1066
#2  0x00005555556af242 in unicodekeys_lookup_unicode (dk=dk@entry=0x55556ce6f460, key=key@entry=0x7ffff73ae860, hash=hash@entry=5657245593745306375)
    at Objects/dictobject.c:1151
#3  0x00005555556b3c99 in _Py_dict_lookup (mp=0x7fffbe6a3c50, key=key@entry=0x7ffff73ae860, hash=hash@entry=5657245593745306375, 
    value_addr=value_addr@entry=0x7fffff7ff0f0) at Objects/dictobject.c:1265
#4  0x00005555556b4746 in _PyDict_GetItemRef_KnownHash (op=<optimized out>, key=key@entry=0x7ffff73ae860, hash=hash@entry=5657245593745306375, 
    result=result@entry=0x7fffff7ff130) at Objects/dictobject.c:2317
#5  0x00005555556ff874 in find_name_in_mro (type=type@entry=0x55556ce6cca0, name=name@entry=0x7ffff73ae860, error=error@entry=0x7fffff7ff194)
    at Objects/typeobject.c:5108
#6  0x00005555556ffa3c in _PyType_LookupRef (type=type@entry=0x55556ce6cca0, name=name@entry=0x7ffff73ae860) at Objects/typeobject.c:5260
#7  0x00005555556ca8b8 in _PyObject_GenericSetAttrWithDict (obj=obj@entry=0x7fffbe6b9920, name=0x7ffff73ae860, value=0x555555ae8100 <_Py_NoneStruct>, 
    dict=dict@entry=0x0) at Objects/object.c:1773
#8  0x00005555556cab5f in PyObject_GenericSetAttr (obj=obj@entry=0x7fffbe6b9920, name=<optimized out>, value=<optimized out>) at Objects/object.c:1849
#9  0x00005555556f541e in wrap_setattr (self=0x7fffbe6b9920, args=<optimized out>, wrapped=0x5555556cab4d <PyObject_GenericSetAttr>) at Objects/typeobject.c:8792
#10 0x000055555567ee41 in wrapperdescr_raw_call (descr=descr@entry=0x7ffff7b002f0, self=self@entry=0x7fffbe6b9920, args=args@entry=0x7fffc5b5bd90, 
    kwds=kwds@entry=0x0) at Objects/descrobject.c:531
#11 0x000055555567f2c5 in wrapperdescr_call (_descr=_descr@entry=0x7ffff7b002f0, args=0x7fffc5b5bd90, args@entry=0x7fffc5a48c50, kwds=kwds@entry=0x0)
    at Objects/descrobject.c:569
#12 0x0000555555672a51 in _PyObject_MakeTpCall (tstate=tstate@entry=0x555555b564c0 <_PyRuntime+299040>, callable=callable@entry=0x7ffff7b002f0, 
    args=args@entry=0x7ffff7e29850, nargs=<optimized out>, keywords=keywords@entry=0x0) at Objects/call.c:242
#13 0x0000555555672c91 in _PyObject_VectorcallTstate (tstate=0x555555b564c0 <_PyRuntime+299040>, callable=callable@entry=0x7ffff7b002f0, 
    args=args@entry=0x7ffff7e29850, nargsf=<optimized out>, kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:166
#14 0x0000555555672ce7 in PyObject_Vectorcall (callable=callable@entry=0x7ffff7b002f0, args=args@entry=0x7ffff7e29850, nargsf=<optimized out>, 
    kwnames=kwnames@entry=0x0) at Objects/call.c:327
#15 0x00005555557a293c in _PyEval_EvalFrameDefault (tstate=0x555555b564c0 <_PyRuntime+299040>, frame=0x7ffff7e297c8, throwflag=0) at Python/generated_cases.c.h:1839
#16 0x00005555557b0957 in _PyEval_EvalFrame (tstate=tstate@entry=0x555555b564c0 <_PyRuntime+299040>, frame=<optimized out>, throwflag=throwflag@entry=0)
    at ./Include/internal/pycore_ceval.h:119
#17 0x00005555557b0a76 in _PyEval_Vector (tstate=0x555555b564c0 <_PyRuntime+299040>, func=0x7ffff6a893d0, locals=locals@entry=0x0, args=0x7fffff7ff650, argcount=3, 
    kwnames=0x0) at Python/ceval.c:1807
#18 0x00005555556728a2 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:413
#19 0x00005555556f4c21 in _PyObject_VectorcallTstate (tstate=0x555555b564c0 <_PyRuntime+299040>, callable=0x7ffff6a893d0, args=0x7fffff7ff650, nargsf=3, 
    kwnames=kwnames@entry=0x0) at ./Include/internal/pycore_call.h:168
#20 0x00005555556f4cd8 in vectorcall_unbound (tstate=<optimized out>, unbound=<optimized out>, func=<optimized out>, args=<optimized out>, nargs=<optimized out>)
    at Objects/typeobject.c:2566
#21 0x0000555555700c61 in vectorcall_method (name=<optimized out>, args=args@entry=0x7fffff7ff650, nargs=nargs@entry=3) at Objects/typeobject.c:2597

I realize these functions are implemented with ctypes and internal to an internal package and hence this issue can be of very low importance. I just report in case they can point to some interesting related bug.

Found using fusil by @vstinner.

CPython versions tested on:

3.13, 3.14, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.1+ (heads/3.13:d51c1444e36, Dec 12 2024, 11:22:09) [GCC 11.4.0]

Linked PRs

• gh-127870: Fix ctypes _as_parameter_ handling #127872
• [3.13] gh-127870: Detect recursive calls in ctypes _as_parameter_ handling (GH-127872) #127917
• [3.12] gh-127870: Detect recursive calls in ctypes _as_parameter_ handling (#127872) #127918

[vstinner]

ctypes c_char_p_from_param_impl() runs into a loop when it gets a MagicMock:

static PyObject *
c_char_p_from_param_impl(PyObject *type, PyTypeObject *cls, PyObject *value)
{
    ...
    if (PyObject_GetOptionalAttr(value, &_Py_ID(_as_parameter_), &as_parameter) < 0) {
        return NULL;
    }
    if (as_parameter) {
        value = c_char_p_from_param_impl(type, cls, as_parameter);
        Py_DECREF(as_parameter);
        return value;
    }
    PyErr_Format(PyExc_TypeError,
                 "'%.200s' object cannot be interpreted "
                 "as ctypes.c_char_p", Py_TYPE(value)->tp_name);
    return NULL;
}

Getting magic_mock._as_parameter creates a new MagicMock object and c_char_p_from_param_impl() is called on it. Then the same thing happens again, until we reach a stack overflow.

Simpler reproducer:

import ctypes.util
from unittest.mock import MagicMock

libc = ctypes.cdll.LoadLibrary("libc.so.6")
libc.strlen.argtypes = [ctypes.c_char_p]
libc.strlen.restype = ctypes.c_size_t

fake_str = MagicMock()
obj = libc.strlen(fake_str)

[vstinner]

I wrote #127872 to fix this issue.

[vstinner]

Fixed, thanks for your bug report.