Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_free_threading.test_monitoring.SetProfileMultiThreaded.test_instrumentation crashes intermittently #128806

Open
mpage opened this issue Jan 14, 2025 · 0 comments
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) tests Tests in the Lib/test dir topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@mpage
Copy link
Contributor

mpage commented Jan 14, 2025

Crash report

What happened?

test_free_threading.test_monitoring.SetProfileMultiThreaded.test_instrumentation crashes intermittently. I am able to reproduce the crash if I run test_free_threading forever, but haven't been able to further minimize the reproducer. It typically takes around 50 iterations before crashing.

This assertion is failing:

assert(debug_check_sanity(tstate->interp, code));

This also happens intermittently on buildbots (ex).

Reproducer

./python -m test test_free_threading -F

Python stack trace:

python: Python/instrumentation.c:1249: _Py_call_instrumentation_line: Assertion `debug_check_sanity(tstate->interp, code)' failed.
python: Python/instrumentation.c:1249: _Py_call_instrumentation_line: Assertion `debug_check_sanity(tstate->interp, code)' failed.
Fatal Python error: Aborted

<Cannot show all threads while the GIL is disabled>
Stack (most recent call first):
  File "python: Python/instrumentation.c:1249: _Py_call_instrumentation_line: Assertion `debug_check_sanity(tstate->interp, code)' failed.
/data/users/mpage/cpython/Lib/unittest/case.py

Native stack trace:

(gdb) bt
#0  0x00007f923008bacc in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007f923003e686 in raise () from /lib64/libc.so.6
#2  0x00007f9230028833 in abort () from /lib64/libc.so.6
#3  0x00007f923002875b in __assert_fail_base.cold () from /lib64/libc.so.6
#4  0x00007f92300373c6 in __assert_fail () from /lib64/libc.so.6
#5  0x00000000006e571e in _Py_call_instrumentation_line (tstate=tstate@entry=0xb54ae0, frame=frame@entry=0x7f922fe5f7f8, instr=instr@entry=0x20006051bd6, prev=0x20006051bea)
    at Python/instrumentation.c:1332
#6  0x000000000066e4b5 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0xb54ae0, frame=0x7f922fe5f7f8, frame@entry=0x7f922fe5f1c8, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:4742
#7  0x00000000006827be in _PyEval_EvalFrame (throwflag=0, frame=0x7f922fe5f1c8, tstate=0xb54ae0) at ./Include/internal/pycore_ceval.h:116
#8  _PyEval_Vector (tstate=<optimized out>, func=<optimized out>, locals=locals@entry=0x0, args=0x7f91f4bff800, argcount=3, kwnames=<optimized out>) at Python/ceval.c:1913
#9  0x00000000004b5497 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:413
#10 0x00000000004b9636 in _PyObject_VectorcallTstate (tstate=0xb54ae0, callable=<function at remote 0x2001e1b02b0>, args=0x7f91f4bff800, nargsf=3, kwnames=0x0)
    at ./Include/internal/pycore_call.h:167
#11 0x00000000004ba02d in method_vectorcall (method=<optimized out>, args=0x2000135f8d8, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:94
#12 0x00000000004b7c52 in _PyVectorcall_Call (tstate=tstate@entry=0xb54ae0, func=0x4b9f19 <method_vectorcall>, callable=callable@entry=<method at remote 0x2000135a1b0>,
    tuple=tuple@entry=(12, [<function at remote 0x2001e1b25b0>, <function at remote 0x2001e1b2310>, <function at remote 0x2001e1b0fd0>, <function at remote 0x2001e1be7f0>, <function at remote 0x2001e1b3110>, <function at remote 0x2001e1bae30>, <function at remote 0x2001e1b41b0>, <function at remote 0x2001e1b32d0>, <function at remote 0x2001e1be710>, <function at remote 0x2001e1b7010>, <function at remote 0x2001e1b7a90>, <function at remote 0x2001e1b1eb0>, <function at remote 0x2001e1b4990>, <function at remote 0x2001e1b9930>, <function at remote 0x2001e1b6130>, <function at remote 0x2001e1b4ed0>, <function at remote 0x2001e1b69f0>, <function at remote 0x2001e1b2f50>, <function at remote 0x2001e1b80b0>, <function at remote 0x2001e1b4610>, <function at remote 0x2001e1b7d30>, <function at remote 0x2001e1b87b0>, <function at remote 0x2001e1b1350>, <function at remote 0x2001e1b5a30>, <function at remote 0x2001e1b1510>, <function at remote 0x2001e1b0710>, <function at remote 0x2001e1b5170>, <function at remote 0x2001e1b2770>, <function ...(truncated), kwargs=kwargs@entry={}) at ./Include/object.h:783
#13 0x00000000004b7fbe in _PyObject_Call (tstate=0xb54ae0, callable=callable@entry=<method at remote 0x2000135a1b0>,
    args=(12, [<function at remote 0x2001e1b25b0>, <function at remote 0x2001e1b2310>, <function at remote 0x2001e1b0fd0>, <function at remote 0x2001e1be7f0>, <function at remote 0x2001e1b3110>, <function at remote 0x2001e1bae30>, <function at remote 0x2001e1b41b0>, <function at remote 0x2001e1b32d0>, <function at remote 0x2001e1be710>, <function at remote 0x2001e1b7010>, <function at remote 0x2001e1b7a90>, <function at remote 0x2001e1b1eb0>, <function at remote 0x2001e1b4990>, <function at remote 0x2001e1b9930>, <function at remote 0x2001e1b6130>, <function at remote 0x2001e1b4ed0>, <function at remote 0x2001e1b69f0>, <function at remote 0x2001e1b2f50>, <function at remote 0x2001e1b80b0>, <function at remote 0x2001e1b4610>, <function at remote 0x2001e1b7d30>, <function at remote 0x2001e1b87b0>, <function at remote 0x2001e1b1350>, <function at remote 0x2001e1b5a30>, <function at remote 0x2001e1b1510>, <function at remote 0x2001e1b0710>, <function at remote 0x2001e1b5170>, <function at remote 0x2001e1b2770>, <function ...(truncated), kwargs={}) at Objects/call.c:348
#14 0x00000000004b8015 in PyObject_Call (callable=callable@entry=<method at remote 0x2000135a1b0>, args=<optimized out>, kwargs=<optimized out>) at Objects/call.c:373
#15 0x000000000065ede4 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0xb54ae0, frame=0x7f922fe5f138, frame@entry=0x7f922fe5f020, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:1778
#16 0x00000000006827be in _PyEval_EvalFrame (throwflag=0, frame=0x7f922fe5f020, tstate=0xb54ae0) at ./Include/internal/pycore_ceval.h:116
#17 _PyEval_Vector (tstate=<optimized out>, func=<optimized out>, locals=locals@entry=0x0, args=0x7f91f4bffd08, argcount=1, kwnames=<optimized out>) at Python/ceval.c:1913
#18 0x00000000004b5497 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:413
#19 0x00000000004ba1ab in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7f91f4bffd08, callable=<function at remote 0x20000a5a810>, tstate=0xb54ae0)
    at ./Include/internal/pycore_call.h:167
#20 method_vectorcall (method=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=0x0) at Objects/classobject.c:72
#21 0x00000000004b7c52 in _PyVectorcall_Call (tstate=tstate@entry=0xb54ae0, func=0x4b9f19 <method_vectorcall>, callable=callable@entry=<method at remote 0x2000135e5f0>, tuple=tuple@entry=(),
    kwargs=kwargs@entry=0x0) at ./Include/object.h:783
#22 0x00000000004b7fbe in _PyObject_Call (tstate=0xb54ae0, callable=<method at remote 0x2000135e5f0>, args=(), kwargs=0x0) at Objects/call.c:348
#23 0x00000000004b8015 in PyObject_Call (callable=<optimized out>, args=<optimized out>, kwargs=<optimized out>) at Objects/call.c:373
#24 0x00000000007b91bf in thread_run (boot_raw=boot_raw@entry=0xb4a2f0) at ./Modules/_threadmodule.c:346
#25 0x00000000007281ef in pythread_wrapper (arg=<optimized out>) at Python/thread_pthread.h:242
#26 0x00007f9230089d82 in start_thread () from /lib64/libc.so.6
#27 0x00007f923010ee20 in clone3 () from /lib64/libc.so.6
(gdb)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a3+ experimental free-threading build (heads/main:5e65a1acc0b, Jan 12 2025, 08:39:41) [GCC 11.5.0 20240719 (Red Hat 11.5.0-2)]

@mpage mpage added tests Tests in the Lib/test dir interpreter-core (Objects, Python, Grammar, and Parser dirs) type-crash A hard crash of the interpreter, possibly with a core dump topic-free-threading labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
interpreter-core (Objects, Python, Grammar, and Parser dirs) tests Tests in the Lib/test dir topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Development

No branches or pull requests

1 participant