gh-128691: Use deferred reference counting on `_thread._local` #128693

colesbury · 2025-01-09T20:42:14Z

This change, along with the LOAD_ATTR specializations, makes the "thread_local_read" micro benchmark in Tools/ftscalingbench/ftscalingbench.py scale well to multiple threads.

Issue: threading.local() access should scale well from multiple threads #128691

This change, along with the LOAD_ATTR specializations, makes the "thread_local_read" micro benchmark in Tools/ftscalingbench/ftscalingbench.py scale well to multiple threads.

Modules/_threadmodule.c

bedevere-bot · 2025-01-10T01:27:25Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit c141748.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/345/builds/9906) and take a look at the build logs.
Check if the failure is related to this commit (c141748) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/345/builds/9906

Failed tests:

test.test_multiprocessing_spawn.test_misc

Failed subtests:

test_large_pool - test.test_multiprocessing_forkserver.test_misc.MiscTestCase.test_large_pool
test_large_pool - test.test_multiprocessing_spawn.test_misc.MiscTestCase.test_large_pool

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from multiprocessing.forkserver import main; main(10, 11, ['__main__'], **{'sys_path': ['/root/buildarea/3.x.angelico-debian-amd64/build/target/lib/python314.zip', '/root/buildarea/3.x.angelico-debian-amd64/build/Lib', '/root/buildarea/3.x.angelico-debian-amd64/build/build/lib.linux-x86_64-3.14'], 'authkey_r': 13})
                                                 ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/forkserver.py", line 324, in main
    pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/build/test_python_2151385æ/@test_2151385_tmpæ", line 4, in <module>
    with multiprocessing.Pool(200) as p:
         ~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
                context=self.get_context())
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 215, in __init__
    self._repopulate_pool()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 306, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
                                        self._processes,
                                        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
                                        self._maxtasksperchild,
                                        ^^^^^^^^^^^^^^^^^^^^^^^
                                        self._wrap_exception)
                                        ^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 329, in _repopulate_pool_static
    w.start()
    ~~~~~~~^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ~~~~~~~~~~~^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/context.py", line 300, in _Popen
    return Popen(process_obj)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
    ~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_forkserver.py", line 59, in _launch
    self.pid = forkserver.read_signed(self.sentinel)
               ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/forkserver.py", line 390, in read_signed
    raise EOFError('unexpected EOF')
EOFError: unexpected EOF
---


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 6607, in test_large_pool
    rc, out, err = script_helper.assert_python_ok(testfn)
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 182, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 167, in _assert_python
    res.fail(cmd_line)
    ~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 80, in fail
    raise AssertionError(f"Process return code is {exitcode}\n"
    ...<10 lines>...
                         f"---")
AssertionError: Process return code is 1
command line: ['/root/buildarea/3.x.angelico-debian-amd64/build/python', '-X', 'faulthandler', '-I', '@test_2151385_tmpæ']


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 6607, in test_large_pool
    rc, out, err = script_helper.assert_python_ok(testfn)
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 182, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 167, in _assert_python
    res.fail(cmd_line)
    ~~~~~~~~^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/script_helper.py", line 80, in fail
    raise AssertionError(f"Process return code is {exitcode}\n"
    ...<10 lines>...
                         f"---")
AssertionError: Process return code is 1
command line: ['/root/buildarea/3.x.angelico-debian-amd64/build/python', '-X', 'faulthandler', '-I', '@test_2126709_tmpæ']


Traceback (most recent call last):
  File "<string>", line 1, in <module>
    from multiprocessing.forkserver import main; main(10, 11, ['__main__'], **{'sys_path': ['/root/buildarea/3.x.angelico-debian-amd64/build/target/lib/python314.zip', '/root/buildarea/3.x.angelico-debian-amd64/build/Lib', '/root/buildarea/3.x.angelico-debian-amd64/build/build/lib.linux-x86_64-3.14'], 'authkey_r': 13})
                                                 ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/forkserver.py", line 324, in main
    pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/build/test_python_2126709æ/@test_2126709_tmpæ", line 4, in <module>
    with multiprocessing.Pool(200) as p:
         ~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
                context=self.get_context())
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 215, in __init__
    self._repopulate_pool()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 306, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
                                        self._processes,
                                        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
                                        self._maxtasksperchild,
                                        ^^^^^^^^^^^^^^^^^^^^^^^
                                        self._wrap_exception)
                                        ^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/pool.py", line 329, in _repopulate_pool_static
    w.start()
    ~~~~~~~^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ~~~~~~~~~~~^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/context.py", line 300, in _Popen
    return Popen(process_obj)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_forkserver.py", line 35, in __init__
    super().__init__(process_obj)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
    ~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/popen_forkserver.py", line 59, in _launch
    self.pid = forkserver.read_signed(self.sentinel)
               ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/forkserver.py", line 390, in read_signed
    raise EOFError('unexpected EOF')
EOFError: unexpected EOF
---

encukou · 2025-01-10T10:10:08Z

The buildbot failed again with the same error -- OSError: [Errno 12] Cannot allocate memory.

colesbury · 2025-01-10T15:15:38Z

This change doesn't affect the default build. The added call is a no-op in that case. I'm pretty sure the buildbot is coincidentally in a bad state. Maybe too many zombie processes from previous runs or something like that.

hugovk · 2025-01-11T18:28:20Z

Perhaps a reboot of the buildbot machine might help? It's angelico-debian-amd64:

https://buildbot.python.org/#/builders/345
https://buildbot.python.org/#/workers/8

cc @Rosuav

…_local` (python#128693)" This reverts commit c141748.

hugovk · 2025-01-12T20:06:21Z

I made a draft revert PR (#128753) just to check the buildbots, and the "AMD64 Debian root" that failed here, passed there.

Reminder that the next 3.14 release is due on Tuesday: https://peps.python.org/pep-0745/

colesbury · 2025-01-12T20:09:34Z

And it also passed on main: https://buildbot.python.org/#/builders/345/builds/9920

hugovk · 2025-01-12T20:10:17Z

... but the newest build on main also passed :)

https://buildbot.python.org/#/builders/345

https://buildbot.python.org/#/builders/345/builds/9920

hugovk · 2025-01-12T20:10:39Z

Yes, all looks good now :)

pythongh-128691: Use deferred reference counting on _thread._local

a8c2e44

This change, along with the LOAD_ATTR specializations, makes the "thread_local_read" micro benchmark in Tools/ftscalingbench/ftscalingbench.py scale well to multiple threads.

colesbury requested a review from mpage January 9, 2025 20:42

bedevere-app bot mentioned this pull request Jan 9, 2025

threading.local() access should scale well from multiple threads #128691

Closed

bedevere-app bot added the awaiting core review label Jan 9, 2025

colesbury added the skip news label Jan 9, 2025

mpage reviewed Jan 9, 2025

View reviewed changes

Modules/_threadmodule.c Outdated Show resolved Hide resolved

Move _PyObject_SetDeferredRefcount call up

a0bcabc

colesbury requested a review from mpage January 10, 2025 00:02

mpage approved these changes Jan 10, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Jan 10, 2025

Merge branch 'main' into pythongh-128691-thread-local-deferred

2a328af

colesbury enabled auto-merge (squash) January 10, 2025 00:45

colesbury merged commit c141748 into python:main Jan 10, 2025
39 checks passed

bedevere-app bot removed the awaiting merge label Jan 10, 2025

colesbury deleted the gh-128691-thread-local-deferred branch January 10, 2025 15:15

hugovk added a commit to hugovk/cpython that referenced this pull request Jan 12, 2025

Revert "pythongh-128691: Use deferred reference counting on `_thread.…

df78b24

…_local` (python#128693)" This reverts commit c141748.

hugovk added a commit to hugovk/cpython that referenced this pull request Jan 12, 2025

Revert "pythongh-128691: Use deferred reference counting on `_thread.…

26de5be

…_local` (python#128693)" This reverts commit c141748.

hugovk mentioned this pull request Jan 12, 2025

Revert "gh-128691: Use deferred reference counting on _thread._local (#128693)" #128753

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-128691: Use deferred reference counting on `_thread._local` #128693

gh-128691: Use deferred reference counting on `_thread._local` #128693

colesbury commented Jan 9, 2025 •

edited by bedevere-app bot

Loading

bedevere-bot commented Jan 10, 2025

encukou commented Jan 10, 2025

colesbury commented Jan 10, 2025

hugovk commented Jan 11, 2025

hugovk commented Jan 12, 2025 •

edited

Loading

colesbury commented Jan 12, 2025

hugovk commented Jan 12, 2025

hugovk commented Jan 12, 2025

gh-128691: Use deferred reference counting on _thread._local #128693

gh-128691: Use deferred reference counting on _thread._local #128693

Conversation

colesbury commented Jan 9, 2025 • edited by bedevere-app bot Loading

bedevere-bot commented Jan 10, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

encukou commented Jan 10, 2025

colesbury commented Jan 10, 2025

hugovk commented Jan 11, 2025

hugovk commented Jan 12, 2025 • edited Loading

colesbury commented Jan 12, 2025

hugovk commented Jan 12, 2025

hugovk commented Jan 12, 2025

gh-128691: Use deferred reference counting on `_thread._local` #128693

gh-128691: Use deferred reference counting on `_thread._local` #128693

colesbury commented Jan 9, 2025 •

edited by bedevere-app bot

Loading

hugovk commented Jan 12, 2025 •

edited

Loading