Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-115999: Add free-threaded specialization for SEND. #127426

Merged
merged 3 commits into from
Dec 3, 2024

Conversation

nascheme
Copy link
Member

@nascheme nascheme commented Nov 29, 2024

No additional thread safety changes are required. Note that sending to a generator that is shared between threads is not safe in the free-threaded build.

No additional thread safety changes are required.  Note that sending to
a generator that is shared between threads is not safe in the
free-threaded build.
@ZeroIntensity
Copy link
Member

Related: GH-126371 makes native generators thread safe for free-threading. Once this gets merged, I can finish that one up and add any needed locking for the specialization instruction.

@nascheme
Copy link
Member Author

nascheme commented Nov 30, 2024

I didn't test too much but applying GH-126371 on top of this PR makes the TestRaces.test_racing_send_gen test run without generating TSAN warnings. Without it, there are warnings and CPython sometimes segfaults. So, perhaps no additional changes will be needed to the specialized opcodes to make it safe.

Copy link
Member

@corona10 corona10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I try to organize unitest codes for specialized opcodes with more understandable function name :)

See:

def binary_subscr_dict():
for _ in range(100):
a = {1: 2, 2: 3}
self.assertEqual(a[1], 2)
self.assertEqual(a[2], 3)
binary_subscr_dict()
self.assert_specialized(binary_subscr_dict, "BINARY_SUBSCR_DICT")
self.assert_no_opcode(binary_subscr_dict, "BINARY_SUBSCR")
def binary_subscr_str_int():
for _ in range(100):
a = "foobar"
for idx, expected in enumerate(a):
self.assertEqual(a[idx], expected)
binary_subscr_str_int()
self.assert_specialized(binary_subscr_str_int, "BINARY_SUBSCR_STR_INT")
self.assert_no_opcode(binary_subscr_str_int, "BINARY_SUBSCR")

Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Lib/test/test_opcache.py Outdated Show resolved Hide resolved
Copy link
Contributor

@mpage mpage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I left a small suggestion inline to make the implementation of _Py_Specialize_Send a little more concise, but feel free to ignore it if you prefer the current code structure.

I've kicked off benchmark runs for both the default and free-threaded builds for this change. I'll post the results once they're available, likely tomorrow morning.

Python/specialize.c Outdated Show resolved Hide resolved
Python/specialize.c Outdated Show resolved Hide resolved
@mpage
Copy link
Contributor

mpage commented Dec 3, 2024

Performance looks neutral on both the default and free-threaded builds. I think it's still worth merging this to stay consistent with the default build.

@nascheme nascheme merged commit 276cd66 into python:main Dec 3, 2024
55 checks passed
@nascheme nascheme deleted the gh-115999-specialize-send branch December 3, 2024 18:25
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
…h-127426)

No additional thread safety changes are required.  Note that sending to
a generator that is shared between threads is currently not safe in the
free-threaded build.
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…h-127426)

No additional thread safety changes are required.  Note that sending to
a generator that is shared between threads is currently not safe in the
free-threaded build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants