Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLIENT-3146] Add support for free threading mode in Python 3.13 #686

Draft
wants to merge 69 commits into
base: dev
Choose a base branch
from

Conversation

juliannguyen4
Copy link
Collaborator

@juliannguyen4 juliannguyen4 commented Oct 21, 2024

todo:

  • replace borrowed ref api calls
  • global state

notes

  • All header files checked for global objs. only src/include/types.h has global objs
  • All source files checked. The ones with global objs that have been addressed are:
  • geospatial/type.c
  • aerospike.c. TODO: m_size
  • exception.c
  • log.c
  • serializer.c

Not replacing PyDict_GetItem() because it will be removed in another PR

todo extra changes:

  • remove unused exception_types.h

@codecov-commenter
Copy link

codecov-commenter commented Oct 21, 2024

Codecov Report

Attention: Patch coverage is 0% with 156 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (ca2f4be) to head (5e946b5).
Report is 7 commits behind head on dev.

Files with missing lines Patch % Lines
src/main/conversions.c 0.00% 77 Missing ⚠️
src/main/client/batch_operate.c 0.00% 14 Missing ⚠️
src/main/client/operate.c 0.00% 13 Missing ⚠️
src/main/client/batch_read.c 0.00% 12 Missing ⚠️
src/main/convert_expressions.c 0.00% 10 Missing ⚠️
src/main/exception.c 0.00% 8 Missing ⚠️
src/main/client/batch_apply.c 0.00% 6 Missing ⚠️
src/main/client/batch_write.c 0.00% 3 Missing ⚠️
src/main/client/type.c 0.00% 3 Missing ⚠️
src/main/client/admin.c 0.00% 2 Missing ⚠️
... and 7 more
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #686       +/-   ##
==========================================
- Coverage   80.89%   0.00%   -80.90%     
==========================================
  Files         102      56       -46     
  Lines       15186   10580     -4606     
==========================================
- Hits        12285       0    -12285     
- Misses       2901   10580     +7679     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@justinlee-aerospike justinlee-aerospike left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some suggested changes for the thread test.

client = aerospike.client(config)
_, _, bins = client.get(key)
nonlocal bin_value_sum
lock.acquire()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't have an unnecessary lock here during the read_bin(). You can simply verify by doing something like the following:

  1. in the test phase, have your base action be to update a record so the bin value changes from x to x + 1

  2. in verification phase, sum the value of all bins you changed before and after the test. The difference should be the number of operations you submitted.

The key here is that you don't require locking during the submission of the updates

@pytest.mark.usefixtures("as_connection")
class TestFreeThreading:
def test_unsafe(self):
key = ("test", "demo", 1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not a great idea to only use one key - you should generate a bunch of keys - the number of keys / number of operations will give you the basic collision rate.

for worker in workers:
worker.join()

assert bin_value_sum == THREAD_COUNT * BIN_VALUE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also report out to a log the timing so you can calculate and report out basic throughput numbers. You can use this to sanity check to see if you get better numbers with a large transaction count, multiple threads running with and without GIL.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be good to have both performance and data race testing?

@juliannguyen4 juliannguyen4 force-pushed the CLIENT-3146-free-threading branch from d440590 to 2b37036 Compare November 21, 2024 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants