Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Developer documentation on how to profile for performance #764

Open
berombau opened this issue Nov 6, 2024 · 3 comments · May be fixed by #781
Open

Developer documentation on how to profile for performance #764

berombau opened this issue Nov 6, 2024 · 3 comments · May be fixed by #781

Comments

@berombau
Copy link
Contributor

berombau commented Nov 6, 2024

Is your feature request related to a problem? Please describe.
In the contributing guide, there is currently no section on benchmarking or optimizing code.

Describe the solution you'd like
The contributing guide should be extended with information on how to profile locally, writing a performance bug report and how to profile a new PR for performance.

Describe alternatives you've considered
There is a lot of existing material we can use or link to:

Additional context
Having a regression testing framework is linked to this, but out-of-scope.

@berombau berombau moved this to 🏗️ In Progress in Basel Hackathon Nov 2024 Nov 12, 2024
@berombau
Copy link
Contributor Author

@Czaki suggested adding to Scientific Python instead of creating new docs for SpatialData.

A plan now would be adding a paragraph in https://spatialdata.scverse.org/en/stable/contributing.html: linking to https://lectures.scientific-python.org/advanced/debugging/index.html, some links from above and https://napari.org/stable/developers/contributing/performance/index.html.

@berombau berombau linked a pull request Nov 13, 2024 that will close this issue
@berombau berombau moved this from 🏗️ In Progress to 👀 In review in Basel Hackathon Nov 2024 Nov 13, 2024
@Czaki
Copy link
Contributor

Czaki commented Nov 13, 2024

Also Profiling section of this classes https://gitlab.mimuw.edu.pl/python-tools/debugger

From scientific python, the https://lectures.scientific-python.org/advanced/optimizing/index.html#profiling-python-code fits better.

The main question is if spatialdata should have own document about performance, or we should try to convince wider community (ex. scientific python) to have one place to store all generic information about performance monitoring in one place.

There are 3 main profilers vmprof, yappi, cProfile.

There is memray from Bloomberg to trace memory https://github.com/bloomberg/memray

@berombau
Copy link
Contributor Author

The PR has the two links to the Scientific Python pages, both debugging and profiling. I think this is good enough for the SpatialData docs for now. We can consider adding a very specific profiling example with SpatialData later maybe.

For profiling in general, we can consider extending the lecture-style docs at https://lectures.scientific-python.org/advanced with your debugging examples and updating the tooling section.

We can also add an extra developer guide of profiling at https://github.com/scientific-python/cookie/tree/main/docs (although this is somewhat linked to their cookie-cutter and sp-repo-review project)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 👀 In review
Development

Successfully merging a pull request may close this issue.

2 participants