Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC + WIP: Support multiprocessing Version 1 #3392

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Conversation

seisman
Copy link
Member

@seisman seisman commented Aug 11, 2024

Description of proposed changes

This PR aims to address our oldest issue #217. See #3398 for the technical notes about GMT CLI sessions and GMT C API sessions.

Test scripts

  1. The widely-used series version:
import pygmt
fig = pygmt.Figure()
fig.basemap(region=[0, 10, 0, 10], projection="X4c/4c", frame=True)
fig.show()
  1. The parallel version:
import pygmt
import multiprocessing as mp
import os

def gmt_fun(n):
    fig = pygmt.Figure()
    fig.basemap(region=[0, 10, 0, 10], projection="X10c", frame=["a", f"WSen+t{n}"])
    fig.savefig("{}.pdf".format(n))

with mp.Pool(2) as p:
    a = p.map(gmt_fun,[x for x in range(0, 2)])
  1. The series version but explicitly calls begin/end to manage sessions.
import pygmt
from pygmt.session_management import begin, end
begin()
fig = pygmt.Figure()
fig.basemap(region=[0, 10, 0, 10], projection="X4c/4c", frame=True)
fig.show()
end()
  1. The paralle version but explicitly calls begin/end to manage sessions
import pygmt
from pygmt.session_management import begin, end
import multiprocessing as mp
import os

def gmt_fun(n):
    begin()
    fig = pygmt.Figure()
    fig.basemap(region=[0, 10, 0, 10], projection="X10c", frame=["a", f"WSen+t{n}"])
    fig.savefig("{}.pdf".format(n))
    end()

with mp.Pool(2) as p:
    a = p.map(gmt_fun,[x for x in range(0, 2)])

Fixes #217.

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
  • If adding new functionality, add an example to docstrings or tutorials.
  • Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash command is:

  • /format: automatically format and lint the code

@seisman seisman force-pushed the states/session branch 4 times, most recently from ff63d95 to 3648074 Compare August 13, 2024 14:26
@seisman seisman changed the title POC + WIP: Support multiprocessing POC + WIP: Support multiprocessing Version 1 Aug 15, 2024
@seisman seisman self-assigned this Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support multiprocessing without importing gmt in each process
1 participant