Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix crash MacOS because of multiprocessing fork #133

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

raryelcostasouza
Copy link
Contributor

@raryelcostasouza raryelcostasouza commented Feb 11, 2019

When running on MacOS High Sierra, soon after starting the multiprocessing.pool it was crashing completely with error below:

The process has forked and you cannot use this CoreFoundation
functionality safely. You MUST exec().
Break on THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS
COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC
_() to debug.

According to documentation of multiprocessing: https://docs.python.org/3.4/library/multiprocessing.html

fork
The parent process uses os.fork() to fork the Python interpreter. The child process, when it begins, is effectively identical to the parent process. All resources of the parent are inherited by the child process. Note that safely forking a multithreaded process is problematic.
Available on Unix only. The default on Unix.

forkserver
When the program starts and selects the forkserver start method, a server process is started. From then on, whenever a new process is needed, the parent process connects to the server and requests that it fork a new process. The fork server process is single threaded so it is safe for it to use os.fork(). No unnecessary resources are inherited.

The default unix fork is not safe for threads... and MacOS doesn't seem to like that... crashing completely.

@raryelcostasouza
Copy link
Contributor Author

The automatic build checker failed because it uses python 2.7 as default, and python 2 doesn't have the set_start_method.

My patch fix the problem for python 3, but for python 2 it would need a bigger workaround (as reported at https://bugs.python.org/issue33725).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants