Skip to content

Commit

Permalink
Fix worker proc hanging at exception (#172)
Browse files Browse the repository at this point in the history
Not all exceptions in Task.run will lead to logging_thread.stop(),
which is required that the proc does not hang at exit,
because logging_thread was not a daemon thread.

We make it a daemon thread now.
And also, we additionally make extra sure that logging_thread.stop()
is called.
(Although there are maybe still other rare cases where this is not effective,
so the daemon thread property is still important as well.)

Fix #171.
  • Loading branch information
albertz authored Jan 10, 2024
1 parent a887596 commit 5556b5b
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions sisyphus/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def __init__(self, job, task, task_id):
self.task = task
self.task_id = task_id
self.start_time = None
super().__init__()
super().__init__(daemon=True)
self.out_of_memory = False
self._cond = Condition()
self.__stop = False
Expand Down Expand Up @@ -248,5 +248,8 @@ def worker_helper(args):
if hasattr(task._job, "_sis_environment") and task._job._sis_environment:
task._job._sis_environment.modify_environment()

# run task
task.run(task_id, resume_job, logging_thread=logging_thread)
try:
# run task
task.run(task_id, resume_job, logging_thread=logging_thread)
finally:
logging_thread.stop()

0 comments on commit 5556b5b

Please sign in to comment.