Skip to content

Commit

Permalink
Fix worker proc hanging at exception
Browse files Browse the repository at this point in the history
Not all exceptions in Task.run will lead to logging_thread.stop(),
which is required that the proc does not hang at exit,
because logging_thread was not a daemon thread.

We make it a daemon thread now.
And also, we additionally make extra sure that logging_thread.stop()
is called.
(Although there are maybe still other rare cases where this is not effective,
so the daemon thread property is still important as well.)

Fix #171.
  • Loading branch information
albertz committed Jan 10, 2024
1 parent a887596 commit afb6c0a
Showing 1 changed file with 6 additions and 3 deletions.
9 changes: 6 additions & 3 deletions sisyphus/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def __init__(self, job, task, task_id):
self.task = task
self.task_id = task_id
self.start_time = None
super().__init__()
super().__init__(daemon=True)
self.out_of_memory = False
self._cond = Condition()
self.__stop = False
Expand Down Expand Up @@ -248,5 +248,8 @@ def worker_helper(args):
if hasattr(task._job, "_sis_environment") and task._job._sis_environment:
task._job._sis_environment.modify_environment()

# run task
task.run(task_id, resume_job, logging_thread=logging_thread)
try:
# run task
task.run(task_id, resume_job, logging_thread=logging_thread)
finally:
logging_thread.stop()

0 comments on commit afb6c0a

Please sign in to comment.