Pipeline frozen on post processing after RunCellpose with Docker

[emiglietta]

Version of Cellprofiler: 4.2.6 (source installation)

After running a pipeline (on Analysis mode) containing two RunCellpose modules running with Docker, the pipeline gets stuck on post processing. This has happened to me sometimes even when the pipeline runs with no errors, but the only way of consistently reproducing this error was to run the pipeline with more workers than CPUs available and induce the RunCellpose module to fail (the easiest way is to decrease the total memory available for Docker) and, when it does, choose continue with analysis, skipping the failing images.

[emiglietta]

I managed to get this error consistently without it failing at any point (except for the freezing during 'post-processing'). When I analyzed just a few image sets (up to 20), it never froze. However, when I tried analyzing more sets (as low as 50), then it consistently froze during post-processing.

[bethac07]

@emiglietta have you checked how, if at all, running with "automatically continue after failing images" turned on affects the outcome? Might be a potentially diagnostic clue (and a good workaround, if it resolves the issue temporarily)

[emiglietta]

Thanks for the suggestion @bethac07!
I tried running the pipeline with the "always skip failing images" option ON, but the result was the same: the pipeline got stuck in "post-processing" after trying to analyze 50 image sets.