Make message receive and handling async #1140
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
All messages from the front-end/server are received and handled synchronously, including custom comm messages (
comm_open
,comm_msg
, andcomm_close
). So, any currently executing cell blocks the IJulia kernel from receiving and handling any IOPub/comm messages. For example, in the following WebIO MWE, a JS function updates an "output"Observable
, and the JS function is triggered by setting an ("input") observable:you can't observe a new
s["out"]
value (aka the result of the JS function) during execution of the same cell that sets["in"]
(which triggers the JS function).Example Julia function that fails (hangs) without async comms
*This example function isn't thread-safe. (The
scp["in"]
observable isn't locked, so concurrently setting it could lead to interleaved/mismatched updates to thescp["out"]
observable.)One example of an actual use-case/benefit is
PlotlyJS.to_image
, which uses the sameJulia => JS => Julia observable setup to retrieve the results of a plotly.js function call.
Currently, the
PlotlyJS.to_image
function soft-fails because the observable that holds the generatedimage is only updated after the current cell finishes execution (when IJulia can process the
comm_msg
from WebIO in the Jupyter frontend/client).Testing
I've manually tested that the above WebIO MWE works with this PR, and that interrupting still works. I realize this is a fairly fundamental rearchitecturing of the message receiving/handling, but I'm not sure what else to test and/or if there is a good way to test any of this in CI. I'm open to any hints/pointers if you want more thorough testing/test cases.
Fixes #858.
P.S. Breadcrumb for the future: This new architecture has a lot of parallels (easily adapted) to the new subshells feature that was recently implemented in ipython/ipykernel#1249.