PythonPlot + PackageCompiler + IJulia: kernel does not start (IJulia not precompiling)

[tfiers]

I want a faster startup time and time-to-first-plot when opening or restarting a notebook.
(Currently about 30 15 seconds total; compared to about 3.2 seconds TTFP with a Python notebook).¹,²

Putting PythonPlot in a system image with PackageCompiler.jl works, and reduces loading and plotting time amazingly, in the terminal.

But when trying to use an IJulia kernel with that sysimg, the kernel does not start.

Error in jupyter terminal:

[I 14:32:27.364 NotebookApp] Kernel started: 31647825-a827-4c36-86f6-f715ba470675, name: julia-mysys-1.9
[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: OutOfMemoryError()
Stacktrace:
  [1] _include_from_serialized(pkg::Base.PkgId, path::String, ocachepath::String, depmods::Vector{Any})
    @ Base .\loading.jl:1001
  [2] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt128)
    @ Base .\loading.jl:1413
  [3] _require(pkg::Base.PkgId, env::String)
    @ Base .\loading.jl:1690
  [4] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base .\loading.jl:1567
  [5] macro expansion
    @ .\loading.jl:1555 [inlined]
  [6] macro expansion
    @ .\lock.jl:267 [inlined]
  [7] require(into::Module, mod::Symbol)
    @ Base .\loading.jl:1518
  [8] include
    @ .\Base.jl:456 [inlined]
  [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base .\loading.jl:1952
 [10] top-level scope
    @ stdin:2
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\IJulia.jl:1
in expression starting at stdin:2
ERROR: LoadError: Failed to precompile IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a] to "C:\\Users\\tfiers\\.julia\\compiled\\v1.9\\IJulia\\jl_19EC.tmp".
Stacktrace:
 [1] error(s::String)
   @ Base .\error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base .\loading.jl:2195
 [3] compilecache
   @ .\loading.jl:2068 [inlined]
 [4] _require(pkg::Base.PkgId, env::String)
   @ Base .\loading.jl:1712
 [5] _require_prelocked(uuidkey::Base.PkgId, env::String)
   @ Base .\loading.jl:1567
 [6] macro expansion
   @ .\loading.jl:1555 [inlined]
 [7] macro expansion
   @ .\lock.jl:267 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base .\loading.jl:1518
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\kernel.jl:1
[I 14:32:45.348 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ffedc8a89ae -- jl_table_assign_bp at C:/workdir/src\iddict.c:48
in expression starting at C:\Users\tfiers\.julia\packages\IJulia\6TIq1\src\IJulia.jl:36
jl_table_assign_bp at C:/workdir/src\iddict.c:48
ijl_idtable_rehash at C:/workdir/src\iddict.c:24
jl_restore_system_image_from_stream_ at C:/workdir/src\staticdata.c:3165
jl_restore_package_image_from_stream at C:/workdir/src\staticdata.c:3314
jl_restore_incremental_from_buf at C:/workdir/src\staticdata.c:3360 [inlined]
ijl_restore_package_image_from_file at C:/workdir/src\staticdata.c:3473
_include_from_serialized at .\loading.jl:1001
jfptr__include_from_serialized_26122 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke79009 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j__include_from_serialized_25818 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
_require_search_from_serialized at .\loading.jl:1413
_require at .\loading.jl:1690
_require_prelocked at .\loading.jl:1567
jfptr__require_prelocked_45234 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke79134 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j__require_prelocked_25262 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
macro expansion at .\loading.jl:1555 [inlined]
macro expansion at .\lock.jl:267 [inlined]
require at .\loading.jl:1518
jfptr_require_25249 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
call_require at C:/workdir/src\toplevel.c:466 [inlined]
eval_import_path at C:/workdir/src\toplevel.c:503
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:731
jl_eval_module_expr at C:/workdir/src\toplevel.c:203 [inlined]
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:715
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1806
_include at .\loading.jl:1866
tojlinvoke77691 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
include at .\Base.jl:456 [inlined]
include_package_for_output at .\loading.jl:1952
jfptr_include_package_for_output_34600 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:226
eval_stmt_value at C:/workdir/src\interpreter.c:177 [inlined]
eval_body at C:/workdir/src\interpreter.c:624
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:762
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:912
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1806
tojlinvoke76665 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
include_string at .\loading.jl:1816 [inlined]
exec_options at .\client.jl:305
jfptr_exec_options_46043 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
tojlinvoke77001 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
j_exec_options_42205 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
_start at .\client.jl:522
jfptr__start_42203 at C:\Users\tfiers\phd\sysimg\mysys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1874 [inlined]
true_main at C:/workdir/src\jlapi.c:573
jl_repl_entrypoint at C:/workdir/src\jlapi.c:717
mainCRTStartup at C:/workdir/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 3674 (Pool: 3657; Big: 17); GC: 0

Not expecting this to work or be worked on, just noting it down here.

versioninfo():

Julia Version 1.9.0-beta3
Commit 24204a7344 (2023-01-18 07:20 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 8 × Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 7 on 8 virtual cores
Environment:
  JULIA_EDITOR = code.cmd
  JULIA_NUM_THREADS = 7
  JULIA_PYTHONCALL_EXE = C:\Users\tfiers\mambaforge\python

Package compilation & kernel install scripts:

build.jl:

using PackageCompiler

create_sysimage(
    ["PythonPlot"],
    sysimage_path = "mysys.dll",
    precompile_execution_file = "compile_me.jl",
    # Note: not `script`.`
)

compile_me.jl:

using PythonPlot

pygui(false)
fig, ax = pyplot.subplots()
plot([1,2,1])
PythonPlot.display_figs()

# Might have to call PythonPlot.__init__() in nb

add_ipython_kernel.jl:

using IJulia

path = abspath(joinpath(@__DIR__, "mysys.dll"))

IJulia.installkernel(
    "Julia-mysys",
    "--project=@.",
    "--sysimage=$path",
)

Footnotes

1. not all of this time is due to PythonCall and PythonPlot: about 10 seconds is IJulia startup itself: https://github.com/JuliaLang/IJulia.jl/issues/1090 ↩

2. after upgrading to julia 1.9.3, from 1.9.0-beta3, I saw an impressive decrease in TTFP, from 30 s, to 15 seconds. :D. About 8.6 seconds is due to IJulia startup still. Which leaves 7.0 seconds for PythonCall & PythonPlot load and startup, and plot display. ↩