Segmentation fault on subsequent conversions #206
Comments
|
Hi @jonatanklosko, thanks for the report. Could you give this a try with a "current thread" tokio runtime? Deno isn't fully compatible with the multi-threaded tokio runtime. While not a segfault, here's a prior issue I ran into that was resolved by switching to the single threaded runtime: denoland/deno#19670 (comment) |
|
@jonmmease unfortunately the segfault still happens :( |
|
If at all useful, here's the stacktrace obtained via lldb: Stacktrace |
|
Thanks for the stack trace. I don't see the crash when running this example locally on macos, so I tried running it in a GitHub action under Ubuntu 22.04, and I still don't see the crash. See #207 and https://github.com/vega/vl-convert/actions/runs/12483749003/job/34840040408?pr=207. A helpful next step would be to get a repro of this in a GitHub actions, so let me know if you have any thoughts on what we could try here. |
|
It looks like I cannot reproduce it on GitHub Actions, I used a minimal setup: jobs:
vl-convert-rs-example2:
runs-on: ubuntu-20.04
container:
image: ubuntu:22.04
steps:
- uses: actions/checkout@v2
- run: |
apt-get update
apt-get install -y git build-essential wget curl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
. "$HOME/.cargo/env"
cd vl-convert-rs
cargo run --example conversion_sequenceI can reproduce the segfault running exactly that code on AWS x86_64 machine with Ubuntu 20.04, and I also run inside |
|
@jonatanklosko and @jonmmease, thank you for attempting to identify what's going on here and for the great packages you maintain. Since I've experienced this on CircleCI I tried to reproduce it in GitHub Actions, but I could not. I also tried with the image I first encountered this on: However, this example from @jonatanklosko also produces a segmentation fault on CircleCI with either the Here's the above GH ci config adapted for CircleCI that I used: version: 2.1
parameters:
elixir-image:
type: string
default: hexpm/elixir:1.17.3-erlang-27.0.1-debian-bookworm-20241202
ubuntu-image:
type: string
default: ubuntu:22.04
executors:
test-container:
docker:
- image: << pipeline.parameters.ubuntu-image >>
commands:
code-setup:
description: "Ensures code is checked out and basic tooling is ready"
steps:
- checkout
- run: apt-get update
- run: apt-get install -y git build-essential wget curl
jobs:
vl-convert-rs-example2:
executor: test-container
steps:
- code-setup
- run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
. "$HOME/.cargo/env"
cd vl-convert-rs
cargo run --example conversion_sequence
workflows:
test-suite:
jobs:
- vl-convert-rs-example2
I was also able to reproduce this locally on a laptop running NixOS on a x86_64 chip with the That fork also has the CircleCI config referenced above and the Let me know if anything else would be helpful! |
|
Thanks for investigating @charlesfsl, having a repro on circleci is very helpful. Though it is a bit troubling that that same Docker image yields different behavior in GitHub Actions and circleci. The first things I was planning to try, after having a repro, is to update to Deno 2. There is a PR that I haven't reviewed yet that makes this update over in #205. If you're interested in trying that branch with your repro that would be helpful. |
|
No problem, @jonmmease . I'll give that a go, probably tonight, and let you know. |
|
@jonmmease I'm seeing the same result after merging in the
Looking at #207, I see you weren't able to reproduce this on MacOS (M3). I've not tried this |
Yeah, if this crash was repro-able on MacOS arm that would be much easier for me to debug. Thanks for your help! |
|
In attempting that, I remembered that converting to SVG did work fine on Mac, but converting to png or jpg did not. I tried to adapt Thanks |
|
@charlesfsl you can try this: Detailsuse vl_convert_rs::converter::VlOpts;
use vl_convert_rs::{VlConverter, VlVersion};
#[tokio::main]
async fn main() {
let vl_spec: serde_json::Value = serde_json::from_str(
r#"
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"data": {"url": "data/movies.json"},
"mark": "circle",
"encoding": {
"x": {
"bin": {"maxbins": 10},
"field": "IMDB Rating"
},
"y": {
"bin": {"maxbins": 10},
"field": "Rotten Tomatoes Rating"
},
"size": {"aggregate": "count"}
}
} "#,
)
.unwrap();
println!("{}", convert(vl_spec.clone()).await.len());
println!("{}", convert(vl_spec.clone()).await.len())
}
async fn convert(vl_spec: serde_json::Value) -> Vec<u8> {
let mut converter = VlConverter::new();
converter
.vegalite_to_png(
vl_spec,
VlOpts {
vl_version: VlVersion::v5_8,
..Default::default()
},
None,
None
)
.await
.expect("Failed to perform Vega-Lite to Vega conversion")
}I couldn't reproduce on mac, but maybe you will have more luck :) |
Thanks @jonatanklosko! Unfortunately I was unable to reproduce the segfault on mac with this test, called with |
|
png export works by converting to SVG first, and then converting from svg to PNG with the pure rust resvg library. So it sounds like the original error inside deno/v8 isn't happening on MacOS. Thanks for giving it a try! I'll try to repro on circleci soon. |
|
I've reproduced the seg fault on circle ci, and I've tried a bunch of things but haven't made any progress on mitigating the issue. I have noticed that this only happens when the Deno worker is created within a thread, which we do because the worker is not As an immediate workaround you can store a single My only current idea for a fix is to do this internally, which would mean that the Deno worker would never be dropped after it is instantiated. |
Hello, thank you for this fantastic package!
The following modified example from
vl-convert-rsresults in a segfault:Note a separate converter created for each conversion. If we change it to reuse the same converter it no longer segfaults.
I can reproduce the segfault on x86_64 Ubuntu Linux, running the example directly against vl-convert-rs main.
The text was updated successfully, but these errors were encountered: