Main thread panic during Result::unwrap() due to InvalidFile on both Data Only and Normal Blueprints

[Matthew-Beckett]

Hi,

I am attempting to dump an assortment of UAsset files as I've tried numerous Blueprints both data-only and normal blueprints however when using the commandline-tool I am unable to list-imports or dump without a panic.

Full stack trace can be found below:

RUST_BACKTRACE=full /home/mbeckett/.cargo/bin/uasset dump BP_XXXXXCharacter.uasset 
BP_XXXXXCharacter.uasset:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidFile', /home/mbeckett/.cargo/registry/src/github.com-1ecc6299db9ec823/uasset-0.3.0/src/main.rs:64:54
stack backtrace:
   0:     0x5595752389dc - std::backtrace_rs::backtrace::libunwind::trace::h11dc6469a6e52543
                               at /rustc/1.59.0/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x5595752389dc - std::backtrace_rs::backtrace::trace_unsynchronized::hd4036938d0c3ae29
                               at /rustc/1.59.0/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x5595752389dc - std::sys_common::backtrace::_print_fmt::hf909b4c4e1107d0b
                               at /rustc/1.59.0/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x5595752389dc - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h4becf6587ad7aa1d
                               at /rustc/1.59.0/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x55957525704c - core::fmt::write::h557b7f443eab4320
                               at /rustc/1.59.0/library/core/src/fmt/mod.rs:1168:17
   5:     0x559575235075 - std::io::Write::write_fmt::h489ea24feb4a5ea4
                               at /rustc/1.59.0/library/std/src/io/mod.rs:1660:15
   6:     0x55957523ace0 - std::sys_common::backtrace::_print::h2e67187980d82d1e
                               at /rustc/1.59.0/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x55957523ace0 - std::sys_common::backtrace::print::ha1439b6972e85b57
                               at /rustc/1.59.0/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x55957523ace0 - std::panicking::default_hook::{{closure}}::hfa0b382729055e14
                               at /rustc/1.59.0/library/std/src/panicking.rs:211:50
   9:     0x55957523a899 - std::panicking::default_hook::h41128bfd08e307a0
                               at /rustc/1.59.0/library/std/src/panicking.rs:228:9
  10:     0x55957523b316 - std::panicking::rust_panic_with_hook::h9e4813ea689fc43b
                               at /rustc/1.59.0/library/std/src/panicking.rs:606:17
  11:     0x55957523b070 - std::panicking::begin_panic_handler::{{closure}}::h522a2eaaa50f1034
                               at /rustc/1.59.0/library/std/src/panicking.rs:502:13
  12:     0x559575238e74 - std::sys_common::backtrace::__rust_end_short_backtrace::h9cefd2b43c0486be
                               at /rustc/1.59.0/library/std/src/sys_common/backtrace.rs:139:18
  13:     0x55957523add9 - rust_begin_unwind
                               at /rustc/1.59.0/library/std/src/panicking.rs:498:5
  14:     0x5595751c0ca1 - core::panicking::panic_fmt::h8ef5ad8e70ca9bc8
                               at /rustc/1.59.0/library/core/src/panicking.rs:116:14
  15:     0x5595751c0d33 - core::result::unwrap_failed::hcfa710d57fe88221
                               at /rustc/1.59.0/library/core/src/result.rs:1690:5
  16:     0x5595751cf4f5 - uasset::main::hee63379db04d2faf
  17:     0x5595751c9f83 - std::sys_common::backtrace::__rust_begin_short_backtrace::h4c51e26936d8eaec
  18:     0x5595751d0919 - std::rt::lang_start::{{closure}}::h6197b27eaee6bedf
  19:     0x5595752381de - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::hd1208b862f1fb9fd
                               at /rustc/1.59.0/library/core/src/ops/function.rs:259:13
  20:     0x5595752381de - std::panicking::try::do_call::hcc194fcbf1b91aa1
                               at /rustc/1.59.0/library/std/src/panicking.rs:406:40
  21:     0x5595752381de - std::panicking::try::ha62166a0d570ae0e
                               at /rustc/1.59.0/library/std/src/panicking.rs:370:19
  22:     0x5595752381de - std::panic::catch_unwind::hdbb49d505e2dff37
                               at /rustc/1.59.0/library/std/src/panic.rs:133:14
  23:     0x5595752381de - std::rt::lang_start_internal::{{closure}}::he1f6f465111b6703
                               at /rustc/1.59.0/library/std/src/rt.rs:128:48
  24:     0x5595752381de - std::panicking::try::do_call::h752b2472b65dc578
                               at /rustc/1.59.0/library/std/src/panicking.rs:406:40
  25:     0x5595752381de - std::panicking::try::h5e5e1df37906fd95
                               at /rustc/1.59.0/library/std/src/panicking.rs:370:19
  26:     0x5595752381de - std::panic::catch_unwind::h74042ea207983c3e
                               at /rustc/1.59.0/library/std/src/panic.rs:133:14
  27:     0x5595752381de - std::rt::lang_start_internal::h57d0ce2765e13f39
                               at /rustc/1.59.0/library/std/src/rt.rs:128:20
  28:     0x5595751cfa62 - main
  29:     0x7f51b4317310 - __libc_start_call_main
  30:     0x7f51b43173c1 - __libc_start_main@GLIBC_2.2.5
  31:     0x5595751c13b5 - _start
  32:                0x0 - <unknown>

I have installed version 3.0 via cargo and am on full patched Manjaro Linux.

[jorgenpt]

@Matthew-Beckett, thanks for the report! I'll try to investigate this today.

In the meantime, maybe you'll be able to answer a few questions to direct me:

• What Unreal version are you creating these assets with?
• Does the issue reproduce on other OSes than Linux?
• Are you able to share an asset that reproduces this issue?

[Matthew-Beckett]

@Matthew-Beckett, thanks for the report! I'll try to investigate this today.

In the meantime, maybe you'll be able to answer a few questions to direct me:

• What Unreal version are you creating these assets with?

4.26

• Does the issue reproduce on other OSes than Linux?

I will try this on Windows and report back.

• Are you able to share an asset that reproduces this issue?

Will look to cleanse an asset of sensitive information to be able to do this but cannot promise.

[jorgenpt]

Ok! GitHub runs some uasset tests against 4.26 assets on Linux as part of the CI, so it should be a supported case. Does cargo test work for you?

[Matthew-Beckett]

Ok! GitHub runs some uasset tests against 4.26 assets on Linux as part of the CI, so it should be a supported case. Does cargo test work for you?

Yes, cargo test passes all cases successfully. I will look into providing an example uasset file which is not working.

[jorgenpt]

I will look into providing an example uasset file which is not working.

Thanks, if it's easier to share it privately than publicly, you can email me at [email protected].

[Matthew-Beckett]

Hi Jørgen,

I have now tested this on Windows and I can dump the files successfully with no issues, so I am not sure what's happening there, definitely needs further investigation.

On another note, can you provide some information on the kind of information stored in a dump, I've tried to identify what some of the key-value pairs are by making changes and diff'ing dumps, however, I am still struggling.

It may be helpful to advise my objective, I am trying to diff the changes made in a blueprint and hopefully generate an output of changes. Hope you can help shine some light on what exactly is included in the dump output!

Thanks for your help and swift replies!
Matt ❤️

[jorgenpt]

I have now tested this on Windows and I can dump the files successfully with no issues, so I am not sure what's happening there, definitely needs further investigation.

Okay, in that case it'd be helpful to see a reproducing asset to help me track down what's going on on Linux.

On another note, can you provide some information on the kind of information stored in a dump, I've tried to identify what some of the key-value pairs are by making changes and diff'ing dumps, however, I am still struggling.

So far, uasset-rs has mostly focused on the shared header information common to all assets, which means that it's mostly about engine versions, imports & exports (i.e. dependency graphs), etc. My first goal with it was to be able to reason about dependencies to help ensure that people don't miss certain assets during check-in, reason about which assets need to be cooked, etc.

It may be helpful to advise my objective, I am trying to diff the changes made in a blueprint and hopefully generate an output of changes. Hope you can help shine some light on what exactly is included in the dump output!

I definitely can understand this motivation, and I'd be interested in investigating some more in-depth parsing of blueprints specifically, i.e. trying to dig into the specifics of the blueprint graph etc, though that would require some consistent time focusing on it, and I'm not sure when I'll be able to pursue that particular goal. I'd be happy to support you if that was something you'd like to expand upon, but I don't know that it's something I'll be able to implement in the foreseeable future.

[jorgenpt]

@Matthew-Beckett Were you able to figure out what about the asset made it crash on Linux? The specific issue should only occur if the "magic bytes" at the start of the file aren't valid, which might mean there's some kind of endianness issue or file corruption. Was it an x86 CPU or some other architecture?