Optimize for size

[probonopd]

I've been considering to rewrite the AppImage runtime in Rust for a while. Looks like you're faster! 👍

One question: Is here a way to optimize for binary size in Rust, so that we might get well under 1 MB?

[VHSgunzo]

@probonopd
Hi! Yes, I can optimize the size a bit if I remove mksquashfs from the release (or hide it behind a separate --features mksquashfs). Then in that case the SquashFS-only runtime version (x86_64) would be less than 1 MB (~960 KB).
And the version with support for both SquashFS and DwarFS will have a size of ~6 MB

[probonopd]

Yes I think that would be a good improvement.

[VHSgunzo]

@probonopd
Added a lite version for AppImage uruntime without mksquashfs in v0.0.9

[probonopd]

Thanks for the improvement @VHSgunzo. Maybe running https://github.com/RazrFalcon/cargo-bloat could help find what could be further optimized? Again, thanks for your efforts. I think that every KB that can be saved here will pay off when multiplied with the millions of downloads this might get over time.

[VHSgunzo]

@probonopd
The main weight here is from assets, and rust part weighs only about ~290 KB
These binaries are embedded in uruntime and run from memory using memfd_exec.

This approach is chosen because there is no native library for DwarFS for rust.
And I also noticed some problems in RunImage container when using AppImage runtime with fuse and SquashFS native libraries.
The problem was that some vulkan layers didn't work (e.g. mangohud overlay didn't show up in games and vkbasalt effects didn't work) and these problems didn't occur when remounting the container image with squashfuse.

upd: I recently tried to reproduce this problem also with the latest version of the original AppImage runtime and got the same effect, the problem persisted.

[probonopd]

Do you optimize all of the embedded binaries with -Os -s?

I don't think that Vulkan issues are caused by the runtime, more like by the contents of the AppImage (probably something is missing?)

[VHSgunzo]

@probonopd
Yes, the build applies size reduction optimizations, super-strip and upx compression and also compiles static fuse and other libraries to support various compression algorithms (also with size reduction optimizations).
and also for the rust part some tricks are applied to reduce the size

I don't think that Vulkan issues are caused by the runtime, more like by the contents of the AppImage (probably something is missing?)

no, the same SquashFS image of container was used in all tests. Only when it is mounted with squashfuse, these problems do not occur.

[probonopd]

Now, that is really interesting!

[VHSgunzo]

Probably almost all possible methods of size reduction have already been applied, unless i can also rebuild dwarfs-universal and all static libraries it depends on with the same size optimizations. And i can also try to remove support for some compression algorithms, though I wouldn't really want to cut functionality in this regard.

[VHSgunzo]

At the moment, dwarfs-universal is taken from the builds of the official repository.

[Samueru-sama]

And i can also try to remove support for some compression algorithms, though I wouldn't really want to cut functionality in this regard.

For appimages the only two algos that are practical today are zstd and lz4 and I'm not very sure about lz4.

There was also this similar change in EFI Zboot dropping all other compression algos and only keeping zstd and gzip.

[VHSgunzo]

Unfortunately I noticed a big speed degradation of squashfuse built on musl libc, so I had to build it on glibc to prevent this. As a result, the uruntime size is a bit bloated.

@probonopd
I also tested AppImage type2-runtime (which is also now compiled on musl) and it too has a lower read speed than the squashfuse+glibc.

[probonopd]

Interesting observation @VHSgunzo. Do you have measurements? What kind of percentage are we talking about? I wonder whether Cosmopolitan or Chimera would perform better:

• Check out cosmopolitan AppImage/AppImageKit#1105
• https://github.com/orgs/AppImage/discussions/1376

[VHSgunzo]

@probonopd

Do you have measurements? What kind of percentage are we talking about?

the difference in my test was ~15%
squashfuse+glibc vs type2-runtime

The image contains the container rootfs of the Arch Linux from lux-wine.
The size of the image along with the runtime is ~1.97 GB, when decompressed it is 5.07 GB, zstd compression, 1 level and 128 Kb block size used.

The test consists of simply copying the contents of the mounted image to tmpfs (with pre-clearing of the page cache, dentries and inode cache).

I wonder whether Cosmopolitan or Chimera would perform better

I'd be interested in that too, but I haven't compared them yet

[xplshn]

Chimera

I don't understand why it'd be relevant, fBSD utils are more bloated compared to Busybox, Toybox or Sbase.

And Chimera is also Musl-based, just like Alpine. I don't think there will be any advantages to using a less-tested system, which is also unstable, and not prepared for containers.

Of course, this is just my opinion, as someone who's daily driven FreeBSD, Alpine Linux (until recently) and Chimera.

Actual testing should be done nonetheless

[Samueru-sama]

I don't understand why it'd be relevant, fBSD utils are more bloated compared to Busybox, Toybox or Sbase.

Because chimera uses mimalloc: https://github.com/orgs/chimera-linux/discussions/2480

[xplshn]

I don't understand why it'd be relevant, fBSD utils are more bloated compared to Busybox, Toybox or Sbase.

Because chimera uses mimalloc: https://github.com/orgs/chimera-linux/discussions/2480

You can use jemalloc/mimalloc in Alpine, @probonopd is already compiling the programs we need in an Alpine CI, he'd have to add the jemalloc/mimalloc package and make all programs link with it.

They can be statically linked or dynamically loaded