Experimental Landlock based sandboxing #597

vlaci · 2023-06-06T15:10:56Z

Implementation of #594 together with onekey-sec/unblob-native#11

Having to first create the extraction directory complicates things a lot.
I am unsure what approach we should take here but it can be seen, that the first directory needs somewhat special treatment.

Alternative integration approaches:

set LANDLOCK_ACCESS_FS_MAKE_DIR on the parent of the extraction root as an escape hatch
introduce a workdir where extraction takes place one level deeper to achieve simpler code.

qkaiser · 2023-06-06T15:47:04Z

I would be in favor of option 1 since it's how unblob works. The extraction path provided with -e is where we create the extraction directory so we do need LANDLOCK_ACCESS_FS_MAKE_DIR.

As long as we're clear about the fact that unblob limits itself to the path provided with -e and not the directory within, it's acceptable to me.

unblob/models.py

unblob/processing.py

vlaci · 2023-06-06T16:10:44Z

I would be in favor of option 1 since it's how unblob works. The extraction path provided with -e is where we create the extraction directory so we do need LANDLOCK_ACCESS_FS_MAKE_DIR.

As long as we're clear about the fact that unblob limits itself to the path provided with -e and not the directory within, it's acceptable to me.

We need this for the parent of extraction directory:

LANDLOCK_ACCESS_FS_MAKE_DIR: Create (or rename) a directory.

And this for the parent directory of report file:

LANDLOCK_ACCESS_FS_MAKE_REG: Create (or rename or link) a regular file.

The latter seems a bit much to me

vlaci · 2023-06-07T21:07:30Z

Hehe, this change is incompatible with code coverage measurement :D

 INTERNALERROR>   File "/home/runner/.cache/pypoetry/virtualenvs/unblob-PkeEArhf-py3.8/lib/python3.8/site-packages/coverage/sqldata.py", line 1064, in _connect
INTERNALERROR>     raise DataError(f"Couldn't use data file {self.filename!r}: {exc}") from exc
INTERNALERROR> coverage.exceptions.DataError: Couldn't use data file '/home/runner/work/unblob/unblob/.coverage.fv-az626-878.3677.365646': unable to open database file

qkaiser · 2024-01-08T10:08:52Z

Two things to do before tagging as ready for review:

solve the ModuleNotFoundError: No module named 'unblob_native.sandbox' error by properly exposing it
add unit testing (unsupported platforms, different access checks)

qkaiser · 2024-02-21T11:09:57Z

onekey-sec/unblob-native#11

qkaiser · 2024-02-21T11:20:09Z

Will rebase once version 0.1.2 of unblob-native is in.

qkaiser · 2024-10-08T16:54:03Z

rebased and solved conflicts. IMO this is ready to be merged once the wip commit is validated

vlaci · 2024-10-09T20:49:06Z

I had an idea to make sandboxing more composable: spawn a new thread, drop privileges, run process_file there, then return the results. This way, the caller's privileges are not affected. I've ran into a mess making the process pool cleanly terminate when receiving a SIGTERM. I'll try to make it work for some more time, if I can't, then I drop this idea.

unblob/pool.py

unblob/sandbox.py

+            self._try_enter_sandbox()
+            try:
+                result = callback(*args, **kwargs)
+            except BaseException as e:


vlaci · 2024-10-10T14:56:51Z

It's interesting that it fails on aarch64-linux. Maybe because of qemu emulation?

qkaiser · 2024-10-10T18:53:29Z

It's interesting that it fails on aarch64-linux. Maybe because of qemu emulation?

created a linux host on aarch64 (qemu) here, will do some experiments tomorrow

qkaiser · 2024-10-11T01:25:27Z

@vlaci must be related to Nix, cause the tests are passing here:

$ uname -avr
Linux debian 6.1.0-26-arm64 #1 SMP Debian 6.1.112-1 (2024-09-30) aarch64 GNU/Linux
$ poetry run pytest tests/ -k test_sandbox
==== test session starts ====
platform linux -- Python 3.11.2, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/quentin/unblob
configfile: pyproject.toml
plugins: cov-5.0.0
collected 919 items / 916 deselected / 3 selected                                                                                                                                                                                                                                                                                                                                          

tests/test_cli.py .                                                                                                                                                                                                                                                                                                                                                                  [ 33%]
tests/test_sandbox.py ..
==== 3 passed, 916 deselected in 190.78s (0:03:10) ====

aarch64 Debian running on qemu-system-aarch64

qkaiser · 2024-10-11T02:29:14Z

Even though it seems to be supported by QEMU on aarch64 since qemu/qemu@3a2f19b, landlock syscalls are actually not supported in user mode emulation.

For demo purposes, let's write this quick-and-dirty PoC:

#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <errno.h>

int main() {
    int result = syscall(SYS_landlock_create_ruleset, NULL, 0, 0);

    if (result == -1) {
        printf("System call failed: %d (errno: %d)\n", result, errno);
    } else {
        printf("System call succeeded.\n");
    }

    return 0;
}

Compile it, make sure it's aarch64, run it in user mode emulation:

$ aarch64-linux-gnu-gcc -static poc.c
$ file a.out 
a.out: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=c42a43dc682c2dc504699d7b816e99a2e32c6bf0, for GNU/Linux 3.7.0, not stripped
$ qemu-aarch64-static ./a.out        
System call failed: -1 (errno: 38)

errno 38 is -ENOSYS or "function not implemented".

So problem is on QEMU user mode emulation not doing the translation for landlock syscalls. However, it should be handled gracefully by unblob-native I think.

Note: tested with qemu version 6.2.0 and 8.2.94

qkaiser · 2024-10-11T08:20:14Z

I built the sandboxer example from rust-landlock and it works on qemu-aarch64-static by returning an error:

$ LL_FS_RO="/bin:/lib:/usr:/proc:/etc:/dev/urandom" \
LL_FS_RW="/dev/null:/dev/full:/dev/zero:/dev/pts:/tmp" \
LL_TCP_BIND="9418" LL_TCP_CONNECT="80:443" \
qemu-aarch64-static \
target/aarch64-unknown-linux-musl/debug/examples/sandboxer bash -i
Error: Landlock is not supported by the running kernel.

So the bug is probably in unblob-native or between unblob-native Rust and unblob-native Python. The exception is giving me the same feeling:

where False = isinstance(AttributeError("'int' object has no attribute 'results'"), PermissionError)

vlaci · 2024-10-11T09:35:07Z

I'll add tests on the rust side as well then :)

vlaci · 2024-10-11T10:52:01Z

I built the sandboxer example from rust-landlock and it works on qemu-aarch64-static by returning an error:
$ LL_FS_RO="/bin:/lib:/usr:/proc:/etc:/dev/urandom" \
LL_FS_RW="/dev/null:/dev/full:/dev/zero:/dev/pts:/tmp" \
LL_TCP_BIND="9418" LL_TCP_CONNECT="80:443" \
qemu-aarch64-static \
target/aarch64-unknown-linux-musl/debug/examples/sandboxer bash -i
Error: Landlock is not supported by the running kernel.
So the bug is probably in unblob-native or between unblob-native Rust and unblob-native Python. The exception is giving me the same feeling:
where False = isinstance(AttributeError("'int' object has no attribute 'results'"), PermissionError)

Ahh, we need this error check at the end...
https://github.com/landlock-lsm/rust-landlock/blob/94721d26b2fd1151e71bd7a3aa5a43c463a22347/examples/sandboxer.rs#L133-L135

l0kod · 2024-10-11T11:18:25Z

Ahh, we need this error check at the end... https://github.com/landlock-lsm/rust-landlock/blob/94721d26b2fd1151e71bd7a3aa5a43c463a22347/examples/sandboxer.rs#L133-L135

Please note that kernels not supporting Landlock should not be an error for programs sandboxing themselves, only for sandboxers that must create sandboxes or error out (like this example). It can be a warning though.

vlaci · 2024-10-11T13:30:55Z

Ahh, we need this error check at the end... https://github.com/landlock-lsm/rust-landlock/blob/94721d26b2fd1151e71bd7a3aa5a43c463a22347/examples/sandboxer.rs#L133-L135

Please note that kernels not supporting Landlock should not be an error for programs sandboxing themselves, only for sandboxers that must create sandboxes or error out (like this example). It can be a warning though.

Yep, unblob itself already just logs the problem, but I like clear failures in tests.
https://github.com/onekey-sec/unblob/pull/597/files#diff-2d4e67217491bddba5546f44b16c52ae6be29d568b52949950866ec5e4c24b0fR92-R95

vlaci · 2024-10-11T15:24:19Z

Failing build resolved-by onekey-sec/unblob-native#65

unblob/pool.py

unblob/sandbox.py

all other tests in this file assert on `process_file` being called with correct arguments. We need specific tests which test that the configuration is interpreted correctly

vlaci · 2024-10-16T16:54:38Z

The one thing requires thorough manual testing is keeping exit on CTRL-C and SIGTERM working. Many shenanigans are added for that purpose.

tests/test_cli.py

tests/test_sandbox.py

unblob/processing.py

unblob/sandbox.py

Instead of juggling with signal handlers and hoping that `ShutDownRequired` will be fired in the appropriate place in `multiprocessing.BasePool`, on exceptional termination, we signal workers via `SIGTERM`. As a side-effect this makes it possible to run `process_file` in non-main thread.

Co-authored-by: Quentin Kaiser <[email protected]>

Unblob now requires 0.1.4, so pyproject.toml version is also updated

e3krisztian · 2024-10-17T09:19:42Z

unblob/pool.py

 import sys
 import threading
 from multiprocessing.queues import JoinableQueue
-from typing import Any, Callable, Union
+from typing import Any, Callable, Set, Union


Maybe not in this MR, but Python 3.8 is EOL for 10 days now, we should probably drop support for it.
3.9 introduced support for generics on collection types.
3.10 introduced | to replace Union.

e3krisztian · 2024-10-17T09:34:27Z

unblob/pool.py

+            proc.terminate()
+
+        self._input.close()
+        if sys.version_info >= (3, 9):


(3.8 is EOL now)

e3krisztian · 2024-10-17T15:01:15Z

unblob/pool.py

-            result = self._output.get()
-            self._result_callback(self, result)
-            self._input.task_done()
+        with contextlib.suppress(EOFError):


Where do this EOFError come from?

from multiprocessing's queue, as it is using pipes

e3krisztian · 2024-10-17T15:22:44Z

unblob/pool.py

This went into the wrong commit...

e3krisztian · 2024-10-17T15:31:01Z

unblob/sandbox.py

+
+from structlog import get_logger
+from unblob_native.sandbox import (
+    AccessFS,


I know this comes from Landlock terminology, but this AccessFS name looks so alien to me (is it a file-system?).
I would use FSAccess, if possible.

e3krisztian · 2024-10-17T15:46:53Z

unblob/sandbox.py

+        extra_restrictions: Iterable[AccessFS] = (),
+    ):
+        self.restrictions = [
+            # Python, shared libraries, extractor binaries and so on
+            AccessFS.read("/"),
+            # Multiprocessing
+            AccessFS.read_write("/dev/shm"),  # noqa: S108
+            # Extracted contents
+            AccessFS.read_write(config.extract_root),
+            AccessFS.make_dir(config.extract_root.parent),
+            AccessFS.read_write(log_path),
+            *extra_restrictions,
+        ]
+
+        if report_file:
+            self.restrictions += [
+                AccessFS.read_write(report_file),
+                AccessFS.make_reg(report_file.parent),
+            ]


Since we have a deny-all, then allow exceptions mechanism, maybe a more specific name could be used.

Suggested change

extra_restrictions: Iterable[AccessFS] = (),

):

self.restrictions = [

# Python, shared libraries, extractor binaries and so on

AccessFS.read("/"),

# Multiprocessing

AccessFS.read_write("/dev/shm"), # noqa: S108

# Extracted contents

AccessFS.read_write(config.extract_root),

AccessFS.make_dir(config.extract_root.parent),

AccessFS.read_write(log_path),

*extra_restrictions,

]

if report_file:

self.restrictions += [

AccessFS.read_write(report_file),

AccessFS.make_reg(report_file.parent),

]

extra_passthrough: Iterable[AccessFS] = (),

):

self.passthrough = [

# Python, shared libraries, extractor binaries and so on

AccessFS.read("/"),

# Multiprocessing

AccessFS.read_write("/dev/shm"), # noqa: S108

# Extracted contents

AccessFS.read_write(config.extract_root),

AccessFS.make_dir(config.extract_root.parent),

AccessFS.read_write(log_path),

*extra_passthrough,

]

if report_file:

self.passthrough += [

AccessFS.read_write(report_file),

AccessFS.make_reg(report_file.parent),

]

qkaiser reviewed Jun 6, 2023

View reviewed changes

unblob/models.py Outdated Show resolved Hide resolved

unblob/processing.py Outdated Show resolved Hide resolved

unblob/processing.py Outdated Show resolved Hide resolved

vlaci force-pushed the landlock branch 4 times, most recently from d2138c8 to 3c51313 Compare June 7, 2023 21:01

vlaci force-pushed the landlock branch from 3c51313 to 1a6861b Compare June 7, 2023 21:14

qkaiser mentioned this pull request Aug 3, 2023

Report extraction problems from handlers/extractors #633

Merged

qkaiser mentioned this pull request Jan 8, 2024

Expose landlock API onekey-sec/unblob-native#11

Merged

qkaiser force-pushed the landlock branch 6 times, most recently from e92d6e3 to 52e9322 Compare January 8, 2024 10:01

qkaiser assigned qkaiser and unassigned qkaiser Jan 9, 2024

qkaiser added enhancement New feature or request dependencies Pull requests that update a dependency file labels Jan 9, 2024

qkaiser force-pushed the landlock branch 3 times, most recently from 59659a3 to c71c40a Compare February 4, 2024 10:47

qkaiser marked this pull request as ready for review February 4, 2024 10:49

qkaiser force-pushed the landlock branch from c71c40a to 58f5de7 Compare February 4, 2024 11:02

qkaiser force-pushed the landlock branch from 58f5de7 to 6a81e08 Compare February 21, 2024 12:43

vlaci force-pushed the landlock branch from 4d3fbb5 to 74384be Compare October 10, 2024 13:34

github-advanced-security bot found potential problems Oct 10, 2024

View reviewed changes

unblob/pool.py Fixed Show fixed Hide fixed

unblob/sandbox.py

self._try_enter_sandbox()

try:

result = callback(*args, **kwargs)

except BaseException as e:

Check notice

Code scanning / CodeQL

Except block handles 'BaseException' Note

Except block directly handles BaseException.

vlaci force-pushed the landlock branch 2 times, most recently from 3dc2926 to 681a54b Compare October 10, 2024 14:36

qkaiser force-pushed the landlock branch from 681a54b to b143767 Compare October 13, 2024 17:26

vlaci force-pushed the landlock branch from b143767 to 13bb299 Compare October 13, 2024 18:28

qkaiser reviewed Oct 16, 2024

View reviewed changes

unblob/pool.py Outdated Show resolved Hide resolved

unblob/sandbox.py Show resolved Hide resolved

tests: skip extensions cli test shouldn't call process file either

c8bbf96

all other tests in this file assert on `process_file` being called with correct arguments. We need specific tests which test that the configuration is interpreted correctly

vlaci force-pushed the landlock branch 2 times, most recently from 01db1b2 to 71eac4c Compare October 16, 2024 16:52

e3krisztian reviewed Oct 16, 2024

View reviewed changes

tests/test_cli.py Show resolved Hide resolved

tests/test_sandbox.py Show resolved Hide resolved

tests/test_sandbox.py Show resolved Hide resolved

unblob/processing.py Show resolved Hide resolved

unblob/sandbox.py Show resolved Hide resolved

vlaci and others added 3 commits October 16, 2024 19:47

feat: introduce landlock based sandboxing

c0d58b6

Co-authored-by: Quentin Kaiser <[email protected]>

unblob-native: 0.1.3 -> 0.1.4

7e864f8

Unblob now requires 0.1.4, so pyproject.toml version is also updated

vlaci force-pushed the landlock branch from 71eac4c to 7e864f8 Compare October 16, 2024 17:48

e3krisztian reviewed Oct 17, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental Landlock based sandboxing #597

Experimental Landlock based sandboxing #597

vlaci commented Jun 6, 2023

qkaiser commented Jun 6, 2023

vlaci commented Jun 6, 2023

vlaci commented Jun 7, 2023

qkaiser commented Jan 8, 2024 •

edited

Loading

qkaiser commented Feb 21, 2024

qkaiser commented Feb 21, 2024

qkaiser commented Oct 8, 2024 •

edited

Loading

vlaci commented Oct 9, 2024

vlaci commented Oct 10, 2024

qkaiser commented Oct 10, 2024

qkaiser commented Oct 11, 2024

qkaiser commented Oct 11, 2024 •

edited

Loading

qkaiser commented Oct 11, 2024

vlaci commented Oct 11, 2024

vlaci commented Oct 11, 2024

l0kod commented Oct 11, 2024

vlaci commented Oct 11, 2024

vlaci commented Oct 11, 2024 •

edited

Loading

vlaci commented Oct 16, 2024

e3krisztian Oct 17, 2024

e3krisztian Oct 17, 2024

e3krisztian Oct 17, 2024

vlaci Oct 18, 2024

e3krisztian Oct 17, 2024

e3krisztian Oct 17, 2024

e3krisztian Oct 17, 2024

Experimental Landlock based sandboxing #597

Are you sure you want to change the base?

Experimental Landlock based sandboxing #597

Conversation

vlaci commented Jun 6, 2023

qkaiser commented Jun 6, 2023

vlaci commented Jun 6, 2023

vlaci commented Jun 7, 2023

qkaiser commented Jan 8, 2024 • edited Loading

qkaiser commented Feb 21, 2024

qkaiser commented Feb 21, 2024

qkaiser commented Oct 8, 2024 • edited Loading

vlaci commented Oct 9, 2024

vlaci commented Oct 10, 2024

qkaiser commented Oct 10, 2024

qkaiser commented Oct 11, 2024

qkaiser commented Oct 11, 2024 • edited Loading

qkaiser commented Oct 11, 2024

vlaci commented Oct 11, 2024

vlaci commented Oct 11, 2024

l0kod commented Oct 11, 2024

vlaci commented Oct 11, 2024

vlaci commented Oct 11, 2024 • edited Loading

vlaci commented Oct 16, 2024

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

vlaci Oct 18, 2024

Choose a reason for hiding this comment

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

e3krisztian Oct 17, 2024

Choose a reason for hiding this comment

qkaiser commented Jan 8, 2024 •

edited

Loading

qkaiser commented Oct 8, 2024 •

edited

Loading

qkaiser commented Oct 11, 2024 •

edited

Loading

vlaci commented Oct 11, 2024 •

edited

Loading