Skip to content

Commit

Permalink
Don't swallow gtest failures when running tests in LLDB
Browse files Browse the repository at this point in the history
In CI, we run our various tests in LLDB as a debugger. LLDB when run
in batch mode like we are doing, always returns 0 as the exit code. It
doesn't return the exit code from the program being run. The LLDB documentation
states this is because more than 1 process can be being debugged so the
concept that GDB has for returning the error code is more problematic with
LLDB.

The result of this is that non-exception errors in CI are not being flagged
because LLDB is eating the exit code from gtest when a test fails. gest offers
the `--gtest_throw_on_failure` option that turns test failures into exceptions.

Those exceptions will be caught when we are using LLDB and reported correctly
in CI.

Long term, we have discussed adding an LLDB module with python to get us the
"return exit code". There is discussion on Stack Overflow of this approach at
https://stackoverflow.com/questions/56456724/quit-lldb-with-exit-code-of-process/76741573#76741573
  • Loading branch information
SeanTAllen committed Nov 27, 2024
1 parent f0d7afd commit 2865574
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,7 @@ endif

ifneq ($(findstring lldb,$(usedebugger)),)
debuggercmd := $(usedebugger) --batch --one-line "breakpoint set --name main" --one-line run --one-line "process handle SIGINT --pass true --stop false" --one-line "process handle SIGUSR2 --pass true --stop false" --one-line "thread continue" --one-line-on-crash "frame variable" --one-line-on-crash "register read" --one-line-on-crash "bt all" --one-line-on-crash "quit 1" --
testextras := --gtest_throw_on_failure
else ifneq ($(findstring gdb,$(usedebugger)),)
debuggercmd := $(usedebugger) --quiet --batch --return-child-result --eval-command="set confirm off" --eval-command="set pagination off" --eval-command="handle SIGINT nostop pass" --eval-command="handle SIGUSR2 nostop pass" --eval-command=run --eval-command="info args" --eval-command="info locals" --eval-command="info registers" --eval-command="thread apply all bt full" --eval-command=quit --args
else ifneq ($(strip $(usedebugger)),)
Expand Down Expand Up @@ -214,10 +215,10 @@ test-check-version: all
test-core: all test-libponyrt test-libponyc test-full-programs-release test-full-programs-debug

test-libponyrt: all
$(SILENT)cd '$(outDir)' && $(debuggercmd) ./libponyrt.tests --gtest_shuffle
$(SILENT)cd '$(outDir)' && $(debuggercmd) ./libponyrt.tests --gtest_shuffle $(testextras)

test-libponyc: all
$(SILENT)cd '$(outDir)' && $(debuggercmd) ./libponyc.tests --gtest_shuffle
$(SILENT)cd '$(outDir)' && $(debuggercmd) ./libponyc.tests --gtest_shuffle $(testextras)

ifeq ($(shell uname -s),FreeBSD)
num_cores := `sysctl -n hw.ncpu`
Expand Down

0 comments on commit 2865574

Please sign in to comment.