MDBF-793 - Retire MSAN clang-15 builder, upgrade to clang-19 #562
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vladbogo
approved these changes
Sep 18, 2024
Add worker for openeuler 24.03. Rename container generation to mathc the same name.
As per https://github.com/MariaDB/server/blob/8478a06cdbb7281e1cc2d507f18c985c736c2ffc/cmake/cpack_rpm.cmake#L374 MariaDB-compat won't be built for OpenEuler.
Clients in our binary taballs are linked dynamically with libncurses, which causes a trouble on systems where it is no longer available As per MDBF-794 only centos7 and debian 10 bintar builders need to link statically with libncurses. - for debian 10: libncurses.a is already present in /scripts/local/lib/ and -DCURSES_NEED_NCURSES=1 flag is enough to link it statically. - for centos7: I've rebuilt the image and pushed manually a dev_ tag. libncurses.a was present on the system but not in the library path: /scripts/local/lib/ Tests available at: - debian10: https://buildbot.dev.mariadb.org/#/builders/187/builds/23 - centos7: https://buildbot.dev.mariadb.org/#/builders/172/builds/292 Reference CMAKE output line: -- Found Curses: /scripts/local/lib/libncurses.a
Adjust images to 1505 notation as well.
|
@grooverdan |
Fedora 41 is including MariaDB 10.11 so use that as the minimum version.
grooverdan
force-pushed
the
feature/msan-clang-19
branch
from
89ab27f
to
852a451
Compare
Centos based have asio-devel in EPEL. SLES is omitted as it requires a subpackage module repository to be enabled. Doing so would increase the package dependencies when using the built packages. https://packagehub.suse.com/packages/asio/ (SLE-Module-PackageHub-Subpackages-Module ) OpenSUSE also has asio-devel but adds a complicated path in with the openssl libraries and boost that we'd be influcting on users if they used a non-bundled version. On RHEL - asio-devel in ubi9 EPEL but not 8.
Parts 1 and 2: Switching minor upgrade into "all" mode and extending it with additional checks (ldd, requirements, capabilities)
Post-review fixes and cosmetic changes
During the "Compile step" - create_initial_db.cmake runs a bootstrap as a sanity check to avoid spending time on test runs. It can happen that some changes on the server may cause a loop during the bootstrap process. Buildbot will terminate the parent process (dojob) when it reaches a timeout, but the MariaDB service will remain up causing new builds to fail. https://buildbot.mariadb.org/#/builders/234/builds/32911 Modified the stop_processes step so handle this.
Use ( deb | archive ).mariadb.org to download the Packages file. This file is mandatory for both the 'All' and 'Columnstore' test types, exit with error if it is not found. Apply uniq to remove duplicate values in the package list and sort the list for readability.
… old bb - .build-id files may also be present on old systems where --noartifact does not work - package dependencies need to be sorted for comparison - dependency collection hangs frequently in buildbot, enabled logging temporarily to get more output for the issue
… old bb - .build-id files may also be present on old systems where --noartifact does not work - package dependencies need to be sorted for comparison - dependency collection hangs frequently in buildbot, enabled logging temporarily to get more output for the issue
Improving build time by excluding the too many files of mariadb-test package from the ldd check when dependencies are collected.
Also use update-alternatives to provide clang/clang++ links. With a 6 monthly new release cycle of clang. Keeping it such that a rebuild of image is sufficient to re-deploy rather than a master restart will facilitate more frequent updates and a fixed builder url.
Requested by Marko to make this a more realistic test of the codebase delivered to users. No -DWITH_DBUG_TRACE=OFF exists in RelWithDebInfo mode
make clang/clang++ alternates as soon as installed
Unit tests run all ok. WITH_SAFE_MALLOC=OFF is default in RelWithDebInfo mode.
Also convient CFLAGS and MSAN environment variables.
to non-instrumented things. Otherwise these sorts of error in testing: LD_LIBRARY_PATH=/msan-libs/ /usr/bin/ctest /usr/bin/ctest: symbol lookup error: /msan-libs/libgmp.so.10: undefined symbol: __msan_va_arg_overflow_size_tls
And the --force is for debian:11 compat where the files exist in the tarball.
Use cmake to install
Didn't instrument sufficiently.
MSAN errors resultin the form:
==9000==WARNING: MemorySanitizer: use-of-uninitialized-value
[Detaching after fork from child process 9003]
#0 0x55c87cbda03f in my_base64_encode /source/mysys/base64.c:113:13
#1 0x55c87af96efb in calculate_server_uid(char*) /source/sql/mysqld.cc:9889:3
#2 0x55c87af96efb in init_common_variables() /source/sql/mysqld.cc:3882:7
#3 0x55c87af9111e in mysqld_main(int, char**) /source/sql/mysqld.cc:5546:7
#4 0x7f6d94ab5249 (/lib/x86_64-linux-gnu/libc.so.6+0x27249) (BuildId: c047672cae7964324658491e7dee26748ae5d2f8)
MariaDB#5 0x7f6d94ab5304 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x27304) (BuildId: c047672cae7964324658491e7dee26748ae5d2f8)
MariaDB#6 0x55c87aeef130 in _start (/build/sql/mariadbd+0x6ee130) (BuildId: abe0c6d0a0f0bdcee26c5e1f69353335cca3bd8a)
The sha1 has that is being encoded wasn't maked as initialized though
the openssl libs wrote the hash there.
grooverdan
force-pushed
the
feature/msan-clang-19
branch
from
0c93f34
to
92566d9
Compare
RazvanLiviuVarzaru
force-pushed
the
dev
branch
from
d7eb898
to
4be5d7c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace MSAN CLANG-15 builder with CLANG 19 on Debian 12.