The openssl built by ruby-build has somewhat broken library dependencies

[eregon]

Steps to reproduce the behavior

On Linux:

$ RUBY_BUILD_VENDOR_OPENSSL=true ruby-build --verbose ruby-3.4.1 ~/.rubies/ruby-3.4.1-openssl-3.4.0
$ cd ~/.rubies/ruby-3.4.1-openssl-3.4.0/openssl
$ bin/openssl version          
bin/openssl: /lib64/libssl.so.3: version `OPENSSL_3.4.0' not found (required by bin/openssl)
bin/openssl: /lib64/libssl.so.3: version `OPENSSL_3.2.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by bin/openssl)
$ ldd bin/openssl     
bin/openssl: /lib64/libssl.so.3: version `OPENSSL_3.4.0' not found (required by bin/openssl)
bin/openssl: /lib64/libssl.so.3: version `OPENSSL_3.2.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by bin/openssl)
bin/openssl: /lib64/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by bin/openssl)
	linux-vdso.so.1 (0x00007ffdc43e5000)
	libssl.so.3 => /lib64/libssl.so.3 (0x00007fa625c4b000) <= BUG
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007fa625600000) <= BUG
	libc.so.6 => /lib64/libc.so.6 (0x00007fa62541e000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fa625c31000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa625d0a000)
$ ldd lib/libssl.so   
lib/libssl.so: /lib64/libcrypto.so.3: version `OPENSSL_3.3.0' not found (required by lib/libssl.so)
lib/libssl.so: /lib64/libcrypto.so.3: version `OPENSSL_3.4.0' not found (required by lib/libssl.so)
lib/libssl.so: /lib64/libcrypto.so.3: version `OPENSSL_3.2.0' not found (required by lib/libssl.so)
	linux-vdso.so.1 (0x00007fffc97f4000)
	libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007feaaca00000) <= BUG
	libc.so.6 => /lib64/libc.so.6 (0x00007feaac81e000)
	libz.so.1 => /lib64/libz.so.1 (0x00007feaad07b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007feaad1b5000)

I added <= BUG to the output to highlight the problematic lines.
So we see bin/openssl links to system libssl and system libcrypto, when it should link those inside the openssl prefix.
And also that the built libssl links to system libcrypto and not in-the-same-dir libcrypto.
Since the system openssl in this case is 3.1.4 it results in relatively clear errors, but if they were closer it could maybe crash or so.

Luckily this doesn't seem to affect the Ruby openssl extension, which links correctly to the correct libssl and libcrypto:

$ chruby ruby-3.4.1-openssl-3.4.0                          
$ ldd $(ruby -ropenssl -e 'puts $".grep(/openssl\.(so|bundle)/)')
	linux-vdso.so.1 (0x00007ffc87fd8000)
	libruby.so.3.4 => /home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/lib/libruby.so.3.4 (0x00007fa5b4400000)
	libssl.so.3 => /home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/openssl/lib/libssl.so.3 (0x00007fa5b42fc000)
	libcrypto.so.3 => /home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/openssl/lib/libcrypto.so.3 (0x00007fa5b3c00000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fa5b421b000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fa5b3a1e000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fa5b4b13000)
	libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007fa5b4ada000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fa5b4ab5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa5b4ba8000)

But that still seems brittle because I'm not sure it's guaranteed the libcrypto dependency there always wins over the dependency declared in libssl. Having them with different values certainly seems bad.
I also checked with ruby -ropen-uri -e 'puts URI.send(:open, %{https://rubygems.org/}) { |f| f.read(1024) }; sleep' and lsof and that looks fine, only the correct ibssl/libcrypto are loaded. But maybe by "chance".

I think it would be nice to fix this so the built openssl has correct library dependencies and we don't run the risk of these conflicting dependencies to stop working at some point.

I noticed this with OpenSSL 3.4 but I think it happens with older versions as well like 3.0.
FWIW this is the patch I used because I wanted to check against OpenSSL 3.4 specifically (for oracle/truffleruby#3724):

diff --git a/share/ruby-build/3.4.1 b/share/ruby-build/3.4.1
index adb10b78..d692532c 100644
--- a/share/ruby-build/3.4.1
+++ b/share/ruby-build/3.4.1
@@ -1,2 +1,2 @@
-install_package "openssl-3.0.15" "https://github.com/openssl/openssl/releases/download/openssl-3.0.15/openssl-3.0.15.tar.gz#23c666d0edf20f14249b3d8f0368acaee9ab585b09e1de82107c66e1f3ec9533" openssl --if needs_openssl:1.0.2-3.x.x
+install_package "openssl-3.4.0" "https://github.com/openssl/openssl/releases/download/openssl-3.4.0/openssl-3.4.0.tar.gz" openssl --if needs_openssl:1.0.2-3.x.x
 install_package "ruby-3.4.1" "https://cache.ruby-lang.org/pub/ruby/3.4/ruby-3.4.1.tar.gz#3d385e5d22d368b064c817a13ed8e3cc3f71a7705d7ed1bae78013c33aa7c87f" enable_shared standard

[eregon]

$ readelf -d /home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/lib/ruby/3.4.0/x86_64-linux/openssl.so

Dynamic section at offset 0x58b48 contains 29 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libruby.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libssl.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libcrypto.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000001d (RUNPATH)            Library runpath: [/home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/openssl/lib:/home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/lib]
 0x000000000000000c (INIT)               0x14000

So CRuby build system adds /home/eregon/.rubies/ruby-3.4.1-openssl-3.4.0/lib to the runpath, that's likely what saves us (but I think it could still break if the library loader decides to load libcrypto from libssl first (instead of from openssl.so), where libssl has no runpath and so it'd pick the system libcrypto).

I think the fix would be to get that runpath/rpath inside libssl and bin/openssl as well.
I'm not sure of the right way to do that with the OpenSSL build system though.