Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[23.1] Fixes version parsing for WARNING situations #444

Merged
merged 4 commits into from
Aug 22, 2024

Conversation

Karm
Copy link
Collaborator

@Karm Karm commented Aug 19, 2024

@Karm Karm requested a review from jerboaa August 19, 2024 15:00
@Karm Karm self-assigned this Aug 19, 2024
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Aug 19, 2024
@Karm
Copy link
Collaborator Author

Karm commented Aug 19, 2024

@jerboaa

 git cherry-pick ff3824b

@Karm
Copy link
Collaborator Author

Karm commented Aug 19, 2024

CI fails on MacOS:

Downloading NINJA_SYNTAX from ['https://lafo.ssw.uni-linz.ac.at/pub/graal-external-deps/ninja_syntax-1.7.2.tar.gz']
JvmFuncsFallbacksBuildTask svm-jvmfuncs-fallback-builder... [/Users/runner/work/mandrel-packaging/mandrel-packaging/mandrel/substratevm/mxbuild/jdk21/svm-jvmfuncs-fallback-builder/gensrc/JvmFuncsFallbacks.c does not exist]
JvmFuncsFallbacksBuildTask svm-jvmfuncs-fallback-builder: Failed due to error: 1

Not sure what it means, I'll try to reproduce on our own machines.

@Karm Karm force-pushed the 23.1-version-check-fails-on-warning branch from d9071bf to cc8e98d Compare August 19, 2024 15:47
@Karm
Copy link
Collaborator Author

Karm commented Aug 21, 2024

I am having hard time reproducing this.
It seems fine on our lab box (don't mind the dir name):

INFO [build] Congratulations you successfully built Mandrel 23.1.4.1-devec7b5936677 based on Java 21.0.5-beta+3-ea
INFO [build] You can find your newly built native-image enabled JDK under /Users/tester/tmp/mandrel-23.1.3.1-Final

...

tester@mandrel-macos-aarch64-1 tmp % /Users/tester/tmp/mandrel-23.1.3.1-Final/Contents/Home/bin/native-image --version
native-image 21.0.5-beta 2024-10-15
OpenJDK Runtime Environment Mandrel-23.1.4.1-devec7b5936677 (build 21.0.5-beta+3-ea)
OpenJDK 64-Bit Server VM Mandrel-23.1.4.1-devec7b5936677 (build 21.0.5-beta+3-ea, mixed mode)

@Karm
Copy link
Collaborator Author

Karm commented Aug 21, 2024

GHA, MacOS amd64 ❌

Archiving GCC_NINJA_TOOLCHAIN... [/Users/runner/work/mandrel-packaging/mandrel-packaging/mx/mxbuild/jdk21/dists/gcc-ninja-toolchain.tar does not exist]
JvmFuncsFallbacksBuildTask svm-jvmfuncs-fallback-builder... [/Users/runner/work/mandrel-packaging/mandrel-packaging/mandrel/substratevm/mxbuild/jdk21/svm-jvmfuncs-fallback-builder/gensrc/JvmFuncsFallbacks.c does not exist]
JvmFuncsFallbacksBuildTask svm-jvmfuncs-fallback-builder: Failed due to error: 1
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Failed, exit code: 1
	at OperatingSystem.exec(build.java:1848)
	at SequentialBuild.lambda$build$0(build.java:656)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at Mx.build(build.java:1068)
	at SequentialBuild.build(build.java:658)
	at build.main(build.java:83)
Caused by: java.lang.RuntimeException: Failed, exit code: 1
	at OperatingSystem.exec(build.java:1837)
	at SequentialBuild.lambda$build$0(build.java:656)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at Mx.build(build.java:1068)
	at SequentialBuild.build(build.java:658)
	at build.main(build.java:83)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at jdk.compiler/com.sun.tools.javac.launcher.Main.execute(Main.java:484)
	at jdk.compiler/com.sun.tools.javac.launcher.Main.run(Main.java:208)
	at jdk.compiler/com.sun.tools.javac.launcher.Main.main(Main.java:135)
Error: Process completed with exit code 1.

Lab box, M2 Pro, MacOS aarch64 ✔️

16550 Archiving GCC_NINJA_TOOLCHAIN... [/Users/tester/tmp/mx/mxbuild/jdk21/dists/gcc-ninja-toolchain.tar does not exist]
16551 JvmFuncsFallbacksBuildTask svm-jvmfuncs-fallback-builder... [/Users/tester/tmp/mandrel/substratevm/mxbuild/jdk21/svm-jvmfuncs-fallback-builder/gensrc/JvmFuncsFallbacks.c does not exist]
16552 Collect from : /Users/tester/tmp/jdk-21.0.5+3/Contents/Home/lib/static/darwin-aarch64/libnet.a
16553 # Directory: /Users/tester/tmp/mandrel/substratevm
16554 env -i TERM=xterm-256color SHELL=/bin/zsh WORKSPACE=/Users/tester/tmp MX_PYTHON=python USER=tester MX_HOME=/Users/tester/tmp/mx MANDREL_VERSION_UNTIL_SPACE=23.1.3.1-Final __CF_USER_TEXT_ENCODING=0x1F6:0x0:0      x0 MANDREL_VERSION_SUBSTRING=23.1.3.1-Final PATH=/Users/tester/tmp/openjdk-21/Contents/Home//bin:/Users/tester/tmp/mx:/opt/homebrew/opt/python/libexec/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/b      in:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/cod      ex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin PWD=/Users/tester/tmp/mandrel/substratevm JAVA_HOME=/Users/tester/tmp/jdk-21.0.5+3/Contents/Home MANDREL_VERSION='23.1.3.1-Final ec7b5936677'       MAVEN_REPO=/Users/tester/.m2/repository JAVA_VERSION='' MANDREL_HOME=/Users/tester/tmp/mandrel-23.1.3.1-Final HOME=/Users/tester SHLVL=3 LOGNAME=karm MANDREL_REPO=/Users/tester/tmp/mandrel LC_CTYPE=UTF-8 M      X__SUITEMODEL=sibling MX_PRIMARY_SUITE_PATH=/Users/tester/tmp/mandrel/substratevm MX_SUBPROCESS_COMMAND_FILE=/tmp/mx_subprocess_command.tsv56nhn \
16555 nm /Users/tester/tmp/jdk-21.0.5+3/Contents/Home/lib/static/darwin-aarch64/libnet.a
16556 # Directory: /Users/tester/tmp/mx
16557 env -i TERM=xterm-256color SHELL=/bin/zsh WORKSPACE=/Users/tester/tmp MX_PYTHON=python USER=tester MX_HOME=/Users/tester/tmp/mx MANDREL_VERSION_UNTIL_SPACE=23.1.3.1-Final __CF_USER_TEXT_ENCODING=0x1F6:0x0:0      x0 MANDREL_VERSION_SUBSTRING=23.1.3.1-Final PATH=/Users/tester/tmp/openjdk-21/Contents/Home//bin:/Users/tester/tmp/mx:/opt/homebrew/opt/python/libexec/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/b      in:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/cod      ex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin PWD=/Users/tester/tmp/mandrel/substratevm JAVA_HOME=/Users/tester/tmp/jdk-21.0.5+3/Contents/Home MANDREL_VERSION='23.1.3.1-Final ec7b5936677'       MAVEN_REPO=/Users/tester/.m2/repository JAVA_VERSION='' MANDREL_HOME=/Users/tester/tmp/mandrel-23.1.3.1-Final HOME=/Users/tester SHLVL=3 LOGNAME=karm MANDREL_REPO=/Users/tester/tmp/mandrel LC_CTYPE=UTF-8 M      X__SUITEMODEL=sibling MX_PRIMARY_SUITE_PATH=/Users/tester/tmp/mandrel/substratevm MX_SUBPROCESS_COMMAND_FILE=/tmp/mx_subprocess_command.63xppncd \
16558 git ls-files /Users/tester/tmp/mx/ninja-toolchains/gcc.ninja[55176: started subprocess 55219: ['nm', '/Users/tester/tmp/jdk-21.0.5+3/Contents/Home/lib/static/darwin-aarch64/libnet.a']]
16559 
16560 [55176: started subprocess 55220: ['git', 'ls-files', '/Users/tester/tmp/mx/ninja-toolchains/gcc.ninja']]

So the error /Users/tester/tmp/mandrel/substratevm/mxbuild/jdk21/svm-jvmfuncs-fallback-builder/gensrc/JvmFuncsFallbacks.c does not exist exists in both logs, but my local one (and the Jenkins driven one) recovers while the GHA one dies.

@zakkak
Copy link
Collaborator

zakkak commented Aug 21, 2024

So the error /Users/tester/tmp/mandrel/substratevm/mxbuild/jdk21/svm-jvmfuncs-fallback-builder/gensrc/JvmFuncsFallbacks.c does not exist exists in both logs, but my local one (and the Jenkins driven one) recovers while the GHA one dies.

Yes, that means that the file doesn't exist indicating why the JvmFuncsFallbacksBuildTask task got triggered. I tried reproducing on a Macbook M3 Air as well without luck.

I wonder if passing --verbose to build.java in the CI job would give some more info. Do you mind giving it a go?

@Karm Karm requested a review from zakkak August 21, 2024 13:13
@Karm
Copy link
Collaborator Author

Karm commented Aug 21, 2024

Looks fixed @zakkak.
And because I scientifically changed more than 1 thing, we don't know what was the fix. IMHO it's the MX version.

@zakkak
Copy link
Collaborator

zakkak commented Aug 21, 2024

Could you please split the "backport" commit in three commits? Two using git cherry-pick -x to get the changes from #415 and #383 and another one for the mx bump (which I think should be 6.46.1)?

This way it's much easier to track things for future reference.

@Karm
Copy link
Collaborator Author

Karm commented Aug 21, 2024

Could you please split the "backport" commit in three commits?
Two using git cherry-pick -x to get the changes from #415 and #383 and another one for the mx bump

Ack. Will do.

(which I think should be 6.46.1)?

refs/tags/6.52.0 was used to release mandrel-23.1.4.0-Final so I'd like to stick to that. 6.46.1 is minimal, right?

This way it's much easier to track things for future reference.

Ack.

Copy link
Collaborator

@jerboaa jerboaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, with changes @zakkak requested.

@zakkak
Copy link
Collaborator

zakkak commented Aug 22, 2024

refs/tags/6.52.0 was used to release mandrel-23.1.4.0-Final so I'd like to stick to that. 6.46.1 is minimal, right?

Correct, I am fine with 6.52.0 then.

zakkak and others added 3 commits August 22, 2024 14:58
`mx` uses `distutils` which is no longer available in python 3.12 which
is the default in `macos-latest`

Use 3.10 instead in all platforms for consistency. The minimum supported
version by `mx` is 3.9 see graalvm/mx#249.

Closes graalvm#382

(cherry picked from commit 6c4edd8)
@Karm Karm force-pushed the 23.1-version-check-fails-on-warning branch from 9189850 to 712ca02 Compare August 22, 2024 13:21
@Karm
Copy link
Collaborator Author

Karm commented Aug 22, 2024

@zakkak @jerboaa THX. All green, comments addressed, merging...

@Karm Karm merged commit d45317e into graalvm:23.1 Aug 22, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants