Upgrade JDK toolchain/target version to 17 and add GraalJS #1207
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…et GraalJS+LiquidJS working via a transformer. There's a rough test in place that shows the relative performance between Liquid.js and a raw javascript call. The results on my mac were about 300x slower to use the templating solution than to just run JSON.stringify(). /Library/Java/JavaVirtualMachines/amazon-corretto-17.jdk/Contents/Home/bin/java -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI ... org.opensearch.migrations.transform.LiquidJsTransformerTest [To redirect Truffle log output to a file use one of the following options: * '--log.file=<path>' if the option is passed using a guest language launcher. * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher. * Configure logging using the polyglot embedding API.] [engine] WARNING: The polyglot engine uses a fallback runtime that does not support runtime compilation to native code. Execution without runtime compilation will negatively impact the guest application performance. The following cause was found: Libgraal compilation is not available on this JVM. Alternatively, the org.graalvm.compiler:compiler module can be put on the --upgrade-module-path. For more information see: https://www.graalvm.org/latest/reference-manual/embed-languages/#runtime-optimization-support. To disable this warning use the '--engine.WarnInterpreterOnly=false' option or the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property. [To redirect Truffle log output to a file use one of the following options: * '--log.file=<path>' if the option is passed using a guest language launcher. * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher. * Configure logging using the polyglot embedding API.] [engine] WARNING: The polyglot engine uses a fallback runtime that does not support runtime compilation to native code. Execution without runtime compilation will negatively impact the guest application performance. The following cause was found: Libgraal compilation is not available on this JVM. Alternatively, the org.graalvm.compiler:compiler module can be put on the --upgrade-module-path. For more information see: https://www.graalvm.org/latest/reference-manual/embed-languages/#runtime-optimization-support. To disable this warning use the '--engine.WarnInterpreterOnly=false' option or the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property. [INFO ] 2024-12-20 04:11:56,654825 [main] LiquidJsTransformerTest - Run 0: PT4.052694709S [INFO ] 2024-12-20 04:11:59,694596 [main] LiquidJsTransformerTest - Run 1: PT3.039395S [INFO ] 2024-12-20 04:12:02,747087 [main] LiquidJsTransformerTest - Run 2: PT3.052563917S [INFO ] 2024-12-20 04:12:05,823644 [main] LiquidJsTransformerTest - Run 3: PT3.076653792S [INFO ] 2024-12-20 04:12:08,833241 [main] LiquidJsTransformerTest - Run 4: PT3.009687375S [To redirect Truffle log output to a file use one of the following options: * '--log.file=<path>' if the option is passed using a guest language launcher. * '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher. * Configure logging using the polyglot embedding API.] [engine] WARNING: The polyglot engine uses a fallback runtime that does not support runtime compilation to native code. Execution without runtime compilation will negatively impact the guest application performance. The following cause was found: Libgraal compilation is not available on this JVM. Alternatively, the org.graalvm.compiler:compiler module can be put on the --upgrade-module-path. For more information see: https://www.graalvm.org/latest/reference-manual/embed-languages/#runtime-optimization-support. To disable this warning use the '--engine.WarnInterpreterOnly=false' option or the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property. [INFO ] 2024-12-20 04:12:08,906295 [main] LiquidJsTransformerTest - Run 0: PT0.021998S [INFO ] 2024-12-20 04:12:08,916581 [main] LiquidJsTransformerTest - Run 1: PT0.010198S [INFO ] 2024-12-20 04:12:08,927812 [main] LiquidJsTransformerTest - Run 2: PT0.011137708S [INFO ] 2024-12-20 04:12:08,938391 [main] LiquidJsTransformerTest - Run 3: PT0.010506958S [INFO ] 2024-12-20 04:12:08,948759 [main] LiquidJsTransformerTest - Run 4: PT0.010288125S [INFO ] 2024-12-20 04:12:08,959167 [main] LiquidJsTransformerTest - Run 5: PT0.010324625S Signed-off-by: Greg Schohn <[email protected]>
…transform invocation scripts and rewrite the type mappings sanitization transformer to be pure javascript as opposed to liquid (or jinja). I've tweaked a bulk request test time the accumulated time for 10K warm runs and the average time to transform is around 0.2ms (without '-ea'), which is a major improvement over liquid/jinja scripts. Signed-off-by: Greg Schohn <[email protected]>
chelma
reviewed
Jan 2, 2025
| @@ -24,6 +24,9 @@ plugins { | |||
| id 'java-test-fixtures' | |||
| } | |||
|
|
|||
| java.sourceCompatibility = JavaVersion.VERSION_11 | |||
There was a problem hiding this comment.
Why have these, specifically, set to JDK 11 when everything else is moving to JDK 17?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Upgrade JDK toolchain and target version for most packages to 17 to get GraalJS+LiquidJS working via a transformer. After noticing extremely dissappointing results for liquid templated transforms (see below), I reworked the type mappings sanitization transform in pure javascript code. The results for recording 10K bulk request transformations went from
Old jinja transform
New javascript based transformation
That's a 14x improvement over the jinja code. Considering that we noticed that RFS performance went down by 20x, that would imply that 95% of the time was spent on transforms. Making that 14x faster could imply a ~10x performance increase (or only 2x worse than what it was without transformations). Notice that there's a lot less serialization back and forth with the javascript transform than with the template approach, but the code's also pretty sub-optimal, possibly like what I would expect to come in from a contributor - at least if they didn't have much experience with javascript.
There are two commits in this PR. The first one implements the TypeMappinsSanitization as an incomplete liquid transform. I stopped developing that because a rough test showed a huge performance difference between Liquid.js and a raw javascript call. The results on my mac were about 300x slower to use the templating solution than to just run JSON.stringify() (likely much worse than jinjava).
Below are some logs showing those differences for the liquid transforms.
Description
Issues Resolved
Testing
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.