Skip to content

Commit

Permalink
Refactoring guides according to demos
Browse files Browse the repository at this point in the history
  • Loading branch information
dnestoro committed Dec 3, 2024
1 parent 67d287e commit 46751a3
Show file tree
Hide file tree
Showing 6 changed files with 566 additions and 289 deletions.
272 changes: 182 additions & 90 deletions docs/src/docs/asciidoc/end-to-end-gradle-guide.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ The GraalVM team
This guide leads you through the process of integrating Native Gradle Plugin with your project.
It starts from adding the plugin to your project, goes through some of the main functionalities that you may use (like collecting the metadata),
all the way to some diagnostics tools that you can use to analyse native executables you have made.
You can see an example of tests on which we tested whole guide <<example,here>>.

In case you only want to see how a simple application works in practise, you can check <<quickstart-gradle-plugin.adoc#,our demo>>.

Expand Down Expand Up @@ -178,10 +179,10 @@ Inside these blocks you can pass the following options:
- `jvmArgs` - Passes the given argument directly to the JVM running the native image builder
- `useFatJar` - Instead of passing each jar individually, builds a fat jar

You can also pass **build-time** and **run-time** options to the Native Image using:
You can also pass **build-time** and **run-time** arguments:

- `buildArgs.add('<buildArg>')` - You can find more about possible build arguments https://www.graalvm.org/latest/reference-manual/native-image/overview/BuildConfiguration/[here]
- `runtimeArgs.add('<runtimeArg>')` - You can find more about possible runtime arguments https://www.graalvm.org/latest/reference-manual/native-image/overview/Options/[here]
- `buildArgs.add('<buildArg>')` - You can find more about possible build arguments https://www.graalvm.org/latest/reference-manual/native-image/overview/Options/[here] and https://www.graalvm.org/latest/reference-manual/native-image/overview/BuildConfiguration/[here]
- `runtimeArgs.add('<runtimeArg>')` - Runtime arguments consumed by your application

Here is an example of additional options usage:

Expand Down Expand Up @@ -258,9 +259,9 @@ The easiest way how this information can be passed to the Native Image is throug
a single `reachability-metadata.json` file (for newer GraalVM versions) or multiple json files (`reflect-config.json`, `resource-config.json`, `proxy-config.json`, `serialization-config.json`, `jni-config.json`).
To learn more about metadata that Native Image consumes, https://www.graalvm.org/latest/reference-manual/native-image/metadata/[see this].

For example, if you run the test that tries to load resource `resource.txt`, and you don't have entry for that resource in the metadata config file, the resource can't be loaded (will be null).
For example, if you run the test that uses the reflection, and you don't have entry for that reflection call in the metadata config file, your test will fail.

To make your test/application work while using resources (like in this example) or other metadata, you should either generate metadata configurations or write them manually.
To make your test/application work while using reflection (like in this example) or other metadata, you should either generate metadata configurations or write them manually.
To generate metadata automatically, you can run your tests (or the main application) with the Native Image Agent, that will collect all the metadata your test/application require.
To enable the agent (through Native Gradle Plugin) you should either:

Expand Down Expand Up @@ -291,8 +292,6 @@ agent {
[.underline]#To generate the metadata file(s) for your `application` just run:#

- `./gradlew run` if you added the agent block to the configuration or `./gradlew -Pagent run` if you didn't. This command runs on JVM with native-image-agent and collects the metadata.
- `./gradlew nativeRun` if you added the agent block to the configuration or `./gradlew -Pagent nativeRun` if you didn't. This command runs on JVM with the native-image agent, collects the metadata and uses it for testing on native-image.


[WARNING]
====
Expand Down Expand Up @@ -380,7 +379,7 @@ Besides that, you can configure `metadataCopy` task through the command line as

Here is an example of a valid `metadataCopy` usage:

[source,bash,subs="verbatim,attributes"]
[source,bash,subs="verbatim,attributes", role="multi-language-sample"]
----
./gradlew metadataCopy --task test --dir resources/META-INF/native-image/org.example
----
Expand Down Expand Up @@ -414,9 +413,9 @@ First thing that you can configure is the agent mode.
There are three possible agent modes:

* `standard` - only generates metadata without any special processing (this is the default mode). No additional options available.
* `conditional` - entries of the generated metadata will be included in the Native Image only if the condition in the entry is satisfied. Consumes following additional options:
* `conditional` - entries of the generated metadata will be included in the Native Image only if the condition in the entry is satisfied. Consumes following options:
** `userCodeFilterPath` - specifies a filter file used to classify classes as user application classes. Generated conditions will only reference these classes See <<agent-filter-file, the following section>>
** `extraFilterPath` - extra filter used to further filter the collected metadata. See <<agent-filter-file, the following section>>
** `extraFilterPath` - (optional) extra filter used to further filter the collected metadata. See <<agent-filter-file, the following section>>
* `direct` - in this mode user configures the agent completely manually by adding all options with:
** `options.add("<option>")`

Expand Down Expand Up @@ -581,7 +580,6 @@ These filter files that agent consumes have the following structure:
The process how you can pass the config files to the agent is described in the <<additional-agent-options,previous section>>.

We can see on the example how different filter files affect generated metadata:
**Note that the following example was created with GraalVM 21 and that the format of the generated metadata can vary from version to version.**

We are starting with the simple filter file:

Expand All @@ -594,104 +592,143 @@ We are starting with the simple filter file:
}
----

This filter file will instruct the agent to include everything and therefore, you will get a massive config files.
For example this is how `resource-config.json` looks like:
This filter file will instruct the agent to include everything and therefore, you will get a massive config file.
For example this is how `reachability-metadata.json` looks like:

[source,json,subs="verbatim,attributes", role="multi-language-sample"]
----
{{
"resources":{
"includes":[{
"condition":{"typeReachable":"jdk.internal.logger.BootstrapLogger$DetectBackend$1"},
"pattern":"\\QMETA-INF/services/java.lang.System$LoggerFinder\\E"
}, {
"condition":{"typeReachable":"jdk.internal.logger.LoggerFinderLoader"},
"pattern":"\\QMETA-INF/services/java.lang.System$LoggerFinder\\E"
}, {
"condition":{"typeReachable":"java.nio.channels.spi.SelectorProvider$Holder"},
"pattern":"\\QMETA-INF/services/java.nio.channels.spi.SelectorProvider\\E"
}, {
"condition":{"typeReachable":"java.time.zone.ZoneRulesProvider"},
"pattern":"\\QMETA-INF/services/java.time.zone.ZoneRulesProvider\\E"
}, {
"condition":{"typeReachable":"org.junit.platform.launcher.core.LauncherFactory"},
"pattern":"\\QMETA-INF/services/org.junit.platform.engine.TestEngine\\E"
}, {
"condition":{"typeReachable":"org.junit.platform.launcher.core.LauncherFactory"},
"pattern":"\\QMETA-INF/services/org.junit.platform.launcher.LauncherDiscoveryListener\\E"
}, {
"condition":{"typeReachable":"org.junit.platform.launcher.core.LauncherFactory"},
"pattern":"\\QMETA-INF/services/org.junit.platform.launcher.LauncherSessionListener\\E"
}, {
"condition":{"typeReachable":"org.junit.platform.launcher.core.LauncherFactory"},
"pattern":"\\QMETA-INF/services/org.junit.platform.launcher.PostDiscoveryFilter\\E"
}, {
"condition":{"typeReachable":"java.util.Iterator"},
"pattern":"\\QMETA-INF/services/org.junit.platform.launcher.TestExecutionListener\\E"
}, {
"condition":{"typeReachable":"org.junit.platform.launcher.core.LauncherConfigurationParameters"},
"pattern":"\\Qjunit-platform.properties\\E"
}, {
"condition":{"typeReachable":"org.slf4j.LoggerFactory"},
"pattern":"\\Qorg/slf4j/impl/StaticLoggerBinder.class\\E"
}, {
"condition":{"typeReachable":"worker.org.gradle.internal.classloader.FilteringClassLoader"},
"pattern":"\\Qorg/slf4j/impl/StaticLoggerBinder.class\\E"
}, {
"condition":{"typeReachable":"java.lang.ClassLoader"},
"pattern":"\\Qresource.txt\\E"
}]},
"bundles":[]
}
----

As you can see, there are lots of resources that you may don't want.
{
{
"reflection": [
{
"condition": {
"typeReached": "java.io.ObjectInputStream"
},
"type": "[Ljava.lang.Object;"
},
{
"condition": {
"typeReached": "java.io.ObjectInputStream"
},
"type": "java.util.LinkedHashSet"
},
{
"condition": {
"typeReached": "org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor"
},
"type": "org.example.NativeTests"
},
{
"condition": {
"typeReached": "org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor"
},
"type": "org.example.NativeTests",
"allDeclaredFields": true
},
{
"condition": {
"typeReached": "org.junit.jupiter.engine.descriptor.ExtensionUtils"
},
"type": "org.example.NativeTests"
},
...
],
"resources": [
{
"condition": {
"typeReached": "org.junit.platform.launcher.core.LauncherFactory"
},
"glob": "META-INF/services/org.junit.platform.engine.TestEngine"
},
{
"condition": {
"typeReached": "java.lang.ClassLoader"
},
"glob": "TestResource.txt"
},
...
],
"bundles": [],
"jni": [
{
"condition": {
"typeReached": "java.net.InetAddress"
},
"type": "java.lang.Boolean",
"methods": [
{
"name": "getBoolean",
"parameterTypes": [
"java.lang.String"
]
}
]
}
]
}
----

As you can see, there are lots of entries that you may don't want.
To reduce the amount of generated metadata, we will use the following `user-code-filter.json`:

[source,json,subs="verbatim,attributes", role="multi-language-sample"]
----
{
"rules": [
{"includeClasses": "**"},
{"excludeClasses": "java.time.zone.**"},
{"excludeClasses": "org.junit.platform..**"}
{"excludeClasses": "org.junit.**"},
{"excludeClasses": "org.gradle.**"},
{"excludeClasses": "worker.org.gradle.**"},
{"excludeClasses": "org.slf4j.**"},
{"excludeClasses": "java.**"}
]
}
----

After we regenerate the metadata with the new filter, `resource-config.json` generated on the same example as above will look like this:
[WARNING]
====
Be careful to remove only redundant metadata and keep metadata necessary for your project.
====

After we regenerate the metadata with the new filter, `reachability-metadata.json` generated on the same example as above will look like this:

[source,json,subs="verbatim,attributes", role="multi-language-sample"]
----
{
"resources":{
"includes":[{
"condition":{"typeReachable":"jdk.internal.logger.BootstrapLogger$DetectBackend$1"},
"pattern":"\\QMETA-INF/services/java.lang.System$LoggerFinder\\E"
}, {
"condition":{"typeReachable":"jdk.internal.logger.LoggerFinderLoader"},
"pattern":"\\QMETA-INF/services/java.lang.System$LoggerFinder\\E"
}, {
"condition":{"typeReachable":"java.nio.channels.spi.SelectorProvider$Holder"},
"pattern":"\\QMETA-INF/services/java.nio.channels.spi.SelectorProvider\\E"
}, {
"condition":{"typeReachable":"java.util.Iterator"},
"pattern":"\\QMETA-INF/services/org.junit.platform.launcher.TestExecutionListener\\E"
}, {
"condition":{"typeReachable":"org.slf4j.LoggerFactory"},
"pattern":"\\Qorg/slf4j/impl/StaticLoggerBinder.class\\E"
}, {
"condition":{"typeReachable":"worker.org.gradle.internal.classloader.FilteringClassLoader"},
"pattern":"\\Qorg/slf4j/impl/StaticLoggerBinder.class\\E"
}, {
"condition":{"typeReachable":"java.lang.ClassLoader"},
"pattern":"\\Qresource.txt\\E"
}]},
"bundles":[]
"reflection": [
{
"condition": {
"typeReached": "org.example.NativeTests"
},
"type": "org.example.NativeTests$Person",
"allDeclaredFields": true
},
{
"condition": {
"typeReached": "sun.security.jca.GetInstance"
},
"type": "sun.security.provider.SHA",
"methods": [
{
"name": "<init>",
"parameterTypes": []
}
]
}
],
"resources": [
{
"condition": {
"typeReached": "org.example.NativeTests"
},
"glob": "TestResource.txt"
}
],
"bundles": []
}
----

As you can see there are no more entries that contain classes from `org.junit.platform.launcher` (as their condition) for example.
As you can see there are no more entries that contain classes from `org.junit` (as their condition) for example.


[[maintain-generated-metadata]]
Expand Down Expand Up @@ -732,12 +769,14 @@ In some cases, when you want to maintain multiple projects that share common met
https://github.com/oracle/graalvm-reachability-metadata/blob/master/CONTRIBUTING.md[Contributing to the repository] should be simple:

- Clone repository locally:

[source,bash,subs="verbatim,attributes", role="multi-language-sample"]
----
git clone [email protected]:oracle/graalvm-reachability-metadata.git
----

- generate metadata and test stubs (replace with the GAV coordinates of library you are providing metadata for):

[source,bash,subs="verbatim,attributes", role="multi-language-sample"]
----
./gradlew scaffold --coordinates com.example:my-library:1.0.0
Expand Down Expand Up @@ -769,11 +808,64 @@ When the Native Image build is completed, you will find a path to the generated
```
------------------------------------------------------------------------------------
Build artifacts:
/build/native/nativeCompile/main (executable)
/build/native/nativeCompile/main-build-report.html (build_info)
/tmp/build/native/nativeCompile/main (executable)
/tmp/build/native/nativeCompile/main-build-report.html (build_info)
====================================================================================
```
You can read more about build report features https://www.graalvm.org/latest/reference-manual/native-image/overview/build-report/[here].
[NOTE]
Note that Build Report features vary depending on a GraalVM version you use.
[[example]]
== Example
Whole guide has been tested on the following test class:
[source,java,subs="verbatim,attributes", role="multi-language-sample"]
----
import org.junit.jupiter.api.Test;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.lang.reflect.Field;
import java.util.Arrays;
import java.util.List;

import static org.junit.jupiter.api.Assertions.assertTrue;

public class NativeTests {

private static final List<String> resources = List.of("/TestResource.txt");

@Test
public void resourceTest() {
try (InputStream is = NativeTests.class.getResourceAsStream(resources.get(0))) {
if (is != null) {
var reader = new BufferedReader(new InputStreamReader(is));
reader.lines().forEach(System.out::println);
} else {
throw new IOException("Cannot read content of: " + resources.get(0));
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}

private static class Person {
private String name;
private String surname;
}

@Test
public void reflectionTest() {
Object person = new Person();
Field[] fields = person.getClass().getDeclaredFields();
List<String> actualFieldNames = Arrays.stream(fields).map(Field::getName).toList();

assertTrue(actualFieldNames.containsAll(Arrays.asList("name", "surname")));
}
}
----
Loading

0 comments on commit 46751a3

Please sign in to comment.