microstream-one
GraalVM native-image works only in fallback mode - without: Could not obtain access to "theUnsafe"
GraalVM native-image works only in fallback mode - without: Could not obtain access to "theUnsafe"
tested on: GraalVM 20.1.0
GraalVM native-image works only in fallback mode. When I compile the jar with microstream with --no-fallback option, i receive the following message.
ash-4.2# ./graal-demo-0.0.1-SNAPSHOT-jar-with-dependencies
Exception in thread "main" java.lang.Error: Could not obtain access to "theUnsafe"
at one.microstream.memory.sun.JdkInternals.getMemoryAccess(JdkInternals.java:61)
at one.microstream.memory.sun.JdkInternals.<clinit>(JdkInternals.java:35)
at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
at java.lang.Class.ensureInitialized(DynamicHub.java:499)
at one.microstream.memory.sun.JdkMemoryAccessor.guaranteeUsability(JdkMemoryAccessor.java:36)
at one.microstream.memory.XMemory.guaranteeUsability(XMemory.java:292)
at one.microstream.persistence.binary.types.Binary.<clinit>(Binary.java:123)
at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:350)
at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:270)
at java.lang.Class.ensureInitialized(DynamicHub.java:499)
at one.microstream.persistence.binary.internal.AbstractBinaryHandlerCustom.chars(AbstractBinaryHandlerCustom.java:72)
at one.microstream.java.lang.BinaryHandlerString.<init>(BinaryHandlerString.java:30)
at one.microstream.java.lang.BinaryHandlerString.New(BinaryHandlerString.java:16)
at one.microstream.persistence.binary.types.BinaryPersistence.createNativeHandlersValueTypes(BinaryPersistence.java:202)
at one.microstream.persistence.binary.types.BinaryPersistence.createDefaultCustomTypeHandlerRegistry(BinaryPersistence.java:141)
at one.microstream.persistence.binary.types.BinaryPersistenceFoundation$Default.lambda$1(BinaryPersistenceFoundation.java:230)
at one.microstream.persistence.binary.types.BinaryPersistenceFoundation$Default.ensureCustomTypeHandlerRegistry(BinaryPersistenceFoundation.java:242)
at one.microstream.persistence.types.PersistenceFoundation$Default.getCustomTypeHandlerRegistry(PersistenceFoundation.java:970)
at one.microstream.persistence.types.PersistenceFoundation$Default.ensureRootsProvider(PersistenceFoundation.java:2441)
at one.microstream.persistence.types.PersistenceFoundation$Default.getRootsProvider(PersistenceFoundation.java:1335)
at one.microstream.storage.types.EmbeddedStorageFoundation$Default.createEmbeddedStorageManager(EmbeddedStorageFoundation.java:727)
at one.microstream.storage.types.EmbeddedStorage.createAndStartStorageManager(EmbeddedStorage.java:595)
at one.microstream.storage.types.EmbeddedStorage.start(EmbeddedStorage.java:481)
at one.microstream.Application.main(Application.java:28)
Caused by: java.lang.NoSuchFieldException: theUnsafe
at java.lang.Class.getDeclaredField(DynamicHub.java:2070)
at one.microstream.memory.sun.JdkInternals.getMemoryAccess(JdkInternals.java:55)
... 24 more
So does it means, we are able to build native-image with microstream, but this works in fallback mode (in jvm). It requires JVM and starts the whole JVM too.
If there is an equivalent to Unsafe in Graal, then we can write a suitable MemoryAccessor for it. The failed test uses JDK (Oracle JVM) specific code, see stacktrace. Maybe Unsafe even exists and only that accessing field ("theUnsafe") does not.
So the next step would be to evaluate how to use Unsafe (or an equivalent) on Graal. I can't imagine there is no solution for that because a lot of libraries (and very public ones) use it. No solution would mean none of those libraries would work.
https://medium.com/graalvm/instant-netty-startup-using-graalvm-native-image-generation-ed6f14ff7692 here are some information how native images are built
https://github.com/oracle/graal/blob/master/substratevm/REFLECTION.md Docu about reflection in GraalVM
Unsafe accesses The Unsafe class, although its use is discouraged, provides direct access to memory of Java objects. The Unsafe.objectFieldOffset() method provides the offset of a field within a Java object. This information is not available by default, but can be enabled with the allowUnsafeAccess attribute in the reflection configuration, for example:
"fields" : [ { "name" : "hash", "allowUnsafeAccess" : true }, ... ]
Note that offsets that are queried during native image generation can be different to the offsets at runtime.
https://medium.com/graalvm/instant-netty-startup-using-graalvm-native-image-generation-ed6f14ff7692 here are some information how native images are built
I don't know if this information is meant as a reply to my comment. If it is, I fail to see the connection. In any case, I wrote a tiny test class to progress with the problem at hand.
The following java class (".txt" only to circumvent the naive limitations of github) tries to access sun.misc.Unsafe as it would/should work on an Oracle JVM or compatible (e.g. android). This class has absolutely nothing to do with microstream, only with Java itself. The accessing method isn't even written by me. I just copied it from the internet.
If it does not work on Graal, then the question is: How does it have to be changed to access sun.misc.Unsafe on Graal? Or, if sun.misc.Unsafe does not exist on Graal (which I strongly doubt, see comment above), what is an equivalent replacement for low-level memory operations on Graal?
I can't imagine there is no solution for that because a lot of libraries (and very public ones) use it. No solution would mean none of those libraries would work.
This is a native-image. This is an option in GraalVM to create code that runs directly without a JVM. Many frameworks are not yet able to run under native-image. For example, Spring is only now adding this option and is still in the experimental phase. Therefore, I do not report this issue as a bug. It is more for the purposes of documentation and documents to decide whether we want to support this feature at all. It is possible that this is only a short-term hype. It works just below the GraalVM Microstream. Fallback mode is on by default when building a native-image.
https://medium.com/graalvm/instant-netty-startup-using-graalvm-native-image-generation-ed6f14ff7692 here are some information how native images are built
I don't know if this information is meant as a reply to my comment. If it is, I fail to see the connection. In any case, I wrote a tiny test class to progress with the problem at hand.
The following java class (".txt" only to circumvent the naive limitations of github) tries to access sun.misc.Unsafe as it would/should work on an Oracle JVM or compatible (e.g. android). This class has absolutely nothing to do with microstream, only with Java itself. The accessing method isn't even written by me. I just copied it from the internet.
If it does not work on Graal, then the question is: How does it have to be changed to access sun.misc.Unsafe on Graal? Or, if sun.misc.Unsafe does not exist on Graal (which I strongly doubt, see comment above), what is an equivalent replacement for low-level memory operations on Graal?
Sorry, bad issue. I was just collecting the links I found and wanted to include them in my issue, in the test project, and I overlooked them. My fault.
If it does not work on Graal, then the question is: How does it have to be changed to access sun.misc.Unsafe on Graal? Or, if sun.misc.Unsafe does not exist on Graal (which I strongly doubt, see comment above), what is an equivalent replacement for low-level memory operations on Graal?
Unfortunately, I can't find much information about that. Everywhere, there is only a brief mention that those class members that are accessed via unsafe must be defined statically in the configuration file.
Native image has a lot of limitations, see this page. Among them is unsafe. https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md
If it does not work on Graal, then the question is: How does it have to be changed to access sun.misc.Unsafe on Graal? Or, if sun.misc.Unsafe does not exist on Graal (which I strongly doubt, see comment above), what is an equivalent replacement for low-level memory operations on Graal?
Unfortunately, I can't find much information about that. Everywhere, there is only a brief mention that those class members that are accessed via unsafe must be defined statically in the configuration file.
Native image has a lot of limitations, see this page. Among them is unsafe. https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md
The limitation mentioned there only applies to accessing fields via Unsafe (direct memory access). While microstream does that pretty much, there is already an alternative to it: accessing the fields via reflection instead. This fallback was necessary for android and already exists.
But the link is interesting in another way: If accessing fields via Unsafe needs special treatment, it means that sun.misc.Unsafe exists and is usable in general on GraalVM. The only question is, how to access it. It might be as simple as the hidden field not being named "the Unsafe" but just "unsafe". Or "internal" or whatever. Maybe that is all it takes.
I googled a little, as well and I, too, only find brief references to Unsafe. One article describes an example for using Unsafe, but doesn't use it at all. Sometimes ... it's ... strange.