unable to work with objects created using net.imglib2.loops.ClassCopyProvider

[carlocastoldi]

Hi!

while working on this https://github.com/BIOP/ijp-atlas/issues/7#issuecomment-2619511900, I got stuck when i tried to get a java.net.imglib2.display.RealARGBColorConverter object built with net.imglib2.display.RealARGBColorConverterFactory. The factory uses net.imglib2.loops.ClassCopyProvider, which internally instantiate the objects with net.imglib2.loops.ClassCopyLoader, a different classloader from the one used by scyjava.

In the end, jpype crashes with:

Traceback (most recent call last):
  File "org.jpype.manager.TypeManager.java", line 0, in org.jpype.manager.TypeManager.findClassForObject
Exception: Java Exception

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  [...]
java.lang.java.lang.IllegalAccessError: java.lang.IllegalAccessError:
    failed to access class net.imglib2.display.RealARGBColorConverterFactory from class net.imglib2.display.RealARGBColorConverterFactory$Imp
    (
        net.imglib2.display.RealARGBColorConverterFactory is in unnamed module of loader 'app';
        net.imglib2.display.RealARGBColorConverterFactory$Imp is in unnamed module of loader net.imglib2.loops.ClassCopier$ClassCopyLoader @25464154
    )

Can you please help me? Do you think there is an ~easy workaround (perhaps telling jpype to avoid trying to find the class for that object?)

[Thrameos]

Did you try to use the java module system to open the package to jpype? Not sure if that will work but perhaps it is possible. I am trying to rework jpype to increase its privilege level. Unfortunately the agent method failed in 1.5.1. Maybe try modules. These are all artificial barriers added for security.

[ctrueden]

Thanks @Thrameos for chiming in!

@carlocastoldi Additional ideas:

• There may be --add-opens arguments we can pass to the JVM initialization to unlock the access.
• You could probably avoid the illegal access errors by running with OpenJDK 11 rather than 17+ (they were merely warnings before that).

@maarzt @tpietzsch Have you encountered any similar (JPMS) problems with ImgLib2's ClassCopier when using OpenJDK 17+? Any suggestions for dealing with it?

[Thrameos]

Long term I am dropping JDK 1.8 support and adding modules. Any modules loaded at startJVM will likely be forced open to a jpype module. Hopefully, that prevents this sort of problem in the future. Though as I am still starting the module process, it is going to be a while.

I am always happy, whether I have the time is sadly another question.

[carlocastoldi]

Thank you a lot for the rapidity with which you came here to help.

Unfortunately (or fortunately) the maintainer of ABBA came and saved me, finding a workaround that I didn't explore cause I am pretty new into ImageJ scripting. In the end I fixed the problem avoiding to use a public, but problematic, ImgLib2 function (i.e. SourceAndConverter.getConverter).

As far as I understand this issue is still relevant, though, right? I would like to help on this, but I am very new to all these libraries and am short on time this week. I hope you won't take it as disrespectful if I won't keep actively working on this. If you want, however, I am available to reproduce the problem on my system until you have problem setting up a minimal example.

[Thrameos]

Tis the nature of open-source work. People only work on what scratches their itch. Setting up minimal examples is an art itself, and one can’t expect everyone will be willing to spend the time to do so. If you do get a reproducer of the general issue that you can post to jpype then I can fix the issue for everyone. If not, then it will just have to wait until the next guy encounters a similar problem (though in many cases the next guy is me).

[carlocastoldi]

this is still using imagej, scyjava and sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper.createSourceAndConverter. For today i think it's the maximum i can do...

# conda create -c conda-forge -n pyimagej python=3.10 openjdk=11 pip maven pyimagej
# conda activate pyimagej

import imagej
import numpy as np
from jpype.types import JString
from scyjava import jimport

dependencies = [
    'sc.fiji:bigdataviewer-playground:0.11.0',
    #'net.imglib2:imglib2-realtransform:4.0.3',
]
ij = imagej.init(dependencies)

AffineTransform3D = jimport('net.imglib2.realtransform.AffineTransform3D')
RandomAccessibleIntervalSource = jimport('bdv.util.RandomAccessibleIntervalSource')
Util = jimport('net.imglib2.util.Util')
SourceAndConverterHelper = jimport('sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper')

array = np.full((1,), fill_value=0.9)
img = ij.py.to_java(array)
pixel_type = Util.getTypeFromInterval(img)
rai_source = RandomAccessibleIntervalSource(img, pixel_type, AffineTransform3D(), JString("test"))
sac = SourceAndConverterHelper.createSourceAndConverter(rai_source)
print(sac.getConverter())

[carlocastoldi]

i feel like i am near with this one, by eliminating the need of pyimagej

# conda create -c conda-forge -n test python=3.10 openjdk=11 scyjava
# conda activate test

import scyjava as sj
from jpype.types import JString

sj.config.add_option("-Djava.awt.headless=true")
if hasattr(sj, "jvm_version") and sj.jvm_version()[0] >= 9:
    sj.config.add_option("--add-opens=java.base/java.lang=ALL-UNNAMED")
    sj.config.add_option("--add-opens=java.base/java.util=ALL-UNNAMED")
    sj.config.add_option("--add-opens=java.desktop/sun.awt.X11=ALL-UNNAMED")

sj.config.endpoints.clear()
sj.config.endpoints.append("sc.fiji:bigdataviewer-playground")
sj.start_jvm()

RandomAccessibleIntervalSource = sj.jimport("bdv.util.RandomAccessibleIntervalSource")
DoubleType = sj.jimport("net.imglib2.type.numeric.real.DoubleType")
SingleCellArrayImg = sj.jimport("net.imglib2.cache.img.SingleCellArrayImg")
SourceAndConverterHelper = sj.jimport("sc.fiji.bdvpg.sourceandconverter.SourceAndConverterHelper")

interval = SingleCellArrayImg(1)
rai_source = RandomAccessibleIntervalSource(interval, DoubleType(0), JString("test"))
sac = SourceAndConverterHelper.createSourceAndConverter(rai_source)
print(sac.getConverter())

[carlocastoldi]

fyi i think i managed to have a minimal example for this problem which satisfies my requirements. I hope it satisfies yours too

I have updated the above comment

[ctrueden]

Thanks @carlocastoldi! I am able to reproduce quickly using your above script. In addition, I verified that the problem happens even with OpenJDK 8, and also produced the full Java stacktrace using the scyjava.jstacktrace function:

net.imglib2.display.RealARGBColorConverterFactory$Imp
	at java.lang.Class.getDeclaringClass0(Native Method)
	at java.lang.Class.getDeclaringClass(Class.java:1235)
	at java.lang.Class.getEnclosingClass(Class.java:1277)
	at java.lang.Class.getSimpleBinaryName(Class.java:1443)
	at java.lang.Class.getSimpleName(Class.java:1309)
	at java.lang.Class.isAnonymousClass(Class.java:1411)
	at org.jpype.manager.TypeManager.findClass(Unknown Source)
	at org.jpype.manager.TypeManager.findClassForObject(Unknown Source)

Now I'll see if the same problem occurs even without JPype/Python...

Edit: OK yep, it works in pure Java. So it has something to do with how JPype does class loading. And it's not just a JPMS thing, because it happens with OpenJDK 8 as well.

[ctrueden]

@Thrameos @carlocastoldi I tried to boil down the example to a JPype-only one with minimal dependencies (only imglib2, which itself has no dependencies), but was unsuccessful in replicating the problem at that level so far. Here is my "failed" attempt (it runs to completion 😆):

loop.py

# conda create -c conda-forge -n scyjava-issue-72 python=3.10 openjdk=8 jpype1=1.5.2
# conda activate scyjava-issue-72
# curl -fL https://search.maven.org/remotecontent\?filepath\=net/imglib2/imglib2/7.1.4/imglib2-7.1.4.jar > imglib2.jar

import jpype

jpype.addClassPath("imglib2.jar")
jpype.startJVM()

########

from jpype import imports

from java.util.function import Consumer

from net.imglib2.img.array import ArrayImgs
from net.imglib2.loops import LoopBuilder
from net.imglib2.util import Util

img = ArrayImgs.ints([1, 2, 3, 4], [2, 2])

print("Before: ", end="")
img.stream().forEach(lambda x: print(x, end=" "))
print()

@jpype.JImplements(Consumer)
class Doubler(object):
    @jpype.JOverride
    def accept(self, t):
        t.setReal(t.getRealDouble() * 2)

LoopBuilder.setImages(img).forEachPixel(Doubler())

print("After: ", end="")
img.stream().forEach(lambda x: print(x, end=" "))
print()

But I haven't debugged into the code yet; I'm not certain that ImgLib2's LoopBuilder is what's actually invoking the ClassCopier thing under the hood. I may find time to dig more later and refine this example to actually fail in the same way without needing the huge set of dependencies brought in by bigdataviewer-playground... but no more time left today. ⌛

[Thrameos]

I will give it a try to see if I can replicate something with my modules patch. I generated a broken case in pure java using a package the exports one package containing a protected member which was in a non-exported package. The result was an object for which any attempt to access it would fail unless it was first cast to an accessible base class. However, I am not sure yet how JPype would respond to such a weird set of permissions.

[ctrueden]

@Thrameos Like I said, the same failure happens even with OpenJDK 8. So I don't think it's JPMS-related...

[Thrameos]

Given the error happen while accessing class structures the best we can do is add exception handling and force the object type to the parent. From the documentation, isAnonymousClass is not supposed to throw.