Tai-e icon indicating copy to clipboard operation
Tai-e copied to clipboard
verified publisher icon pascal-lab

Null pointer exception when running BenchmarkRunner on benchmark `columba-1.4`

Open enochii opened this issue 1 year ago • 5 comments

Overall Description

Hello, I am using Tai-e's BenchmarkRunner to run the benchmark columba-1.4, and encounter a null pointer exception.

Current Behavior

image

When running BenchmarkRunner on benchmark columba-1.4, a null pointer exception is thrown. Here is the output:

> Task :BenchmarkRunner.main()

Analyzing columba-1.4
Tai-e starts ...
Output directory: C:\code\Tai-e\output
Writing options to C:\code\Tai-e\output\options.yml
Writing log to C:\code\Tai-e\output\tai-e.log
Writing analysis plan to C:\code\Tai-e\output\tai-e-plan.yml
WorldBuilder starts ...
The world cache mode is enabled.
Loading the world cache from C:\code\Tai-e\cache\world-cache-685358273.bin
[Load the world cache] elapsed time: 6.82s
12396 classes with 115878 methods in the world
WorldBuilder finishes, elapsed time: 6.84s
pta starts ...
Using reflection log from C:\code\Tai-e\java-benchmarks\columba\1.4\refl.log
Reflective target 'byte[]' for Class.forName is not found
Reflective target 'java.lang.String[]' for Class.forName is not found
Reflective caller class 'sun.nio.cs.AbstractCharsetProvider' is absent
Reflective target 'sun.security.pkcs.SignerInfo[]' for Class.forName is not found
Reflective target '<com.sun.crypto.provider.BlowfishCipher: void <init>()>' for Constructor.newInstance is not found
Reflective target '<com.sun.crypto.provider.SunJCE: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Big5: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Big5_HKSCS: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_CN: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_JP: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_KR: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_TW: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ExtendedCharsets: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.GB18030: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.GBK: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO2022_JP: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO2022_KR: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO_8859_3: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Johab: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.MS1258: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.MS932: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.SJIS: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.TIS_620: void <init>()>' for Constructor.newInstance is not found
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "Object.equals(Object)" because "superclass" is null
	at pascal.taie.language.classes.ClassHierarchyImpl.isSubclass(ClassHierarchyImpl.java:397)
	at pascal.taie.language.type.TypeSystemImpl.isSubtype(TypeSystemImpl.java:180)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.isAssignable(TypeFilter.java:66)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.lambda$apply$0(TypeFilter.java:58)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.apply(TypeFilter.java:59)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.addPFGEdge(DefaultSolver.java:787)
	at pascal.taie.analysis.pta.core.solver.Solver.addPFGEdge(Solver.java:173)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.lambda$processArrayStore$7(DefaultSolver.java:430)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.processArrayStore(DefaultSolver.java:425)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.analyze(DefaultSolver.java:320)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.solve(DefaultSolver.java:246)
	at pascal.taie.analysis.pta.PointerAnalysis.runAnalysis(PointerAnalysis.java:119)
	at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:107)
	at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:64)
	at pascal.taie.analysis.AnalysisManager.runProgramAnalysis(AnalysisManager.java:148)
	at pascal.taie.analysis.AnalysisManager.runAnalysis(AnalysisManager.java:135)
	at pascal.taie.analysis.AnalysisManager.lambda$execute$0(AnalysisManager.java:104)
	at pascal.taie.util.Timer.runAndCount(Timer.java:93)
	at pascal.taie.analysis.AnalysisManager.lambda$execute$1(AnalysisManager.java:103)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at pascal.taie.analysis.AnalysisManager.execute(AnalysisManager.java:102)
	at pascal.taie.Main.executePlan(Main.java:152)
	at pascal.taie.Main.lambda$main$0(Main.java:60)
	at pascal.taie.util.Timer.lambda$runAndCount$0(Timer.java:112)
	at pascal.taie.util.Timer.runAndCount(Timer.java:93)
	at pascal.taie.util.Timer.runAndCount(Timer.java:111)
	at pascal.taie.util.Timer.runAndCount(Timer.java:107)
	at pascal.taie.Main.main(Main.java:51)
	at pascal.taie.analysis.pta.BenchmarkRunner.run(BenchmarkRunner.java:72)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at pascal.taie.analysis.pta.BenchmarkRunner.runAll(BenchmarkRunner.java:67)
	at pascal.taie.analysis.pta.BenchmarkRunner.main(BenchmarkRunner.java:60)

> Task :BenchmarkRunner.main() FAILED

Execution failed for task ':BenchmarkRunner.main()'.
> Process 'command 'C:\Users\enochii\.jdks\graalvm-ce-17\bin\java.exe'' finished with non-zero exit value 1

Expected Behavior

No null pointer exception.

Tai-e Version

083f0cc2368162631cc8e7b58cf96634fbb4fe15

Tai-e Arguments

-cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci

JDK Version

graalvm-ce-17

System Environment

Windows 10 x64

Additional Information

optiona.yml:

optionsFile: null
printHelp: false
classPath:
- java-benchmarks/columba/1.4/lib
- java-benchmarks/columba/1.4/lib/bsh-2.0b2.jar
- java-benchmarks/columba/1.4/lib/checkstyle-all-3.4.jar
- java-benchmarks/columba/1.4/lib/commons-cli-1.0.jar
- java-benchmarks/columba/1.4/lib/commons-logging.jar
- java-benchmarks/columba/1.4/lib/core-renderer.jar
- java-benchmarks/columba/1.4/lib/flexdock-0.4.1.jar
- java-benchmarks/columba/1.4/lib/forms-1.0.5.jar
- java-benchmarks/columba/1.4/lib/frapuccino-1.0pre1.jar
- java-benchmarks/columba/1.4/lib/groovy-1.0.jar
- java-benchmarks/columba/1.4/lib/ical4j-0.9.18.jar
- java-benchmarks/columba/1.4/lib/jarbundler-1.9.jar
- java-benchmarks/columba/1.4/lib/jcoverage.jar
- java-benchmarks/columba/1.4/lib/jdom-1.0.jar
- java-benchmarks/columba/1.4/lib/je-2.1.30.jar
- java-benchmarks/columba/1.4/lib/jhall-2.0_02.jar
- java-benchmarks/columba/1.4/lib/jpim.jar
- java-benchmarks/columba/1.4/lib/jreleaseinfo-1.2.0.jar
- java-benchmarks/columba/1.4/lib/jscf-0.3.jar
- java-benchmarks/columba/1.4/lib/junit.jar
- java-benchmarks/columba/1.4/lib/jwizz-0.1.3.jar
- java-benchmarks/columba/1.4/lib/jython-2.2a1.jar
- java-benchmarks/columba/1.4/lib/looks-2.0.2.jar
- java-benchmarks/columba/1.4/lib/lucene-1.4.3.jar
- java-benchmarks/columba/1.4/lib/macchiato-1.0pre1.jar
- java-benchmarks/columba/1.4/lib/mac_mock.jar
- java-benchmarks/columba/1.4/lib/migcalendar-5.6.jar
- java-benchmarks/columba/1.4/lib/quaqua.jar
- java-benchmarks/columba/1.4/lib/ristretto-1.2-all.jar
- java-benchmarks/columba/1.4/lib/swingx.jar
- java-benchmarks/columba/1.4/lib/tagsoup-1.0rc3.jar
- java-benchmarks/columba/1.4/lib/usermanual.jar
- java-benchmarks/columba/1.4/native/linux
appClassPath:
- java-benchmarks/columba/1.4/columba.jar
mainClass: org.columba.core.main.Main
inputClasses: []
javaVersion: 6
prependJVM: false
allowPhantom: true
worldBuilderClass: pascal.taie.frontend.soot.SootWorldBuilder
outputDir: output
preBuildIR: false
worldCacheMode: true
scope: APP
nativeModel: true
planFile: null
analyses:
  pta: advanced:null;reflection-log:java-benchmarks\columba\1.4\refl.log;cs:ci;distinguish-string-constants:null;merge-string-objects:false;reflection-inference:null
  may-fail-cast: ""
  poly-call: ""
onlyGenPlan: false
keepResult:
- $KEEP-ALL

enochii avatar May 08 '24 05:05 enochii

Sorry for the delayed response. This problem seems comprehensive and relates to many different parts of Tai-e. I guess this problem may be introduced by the commit 591ecc8709c48c0dad23170a6279b4f4b03cc2f3 to 648b3a6acc908cbc8bf0195322051e05b3ca0e72. Can you use the latest release or the commit 99501551b49b1fd2f2904bf43fd13c727e16c9cf?

We'll fix it as soon as possible.

ayanamists avatar May 23 '24 07:05 ayanamists

Thanks a lot! When it is fixed, please kindly remind me here if it's not too much trouble~

enochii avatar May 28 '24 17:05 enochii

Have you made any progress on this issue? I'm having the same problem.

q983868425 avatar Aug 15 '24 02:08 q983868425

After some basic debugging, I discovered that the type org.columba.ristretto.auth.mechanism.LoginMechanism is missing its corresponding class, indicating a bug in Soot's frontend robustness—potentially related to reference analysis.

For the current issue (i.e., running Tai-e with -cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci), the following modification is needed:

In java-benchmarks/benchmark-info.yml:

-  apps: [ columba/1.4/columba.jar ]
+  apps: [ columba/1.4/columba.jar, columba/1.4/lib/ristretto-1.2-all.jar ]

This also serves as a reminder that until our new frontend is released, we need to manually specify the actual application classes using the -acp or --input-classes options.

zhangt2333 avatar Aug 15 '24 07:08 zhangt2333

After some basic debugging, I discovered that the type org.columba.ristretto.auth.mechanism.LoginMechanism is missing its corresponding class, indicating a bug in Soot's frontend robustness—potentially related to reference analysis.

For the current issue (i.e., running Tai-e with -cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci), the following modification is needed:

In java-benchmarks/benchmark-info.yml:

-  apps: [ columba/1.4/columba.jar ]
+  apps: [ columba/1.4/columba.jar, columba/1.4/lib/ristretto-1.2-all.jar ]

This also serves as a reminder that until our new frontend is released, we need to manually specify the actual application classes using the -acp or --input-classes options.

Thank you for your prompt reply. The stability problem of soot you mentioned, I also found it when testing other benchmarks. Some classes in some dependent jars cannot be parsed.

q983868425 avatar Aug 19 '24 03:08 q983868425