contribution
contribution copied to clipboard
checkstyle
Regression report displays exceptions inconsistently
Discovered at https://github.com/checkstyle/checkstyle/pull/14791#issuecomment-2053817456
Regression report displays exceptions inconsistently.
Sometimes there is a full stacktrace for an exception:
https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/8039a13_2024203306/reports/diff/elasticsearch/index.html#A139
Sometimes there is only a line of the exception name:
https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/8039a13_2024203306/reports/diff/elasticsearch/index.html#A1
Both these two cases came from the same report generation: https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/8039a13_2024203306/reports/diff/index.html
To reproduce
- Create a new branch contains the fix for https://github.com/checkstyle/checkstyle/pull/14791.
- Create a new branch from the commit before https://github.com/checkstyle/checkstyle/pull/14791 is merged.
- Prepare the following configuration: Diff Regression config: https://gist.githubusercontent.com/Lmh-java/95ba8a3850cf6d670b2d65262531fb35/raw/7f17357bf8ec422286181c2d470ed578f8da485d/magic-number-config.xml project lists: use the default (or only the following line for bare minimum)
elasticsearch|git|https://github.com/elastic/elasticsearch|v1.5.2||
- run
groovy diff.groovy --localGitRepo [location of checkstyle project] --baseBranch [branch name without fix] --patchBranch [branch name with fix] --config magic-number-config.xml --listOfProjects test.properties -xm -Dcheckstyle.failsOnError=false --allowExcludes
Some analysis
After investigation, this difference presents before the results are sent to patch-diff-report-tool to compare. This difference is presented in checkstyle-result.xml. For example:
<file name="/Users/liminghao/Programs/IdeaProjects/contribution/checkstyle-tester/repositories/elasticsearch/src/test/java/org/elasticsearch/bwcompat/UnicastBackwardsCompatibilityTest.java">
<error line="1" severity="error" message="Got an exception - java.lang.NullPointerException: Cannot invoke "com.puppycrawl.tools.checkstyle.api.DetailAST.getType()" because "node" is null #x9;at com.puppycrawl.tools.checkstyle.checks.coding.MagicNumberCheck.isFieldDeclaration(MagicNumberCheck.java:390) #x9;at com.puppycrawl.tools.checkstyle.checks.coding.MagicNumberCheck.visitToken(MagicNumberCheck.java:219) #x9;at com.puppycrawl.tools.checkstyle.TreeWalker.notifyVisit(TreeWalker.java:335) #x9;at com.puppycrawl.tools.checkstyle.TreeWalker.processIter(TreeWalker.java:408) #x9;at com.puppycrawl.tools.checkstyle.TreeWalker.walk(TreeWalker.java:273) #x9;at com.puppycrawl.tools.checkstyle.TreeWalker.processFiltered(TreeWalker.java:154) #x9;at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.process(AbstractFileSetCheck.java:101) #x9;at com.puppycrawl.tools.checkstyle.Checker.processFile(Checker.java:340) #x9;at com.puppycrawl.tools.checkstyle.Checker.processFiles(Checker.java:299) #x9;at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:226) #x9;at org.apache.maven.plugins.checkstyle.exec.DefaultCheckstyleExecutor.executeCheckstyle(DefaultCheckstyleExecutor.java:202) #x9;at org.apache.maven.plugins.checkstyle.AbstractCheckstyleReport.executeReport(AbstractCheckstyleReport.java:533) #x9;at org.apache.maven.plugins.checkstyle.CheckstyleReport.executeReport(CheckstyleReport.java:57) #x9;at org.apache.maven.reporting.AbstractMavenReport.generate(AbstractMavenReport.java:255) #x9;at org.apache.maven.plugins.site.render.ReportDocumentRenderer.renderDocument(ReportDocumentRenderer.java:226) #x9;at org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.render(DefaultSiteRenderer.java:348) #x9;at org.apache.maven.plugins.site.render.SiteMojo.renderLocale(SiteMojo.java:194) #x9;at org.apache.maven.plugins.site.render.SiteMojo.execute(SiteMojo.java:143) #x9;at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:328) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:316) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:174) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:75) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:162) #x9;at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39) #x9;at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:159) #x9;at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105) #x9;at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73) #x9;at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53) #x9;at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118) #x9;at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261) #x9;at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173) #x9;at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101) #x9;at org.apache.maven.cli.MavenCli.execute(MavenCli.java:906) #x9;at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:283) #x9;at org.apache.maven.cli.MavenCli.main(MavenCli.java:206) #x9;at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) #x9;at java.base/java.lang.reflect.Method.invoke(Method.java:580) #x9;at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283) #x9;at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226) #x9;at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407) #x9;at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348) " source="com.puppycrawl.tools.checkstyle.Checker"/>
</file>
<file name="/Users/liminghao/Programs/IdeaProjects/contribution/checkstyle-tester/repositories/elasticsearch/src/test/java/org/elasticsearch/script/ScriptFieldTests.java">
<error line="1" severity="error" message="Got an exception - java.lang.NullPointerException " source="com.puppycrawl.tools.checkstyle.Checker"/>
</file>
This means, when the checkstyle report is generated, this difference is already there. I also noticed that the single line stacktrace has 
(\n) at the very end. This might be the cause of the incorrect termination of the line.
Please share a reliable project/ config combination to reproduce this with.
Please share a reliable project/ config combination to reproduce this with.
Done, updated in the issue description. Please see whether that's sufficient to reproduce.