secretlinter and jscpd run ordrer is not respected

[lvjp]

Describe the bug Sporadically, when jscpd and secretlinter are run in parallel, you can encounter this error :

[Error: ENOENT: no such file or directory, open '/github/workspace/megalinter-reports/copy-paste/html/jscpd-report.json'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/github/workspace/megalinter-reports/copy-paste/html/jscpd-report.json'
}

To Reproduce Run a lot of time megalinter

Expected behavior Only run secretlinter after a successful run of jscpd

Screenshots

2024-09-09T13:21:14.0247818Z Processing linters on [4] parallel cores… (can be decreased with variable PARALLEL_PROCESS_NUMBER in case of performance issues)
2024-09-09T13:21:15.4280679Z ##[group][32m✅ Linted [MARKDOWN] files with [markdown-link-check] successfully - (0.76s)[0m (expand for details)
2024-09-09T13:21:15.4283160Z - Using [markdown-link-check v3.12.2] https://megalinter.io/8.0.0/descriptors/markdown_markdown_link_check
2024-09-09T13:21:15.4284636Z - MegaLinter key: [MARKDOWN_MARKDOWN_LINK_CHECK]
2024-09-09T13:21:15.4285628Z - Rules config: [.markdown-link-check.json]
2024-09-09T13:21:15.4286686Z - Number of files analyzed: [1]
2024-09-09T13:21:15.4287796Z ##[endgroup]
2024-09-09T13:21:15.4784250Z ##[group][32m✅ Linted [REPOSITORY] files with [devskim] successfully - (1.25s)[0m (expand for details)
2024-09-09T13:21:15.4796006Z - Using [devskim v1.0.33] https://megalinter.io/8.0.0/descriptors/repository_devskim
2024-09-09T13:21:15.4797212Z - MegaLinter key: [REPOSITORY_DEVSKIM]
2024-09-09T13:21:15.4798234Z - Rules config: [.devskim.json]
2024-09-09T13:21:15.4799303Z ##[endgroup]
2024-09-09T13:21:16.0565321Z ##[group][32m✅ Linted [COPYPASTE] files with [jscpd] successfully - (1.26s)[0m (expand for details)
2024-09-09T13:21:16.0623931Z - Using [jscpd v4.0.5] https://megalinter.io/8.0.0/descriptors/copypaste_jscpd
2024-09-09T13:21:16.0625633Z - MegaLinter key: [COPYPASTE_JSCPD]
2024-09-09T13:21:16.0626460Z - Rules config: [.jscpd.json]
2024-09-09T13:21:16.0627661Z ##[endgroup]
2024-09-09T13:21:16.5622894Z ##[group][31m❌ Linted [REPOSITORY] files with [secretlint]: Found 1 error(s) - (0.63s)[0m (expand for details)
2024-09-09T13:21:16.5625067Z - Using [secretlint v8.2.4] https://megalinter.io/8.0.0/descriptors/repository_secretlint
2024-09-09T13:21:16.5626366Z - MegaLinter key: [REPOSITORY_SECRETLINT]
2024-09-09T13:21:16.5627249Z - Rules config: [.secretlintrc.json]
2024-09-09T13:21:16.5628121Z --Error detail:
2024-09-09T13:21:16.5629474Z [Error: ENOENT: no such file or directory, open '/github/workspace/megalinter-reports/copy-paste/html/jscpd-report.json'] {
2024-09-09T13:21:16.5630885Z   errno: -2,
2024-09-09T13:21:16.5631570Z   code: 'ENOENT',
2024-09-09T13:21:16.5632194Z   syscall: 'open',
2024-09-09T13:21:16.5633206Z   path: '/github/workspace/megalinter-reports/copy-paste/html/jscpd-report.json'
2024-09-09T13:21:16.5634769Z }
2024-09-09T13:21:16.5635007Z 
2024-09-09T13:21:16.5635717Z ##[endgroup]

Additional context I run megalinter on GitHub Actions with this configuration :

permissions: {}

jobs:
  mega-linter:
    runs-on: ubuntu-22.04

    steps:
      - name: Checkout Code
        uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

      - name: MegaLinter
        id: ml
        uses: oxsecurity/megalinter@c217fe8f7bc9207062a084e989bd97efd56e7b9a # v8.0.0
        env:
          VALIDATE_ALL_CODEBASE: true
          # GO needs go1.23 which is not supported
          # SPELL is a little bit tricky
          DISABLE: GO,SPELL

      - name: Archive Megalinter artifacts
        if: success() || failure()
        uses: actions/upload-artifact@50769540e7f4bd5e21e526ee35c689e35e0d6874 # v4.4.0
        with:
          name: MegaLinter reports
          path: |
            megalinter-reports
            mega-linter.log

[nvuillam]

@lvjp you can bypass the issue by ignoring megalinter-reports in .jscpd.json

This is the case with default .jscpd.json shipped with MegaLinter :)

https://github.com/oxsecurity/megalinter/blob/main/TEMPLATES/.jscpd.json

[lvjp]

I think that the problem is not for jscpd but in secretlint.

[nvuillam]

@lvjp you're totally right, sorry :)

You can define a .secretlintignore file containing **/megalinter-reports/** :)

[lvjp]

You can define a .secretlintignore file containing **/megalinter-reports/** :)

@nvuillam I think that this definition should be in the default .secretlintignore to be able to directly use it out of the box with the default behavior.

[nvuillam]

@lvjp 100% agree with you, but today MegaLinter architecture contain default config files, not default ignore files :/

It will be faster for you to define one in your repo than to wait that we implement it here ^^ ( except if you're interested to make a Pull Request of course ^^ )

[lvjp]

It's a little bit confusing for me since in the file TEMPLATES/.jscpd.json whe can found :

{"ignore": ["**/node_modules/**"]}

Why we cannot create the file TEMPLATES/.secretlintignore with only content **/megalinter-reports/** ?

[nvuillam]

The architecture does not allow that for now ^^ (the name of ignore file parameter is not configurable)

[lvjp]

Ok, I see.

Maybe we can close this issue and add the label wontfix

[nvuillam]

It can be a future evolution :)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.