docs icon indicating copy to clipboard operation
docs copied to clipboard

`Only the top 5,000 results will be included, prioritized by severity.` does not describe actual deployed behavior

Open jsoref opened this issue 8 months ago • 8 comments

Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning#validating-your-sarif-file

What part(s) of the article would you like to see updated?

The table says:

SARIF data Maximum values Data truncation limits
Results per run 25,000 Only the top 5,000 results will be included, prioritized by severity.

The current implementation doesn't appear to do that.

Either the text should be updated to say something else (my guess is that it's the top 5,000 results per severity), or the implementation should be changed to match the documentation (which would probably make more sense than the current behavior)

Additional information

https://github.com/check-spelling-sandbox/cert-manager/security

Image

check-spelling is reporting warnings. Check the status page for help.

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning/tools/check-spelling/status/configurations/actions-FZTWS5DIOVRC653POJVWM3DPO5ZS643QMVWGY2LOM4XHS3LM/e511b5682fa14795a6796791aeed75c7a0b4745efbf2807c37c878e23539b510

Image

Status 1 warning

Analysis SARIF file exceeded alert limits View workflow run An analysis file contained 5421 results which is more than our limit of 5000. Only 5000 were stored, the additional ones were ignored.

Learn more about limits in SARIF uploads.

^ This is the link to the page in question

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling

Image

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling+severity%3Anote%2Cwarning

Image

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling+severity%3Aerror

Image

https://ghsecuritylab.slack.com/archives/CQUMTHL1M/p1746543939781819

jsoref avatar May 06 '25 15:05 jsoref

Thanks for opening an issue! We've triaged this issue for technical review by a subject matter expert :eyes:

github-actions[bot] avatar May 07 '25 17:05 github-actions[bot]

@jsoref Okay, I got an SME reply to this issue, and I understand...that these are words. I'm going to quote him directly here to make sure I don't mess any of it up:

I think the one for limits (IIUC) is invalid. I think the docs and the implementation are consistent (though I implemented it so not a very good judge here).

I suspect the point they are trying to make is that in the UI they end-up with more than 5k alerts. This is intended. The limit is on the processing of a single upload, not on the overall number of alerts. The limit for the latter is much higher.

In the screenshot in which they see some alerts with lower severity, you can see that the date of the upload is several months ago, while for the ones with error severity it is "an hour ago".

If those uploads use different categories, then they can both coexist and surpass the limit of a single upload.

Sharra-writes avatar May 27 '25 17:05 Sharra-writes

This is a gentle bump for the docs team that this issue is waiting for technical review.

github-actions[bot] avatar Jun 25 '25 16:06 github-actions[bot]

A stale label has been added to this issue, because it has been open for 30 days with no activity. If you think this issue should remain open, please add a new comment.

github-actions[bot] avatar Jul 29 '25 16:07 github-actions[bot]

A stale label has been added to this issue, because it has been open for 30 days with no activity. If you think this issue should remain open, please add a new comment.

github-actions[bot] avatar Sep 02 '25 16:09 github-actions[bot]

Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning#validating-your-sarif-file

What part(s) of the article would you like to see updated?

The table says:

SARIF data Maximum values Data truncation limits Results per run 25,000 Only the top 5,000 results will be included, prioritized by severity. The current implementation doesn't appear to do that.

Either the text should be updated to say something else (my guess is that it's the top 5,000 results per severity), or the implementation should be changed to match the documentation (which would probably make more sense than the current behavior)

Additional information

https://github.com/check-spelling-sandbox/cert-manager/security

Image > check-spelling is reporting warnings. Check the [status page](https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning/tools/check-spelling/status/configurations/actions-FZTWS5DIOVRC653POJVWM3DPO5ZS643QMVWGY2LOM4XHS3LM/e511b5682fa14795a6796791aeed75c7a0b4745efbf2807c37c878e23539b510) for help.

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning/tools/check-spelling/status/configurations/actions-FZTWS5DIOVRC653POJVWM3DPO5ZS643QMVWGY2LOM4XHS3LM/e511b5682fa14795a6796791aeed75c7a0b4745efbf2807c37c878e23539b510

Image > Status > 1 warning > Analysis SARIF file exceeded alert limits > [View workflow run](https://github.com/check-spelling-sandbox/cert-manager/actions/runs/14862579061) > An analysis file contained 5421 results which is more than our limit of 5000. Only 5000 were stored, the additional ones were ignored. > [Learn more about limits in SARIF uploads](https://docs.github.com/en/code-security/code-scanning/integrating-with-code-scanning/sarif-support-for-code-scanning#validating-your-sarif-file).

^ This is the link to the page in question

https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling

Image https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling+severity%3Anote%2Cwarning Image https://github.com/check-spelling-sandbox/cert-manager/security/code-scanning?query=is%3Aopen+branch%3Aspell-check-with-spelling+tool%3Acheck-spelling+severity%3Aerror Image https://ghsecuritylab.slack.com/archives/CQUMTHL1M/p1746543939781819

https://github.com/github/docs/issues/38085#issue-3043297377

syedtoha8-png avatar Sep 15 '25 21:09 syedtoha8-png

A stale label has been added to this issue, because it has been open for 30 days with no activity. If you think this issue should remain open, please add a new comment.

github-actions[bot] avatar Nov 11 '25 16:11 github-actions[bot]