DirectStableKeywords error should trigger --ask event
It has now already happened twice that I copy an ebuild for revision bump, forget to remove the stable keywords, commit and push the revised ebuild, only to have pkgdev (rightfully) complain but continue with the push anyway. With repoman repoman ci would bail out if there were stable keywords in the ebuild, so this situation could never occur.
I do use the --ask argument on pkgdev push but apparently DirectStableKeywords is not severe enough to trigger a request for user confirmation.
I think the --ask parameter should trigger a request for confirmation on every new issue introduced by the commits. Otherwise what is the point of using pkgdev push when you're going to have to run pkgcheck scan --commit manually beforehand anyway? Possibly we could also retain the current behaviour of --ask and instead add a new argument --ask-strict?
A comment directed at current maintainers (and Gentoo in general): DirectStableKeywords is already an error result but what controls pkgdev erroring out by default is the list of keywords it sets for the GentooCI target (https://github.com/pkgcore/pkgcheck/blob/master/data/pkgcheck.conf). This file should be updated by cd-ing to the directory in the repo checkout and running the ci.py script that will pull the latest CI error keyword settings (assuming the format hasn't changed).
To make DirectStableKeywords also error out by default, it would have to be added to that same list (and I personally think that is what should be done instead of adding new options like --ask-strict).
Wrt triggering confirmation requests for new issues introduced by commits, to do that correctly would require caching results from local full tree scans since CI doesn't do it per commit and pkgcheck isn't fast enough for that to be feasible on the full gentoo repo (especially on low-end user hardware). If revdeps cache support was ever finished that would be another way to trim the target set to an acceptable level in most cases, but I didn't find implementing that in python to be performant enough to be relevant (in essence then you'd have to keep another local full tree db updated).
What I assume the request is alluding to is limiting this to only localized pkg commits (no profile changes, no pkg removals that affect revdeps, etc) which may be possible to do in some fashion, but I think that would get tricky fast unless those conditions cause the result diffing phase to be skipped which wouldn't be good since it wouldn't provide a consistent experience to the end user.
I forgot to mention it should be possible to personally configure your own set of error results that pkgdev errors out on by adding a customized keywords set called GentooCI to your personal pkgcheck config. Otherwise you could name it something else but then you'd also have to add to pkgdev's config to change the default --exit target as well.
You could even use separate sets per repo if you commit to different repos with different QA standards using pkgdev.
Alternatively, a repo could embed the default settings inside itself like other regular linting configs, e.g. using a .pylintrc file in a python project repo, since pkgcheck checks for and loads repo configs if they exist at metadata/pkgcheck.conf, but for the gentoo repo that would require getting the community to agree.
Personally, I think it would be better if Gentoo took this route as then the community at large could maintain the list and not whoever is currently the pkgcheck maintainer.
Alternatively, a repo could embed the default settings inside itself like other regular linting configs, e.g. using a .pylintrc file in a python project repo, since pkgcheck checks for and loads repo configs if they exist at
metadata/pkgcheck.conf, but for the gentoo repo that would require getting the community to agree.Personally, I think it would be better if Gentoo took this route as then the community at large could maintain the list and not whoever is currently the pkgcheck maintainer.
This would be great I think. This way we can have a rather strict setting for ::gentoo while keeping the default more lenient for third party repositories.
Filed https://bugs.gentoo.org/861563 for doing that.
Done in https://github.com/pkgcore/pkgdev/commit/047ac2bdcfe019107b13646825818a0bc5339b9d