Automation analysis

[simcha-vos-from-tu-delft]

I'm with the Software Engineering Research Group at the TU Delft, and I am investigating automations. I have looked at your repository and which GitHub workflows and Maven plugins you are using. I'll give you a quick summary of what I found, and the automation tasks I recommend you to focus on next (:

Level of maturity	Basic	Intermediate	Advanced
Collaboration	‎ ✅ Completed this level!
	✔️ Prepare or create documentation artifacts	❌ Generate documentation from source code ❌ Commit validation ❌ Bot commits	❌ Publish documentation ❌ Issues or PRs management
Code quality	‎ ✅ Completed this level!
	✔️ Run tests ✔️ Static code style analysis	✔️ Test coverage and validity ✔️ Generate test reports ✔️ Static code quality analysis ❌ Automatic code formatting	✔️ Verify packaging correctness ✔️ License checks ❌ Vulnerability scans ❌ Sign artifacts
Development	‎ ⚠️ Still working on this level!
	✔️ Build environment configuration ❌ Package management	❌ Build files configuration	❌ Optimization
Artifacts	‎ ✅ Completed this level!
	✔️ Code compilation ✔️ Dependency management of artifact	✔️ Build tasks, resources and configuration ✔️ Packaging ✔️ Release tagging ✔️ Publish artifacts to a registry ❌ Generate source and metadata artifacts ❌ Source code version control	❌ Generate release notes ❌ Containerization ❌ Push container to remote

And now? Next steps!

It is not always clear which automation tasks should be prioritized. It is however important to balance your automation efforts, as a uniform level of maturity is most productive. I'm here to help! Below is a list of tasks that you can work on to help level up your maturity across the automation domains:

• Implement Package management; implemented by 36% of GitHub repositories
• Implement Build files configuration; implemented by 13% of GitHub repositories

What do you think?

Do you think my analysis is correct, do these recommendations help you? Or have I missed something?

For my master's thesis I'm doing research into the use of automations in GitHub repositories. With my findings, I want to see if I can help out developers with what automations they could focus on. Do you want to help me out? Leaving a response is much appreciated!

[b-fein]

Hi,

in general, I think that the analysis is correct. For all the categories I can also easily understand why some item is marked as completed or not.

However, there are two exceptions exactly for your recommendations:

Package management

This seems to be heavily focussed on Python. I guess this is classified as ‘package’ rather than ‘dependency’ management because you can install other supporting tools like ruff, black, mypy, reuse, … through pip/poetry/uv? However, just installing the regular dependencies for the project already fulfills the requirements, even though one is using pip only as a dependency manager rather than a package manager. I’d also argue, that for a Maven project the installation of support tools is often not necessary (i.e. they are integreted as plugins), or that Maven fulfills the same function as package manager. E.g. we use the checkstyle, spotbugs, and pmd plugins to download & run these linter tools. It just happens that these separate tools are also available as integration so that I do not need to install them as separate executables like for Python.

Build files configuration

What is the ‘build file’? Is that the pom.xml that defines how Maven builds the Java project, or does this refer to the GitHub workflow YAML? How does mvn clean configure the build plan? If my CI starts in a fresh container anyway every time, why would I need that? Like for the package management, I would also argue that there are a lot more direct integrations for Maven in the form of plugins. Therefore, downloading some external tools/script via curl/wget is rarely necessary.

---

I guess my points/questions above also apply to Gradle projects equally, in case the analysis would be extended in the future to another package/dependency/project management tool.

[simcha-vos-from-tu-delft]

Hi,

in general, I think that the analysis is correct. For all the categories I can also easily understand why some item is marked as completed or not.

However, there are two exceptions exactly for your recommendations:

Package management

This seems to be heavily focussed on Python. I guess this is classified as ‘package’ rather than ‘dependency’ management because you can install other supporting tools like ruff, black, mypy, reuse, … through pip/poetry/uv? However, just installing the regular dependencies for the project already fulfills the requirements, even though one is using pip only as a dependency manager rather than a package manager. I’d also argue, that for a Maven project the installation of support tools is often not necessary (i.e. they are integreted as plugins), or that Maven fulfills the same function as package manager. E.g. we use the checkstyle, spotbugs, and pmd plugins to download & run these linter tools. It just happens that these separate tools are also available as integration so that I do not need to install them as separate executables like for Python.

Build files configuration

What is the ‘build file’? Is that the pom.xml that defines how Maven builds the Java project, or does this refer to the GitHub workflow YAML? How does mvn clean configure the build plan? If my CI starts in a fresh container anyway every time, why would I need that? Like for the package management, I would also argue that there are a lot more direct integrations for Maven in the form of plugins. Therefore, downloading some external tools/script via curl/wget is rarely necessary.

I guess my points/questions above also apply to Gradle projects equally, in case the analysis would be extended in the future to another package/dependency/project management tool.

Hello! I agree, package management is not 100% appropriate, and I was in fact doubting whether to add it, as I think it maybe does not tell us anything about how well you're doing your automation efforts. I will reconsider this! You are indeed quite right with your observation about a maven project.

I understand your point about build files configuration. It is supposed to be any runner-related setup, putting any necessary files in place, indeed cleaning, etc. I agree that it is not (always) necessary to do anything with this. I might get rid of this category for that reason.

I appreciate it a lot, many thanks for your insightful comments!

[simcha-vos-from-tu-delft]

Hi,

in general, I think that the analysis is correct. For all the categories I can also easily understand why some item is marked as completed or not.

However, there are two exceptions exactly for your recommendations:

Package management

This seems to be heavily focussed on Python. I guess this is classified as ‘package’ rather than ‘dependency’ management because you can install other supporting tools like ruff, black, mypy, reuse, … through pip/poetry/uv? However, just installing the regular dependencies for the project already fulfills the requirements, even though one is using pip only as a dependency manager rather than a package manager. I’d also argue, that for a Maven project the installation of support tools is often not necessary (i.e. they are integreted as plugins), or that Maven fulfills the same function as package manager. E.g. we use the checkstyle, spotbugs, and pmd plugins to download & run these linter tools. It just happens that these separate tools are also available as integration so that I do not need to install them as separate executables like for Python.

Build files configuration

What is the ‘build file’? Is that the pom.xml that defines how Maven builds the Java project, or does this refer to the GitHub workflow YAML? How does mvn clean configure the build plan? If my CI starts in a fresh container anyway every time, why would I need that? Like for the package management, I would also argue that there are a lot more direct integrations for Maven in the form of plugins. Therefore, downloading some external tools/script via curl/wget is rarely necessary.

I guess my points/questions above also apply to Gradle projects equally, in case the analysis would be extended in the future to another package/dependency/project management tool.

Hello, I have a final question to help me better understand your interest in tools like this. Given that your feedback would be implemented in this tool and maybe would be extended to other automations such as scripts/gradle, would you be interested in using such a tool, as a way to easily report on your current state of automations, and to get recommended next automations to work on? For example, if it could be available as a badge in your readme? Thank you!

[b-fein]

Yes, I think I would use that.

From my point of view, the basic requirement would be that the shown recommendations are somewhat tailored towards the project. E.g. if I have a Maven project, I want to see recommendations like ‘use command mvn … in your workflow to …’. I do not care about examples of poetry/uv/… commands for this specific Maven project.

A next step would be to have copy-pasteable workflows (or workflow parts) that I can use as starting point with a few modifications. I.e. an extended version of the recommended basic templates shown by GitHub when creating a new workflow:

[simcha-vos-from-tu-delft]

Yes, I think I would use that.

From my point of view, the basic requirement would be that the shown recommendations are somewhat tailored towards the project. E.g. if I have a Maven project, I want to see recommendations like ‘use command mvn … in your workflow to …’. I do not care about examples of poetry/uv/… commands for this specific Maven project.

A next step would be to have copy-pasteable workflows (or workflow parts) that I can use as starting point with a few modifications. I.e. an extended version of the recommended basic templates shown by GitHub when creating a new workflow:

Thanks a lot for the good ideas! I will definitely take these up in my thesis writing. Cheers, Simcha