OctopusDeploy
Improve the mechanism which Octopus uses for interacting with repositories in Config as Code Projects
The current cloning mechanism used in CaC projects requires pulling down the entire repository. This can be less performant than ideal, particularly for cloud customers where this background process may happen any time an upgrade is pushed to a customer instance. In addition, some customers share their Octopus CaC configuration with their application code, which may result in many Gb of storage used just for us to access a few files.
Some experimentation suggests that interacting directly with the APIS of hosted git repository vendors may substantially improve this process, eliminating the time to clone (with some minimal tradeoff for accessing on demand) and eliminating the disk usage of cloned repositories.
If that's git-based, one could also make it better by using something like
git clone --bare --filter=tree:0 --no-checkout $repo .git
git config --unset core.bare
git sparse-checkout set .octopus
git fetch origin +main:refs/remotes/origin/main
git checkout -b main
If that's git-based, one could also make it better by using something like...
Thanks @Lexy2 , We don't use the bare git cli under the hood and instead use LibGit2Sharp which does not support some of these more recent and advanced git scenarios.
From our experiments there are tradeoffs with using some of these sort of partial cloning + sparse checkout techniques when the repo is read\written to in different ways and the biggest wins we can make will be by avoiding cloning anything at all.
Watch this space!
We suffer from this issue all the time. We do keep our octopus code and application code in the same repository. This seems like the most logical way to do it honestly. It looks like LibGit2Sharp is nearly a dead project, they have been asking for new maintainers since 2020.
What other options are there and do you have a timeline for getthing this addressed? When we load the UX after a version upgrade of our cloud instance we have page load delays in the 30-60 second range. When this happens the application is pretty much until it finishes the refresh.
@twurm we are rolling out a change to internally dogfood that utilizes the GitHub Api directly and improves CaC specific functionality for projects that use the Octopus GitHubApp for Authentication. If you can reach out via your account manager or standard support channels we can look at giving your cloud instance early access to the alpha.