concourse
Allow separate cache in / out directories
Hey!
We use the oci-build-task quite heavily, but have noticed that over time the cache folder grows as our Dockerfiles are changed, and older layers are not removed when they are no longer relevant to the built image. This can cause the moving of the cache between workers to take more time than is saved by using the cache, which somewhat defeats the purpose.
We could use the caches directive to only keep the cache on the worker the task is run on, but this leads to rebuilding the image entirely whenever the task lands on a different worker. Without sharing the cache between workers, image layers would be re-built and duplicated in our container image repository too (even when the underlying files have not changed).
This PR adds cache-in and cache-out folders to the task. If a cache folder is not included in the task configuration, then these folders will be checked to see if they exist and, if so, used instead of the cache folder.
As the name implies, cache-in is where the cache will be imported from, and cache-out is where it will be exported to. Exporting the cache will (should?) only include the relevant layers, keeping the cache from including unnecessary layers and growing over time.
