Support file based persistence for Vault Agent cache

[F21]

Is your feature request related to a problem? Please describe. Vault Agent currently only supports persistent caching for Kubernetes. There are lots of other cases where Vault Agent is not running in Kubernetes, but would still benefit from persistent caching. Being able to cache to a file or a folder of files would be extremely useful.

Describe the solution you'd like Being able to persist the cache to a file or a folder of files.

Describe alternatives you've considered None.

Explain any additional use-cases None.

Additional context As the cache is a BoltDB file, it should not require too much engineering effort to cache this to a file on disk or some other storage.

[carlzogh]

+1 to this

[carlzogh]

We have worked around this and currently have Vault Agent / Proxy set up with persistent caching in "kubernetes" mode without actually running in Kubernetes.

"type = kubernetes" really only signals the Agent/Proxy to fetch the Service Account JWT from the path specified in service_account_token_file. This token file is read and its value is used as the encryption key for the local cache.

Our cache configuration:

# ref. https://developer.hashicorp.com/vault/docs/agent-and-proxy/proxy/caching
cache {
    disable_caching_dynamic_secrets = true
    # ref. https://developer.hashicorp.com/vault/docs/agent-and-proxy/proxy/caching/static-secret-caching
    cache_static_secrets = true
    persist {
        type = "kubernetes"  # mocking k8s by providing a (secret) static service account JWT token as AAD
        path = "/var/run/cache"
        service_account_token_file = "/etc/config/cache-persistence-token"
        keep_after_import = true
    }
}

In our model, we inject a secret into the /var/run/cache-persistence-token file to ensure that the cache can be encrypted and decrypted only by these instances of Vault Agent/Proxy.

---

Vault Agent/Proxy really shouldn't care about whether the environment it is running in is Kubernetes or not, as it only needs a filesystem path to store the cache in, and an encryption secret.

This could instead be modeled as (following conventions set by other stanzas, eg. auto-auth sinks):

cache {
    # ...
    persist {
        type = "filesystem"
        path = "/var/run/cache"
        aad_file = "/etc/config/cache-persistence-token"  # alternatively, "aad" / "aad_env_var" could be specified
        keep_after_import = true
    }
}