Support configuration using files

[mikesir87]

Background

Recognizing that some of the configuration points are secrets, it would be great to support the ability to provide the configuration using files. This would also help support other mechanisms to load secrets (such as sidecars that load secrets from stores into a shared volume).

Proposal

For each environment variable, support the variable with a _FILE suffix. If defined, the contents of the file are used as the secret. I would expect only the following to be needed:

• SLACK_URL_FILE
• DATADOG_API_KEY_FILE
• MSTEAMS_URL_FILE

I'd be happy to help contribute some time to making this work. But, figured I'd open the issue first to see if others are ok/want changes to it before doing the work 👍

[marcosnils]

SGTM makes total sens since CSI doesn't support env variables. I guess we just need ot make sure that env variables (if defined) will override file definitions?

[mikesir87]

I'm thinking that we simply update DefaultConfig.Optional and DefaultConfig.Required methods to check for the presence of a ${key}_FILE variable first. If present, use the contents of the file it's pointing to. If the var is set, but no file is found, it's an error. If the file var isn't set, fall back to the direct env var. Sound alright? If so, I'll work on a quick PR to update that and create some tests for it.

[marcosnils]

If the file var isn't set, fall back to the direct env var. Sound alright? I

correct. And if both are set and the file is found, the env var should override the file one since it's more specific?

[mikesir87]

Good question. If both are set, I think that's an error. I just tried it against the official MySQL image and it throws an error and stops execution.

[fdpeiter]

Good question. If both are set, I think that's an error. I just tried it against the official MySQL image and it throws an error and stops execution. SGTM

Also, just to be clear, are you planning to load the whole exporter configuration from a file? Or just sensitive data? I would rather load the whole configuration from the file.

[mikesir87]

are you planning to load the whole exporter configuration from a file? Or just sensitive data?

I not planning on having a single file specify everything, but support the ability to specify a pointer for supported environment variables. As an example...

Imagine my slack webhook URL were (obviously made up) https://webhook.slack.com/12345. I could provide that in one of two ways:

• Set SLACK_URL to have a value of https://webhook.slack.com/12345
• Create a file (say /run/secrets/slack-url) that has the contents https://webhook.slack.com/12345. Then, I'd set SLACK_URL_FILE to have a value of /run/secrets/slack-url

[marcosnils]

^ correct. This is going to be a hybrid approach.

You can potentially load all the information from a file if you want (that can be injected through CSI secrets) or have a mix between file config and env variables.

:muscle:

[fdpeiter]

I would like the approach of having a single ENV variable pointing to a file where you can have only the variables you want to override/replace. Variables not present on the file can be loaded from the ENV.

Something like this, maybe: SLACK_CONFIG_FILE - xxx.json (any format)

{
  "URL": "xxx",
  "TOKEN": "yyy",
}

WDYT?

[andrei-dascalu]

@fdpeiter are you thinking something like adding godotenv, with loading a file and fallback on environment?

[fdpeiter]

Could be, just got the idea of having one single file per exporter