Improve secrets management

[debovema]

TL;DR : The current way of managing secrets should be improved.

Current situation

Currently, the app descriptor (flogo.json) is pre-processed during engine startup to replace all occurrences of "SECRET:xxx" by the decrypted value of xxx based on AES-256 with a secret decryption key provided by an environment variable FLOGO_DATA_SECRET_KEY.

See line https://github.com/project-flogo/core/blob/98a1d797692be81e068e9fecb1f80856e0ebab4e/engine/secret/config.go#L11 and following for reference.

Default value of FLOGO_DATA_SECRET_KEY encryption key is flogo. If this default value is not changed, all secret values can be decrypted easily, leaving the user in the illusion of security.

Moreover, where the secret values will be encoded ?

• If it's in the Flogo UI, it might use the default value flogo as encryption key. It could certainly be changed but that would complexify the configuration of Flogo UI.
• If it's manually, it would require tooling or tedious documentation.

Eventually, this pre-processing of app descriptor triggered by https://github.com/project-flogo/core/blob/98a1d797692be81e068e9fecb1f80856e0ebab4e/engine/appconfig.go#L61 is in conflict with properties finalization triggered by https://github.com/project-flogo/core/blob/98a1d797692be81e068e9fecb1f80856e0ebab4e/engine/engineimpl.go#L103

Especially, secret.PropertyProcessor function at https://github.com/project-flogo/core/blob/98a1d797692be81e068e9fecb1f80856e0ebab4e/engine/secret/config.go#L33 will not replace anything since properties with values prefixed by "SECRET:" have already been replaced. That's a pity because the property.PostProcessor approach seems the best.

Proposal

1. Simplify properties post-processing implementation

See PR #13.

With this simplification it will be easier to add new properties post-processors to manage secrets values.

2. Remove JSON pre-processing for "SECRET:*" values

See PR #14.

As said above, this pre-processing is conflicting with the actual property.PostProcessor post-processor.

3. Make the secrets "type-safe"

Considering the following properties array:

  "properties": [
    {
      "name": "backend.url",
      "type": "string",
      "value": "http://xxx"
    },
    {
      "name": "backend.username",
      "type": "string",
      "value": "user"
    },
    {
      "name": "backend.password",
      "type": "string",
      "value": "SECRET:xjOqSXAFanom8Y8NW5MCJg=="
    }
  ]

We could replace it by:

  "properties": [
    {
      "name": "backend.url",
      "type": "string",
      "value": "http://xxx"
    },
    {
      "name": "backend.username",
      "type": "string",
      "value": "user"
    },
    {
      "name": "backend.password",
      "type": "secret",
      "value": "xjOqSXAFanom8Y8NW5MCJg=="
    }
  ]

4. Create (external) secret resolvers

Following the model of external property resolvers, we could implement external secret resolvers. Especially the two kinds of resolvers would match the properties to process based on their type defined in 3.