segmentio
Allow reading from and writing to plain text using secretsmanager backend
My AWS Secrets Manager secret contains only plain text information so no key-value
key-value works
$ chamber -b secretsmanager list devops/key-value
Key Version LastModified User
<redacted> 3 2022-09-01 12:37:33 arn:<redacted>
<redacted> 1 2022-09-01 12:38:21 arn:<redacted>
but plain text does not
$ chamber -b secretsmanager list devops/plain-text
Error: Failed to list store contents: invalid character '-' in numeric literal
$ chamber -b secretsmanager read devops/plain-text
<shows help>
Here's the creation using the awscli of both of the types above
https://stackoverflow.com/a/58943565
Workaround for the list (or list-services) is to go back to awscli
aws secretsmanager list-secrets --query 'SecretList[].Name'
Here is the list function
https://github.com/segmentio/chamber/blob/0fda2320b6b5606f89302c75c6e8c78f13af2f9d/store/secretsmanagerstore.go#L365-L378
which calls readLatest
https://github.com/segmentio/chamber/blob/0fda2320b6b5606f89302c75c6e8c78f13af2f9d/store/secretsmanagerstore.go#L334-L354
which calls jsonToSecretValueObject
https://github.com/segmentio/chamber/blob/0fda2320b6b5606f89302c75c6e8c78f13af2f9d/store/secretsmanagerstore.go#L556-L562
which tries to do a json.Unmarshal and fails
Here is the read function
https://github.com/segmentio/chamber/blob/0fda2320b6b5606f89302c75c6e8c78f13af2f9d/store/secretsmanagerstore.go#L229-L239
I haven't been involved in this project previously, but would like to respond. (I'm a Segment employee.)
It does look like chamber requires secret values in AWS Secrets Manager to be a specific JSON structure, containing keys and values and other expected metadata. So, a secret value that's plain text - say, something like "foo" or even "foo=bar" - just isn't supported. I don't see a way to support plain text in the secret value and retain parity between Secrets Manager and other backends.
If the problem is writing a plain text value into a secret that can be read by chamber, then the chamber write command should do the trick, as opposed to AWS CLI usage. You will still need to provide a service name and key name, but the value can be just about any string. (The service name becomes the name / ID of the secret in Secrets Manager, but that's a detail of the current implementation.)
chamber -b secretsmanager write my-service key1 plaintext
If I'm missing the point of the problem, please respond with more details. Thanks!
Thanks for commenting.
I suppose writing plain text works (I haven't tried and I will) but reading plain text doesn't work and reading isn't possible for this backend and have it work with other backends. Thanks for confirming.
If we can leave this open for now it would be nice if there are other ideas. There must be a way to do it
If we can leave this open for now it would be nice if there are other ideas. There must be a way to do it
Sure, let's leave this open until we work it out! Digging deeper:
In your original example, there are two service arguments used: devops/key-value and devops/plain-text. I assume that these are intended to be two different services with those exact names, and not just one "devops" service.
I think the problem may be down to how chamber expects to unmarshal the secret string value as JSON. (In secretsmanagerstore.go, it's the readLatest and jsonToSecretValueObject functions.) If the string is just ordinary text, the unmarshaling fails. I tried this myself with a secret value of "this-is-just-plain-text":
$ chamber -b secretsmanager list plaintext
readLatest: invalid character 'h' in literal true (expecting 'r')
Error: Failed to list store contents: invalid character 'h' in literal true (expecting 'r')
When I change the raw string to {"value":"this-is-just-plain-text"}, chamber is happy.
$ ./chamber -b secretsmanager list -e plaintext
Key Version LastModified User Value
value 0 0000-12-31 19:03:58 this-is-just-plain-text
So, at the moment, by design, chamber requires a JSON structure. It can deal with a flat map structure that wasn't originally written by chamber, in order to consume values written by the AWS console. Even with that extended support, though, there's no chamber metadata available, so stuff like history isn't available.
With some code changes, I guess chamber could be able to read an arbitrary string in as just a single, unnamed secret. However, one problem with that is that chamber wouldn't be able to tell if it's reading an arbitrary string or malformed JSON. Maybe a command line option could give it a hint? Regardless, it seems safer to update whatever writes the secret either use chamber write or write a basic JSON map that gives the secret some arbitrary key name.
Thoughts?
Thanks for diving in again.
The issue only comes up when reading plain-text secrets that weren't originally saved by chamber. It's true, it's possible to overwrite the secret with a json value, however that would also require updating service code or infrastructure to ensure it understands the secret and its new structure.
If you have 1000s of plain-text secrets, now you're talking about a very long migration plan and for no business value so in the end, the awscli would then be more preferable to use. 😭