gonymizer
gonymizer copied to clipboard
Add S3 support to all input/output files
We need to support AWS-Lambda which will allow Gonymizer to run as a serverless application. To do this we need to be able to store all files in S3 (not just the processed file from the dump command and row_counts file from the dump and load comma). This should be pretty easy to implement since we already have the S3 functions in s3.go
We can use tmp space in lambda jobs. See: https://forums.aws.amazon.com/thread.jspa?threadID=174119
Instead, I found that file storage should be generalized to use either local or s3 storage depending on the path the user supplies on the CLI interface.