gonymizer icon indicating copy to clipboard operation
gonymizer copied to clipboard

Add S3 support to all input/output files

Open junkert opened this issue 6 years ago • 1 comments

We need to support AWS-Lambda which will allow Gonymizer to run as a serverless application. To do this we need to be able to store all files in S3 (not just the processed file from the dump command and row_counts file from the dump and load comma). This should be pretty easy to implement since we already have the S3 functions in s3.go

junkert avatar Mar 19 '19 18:03 junkert

We can use tmp space in lambda jobs. See: https://forums.aws.amazon.com/thread.jspa?threadID=174119

Instead, I found that file storage should be generalized to use either local or s3 storage depending on the path the user supplies on the CLI interface.

junkert avatar Jun 20 '19 20:06 junkert