linuxkit icon indicating copy to clipboard operation
linuxkit copied to clipboard

Why do the AWS docs instruct us to use the raw (uncompressed) file format? AWS supports other formats and the raw file format is extremely inefficient

Open colbyn opened this issue 4 years ago • 1 comments

Why does the AWS docs instruct us to build our images using some "raw" file format?

Why not just use e.g. VMDK which is significantly smaller (e.g. from 1GB to ~100M)?

colbyn avatar Jul 02 '21 23:07 colbyn

I don't know if this is related. But for some reason I couldn't get LinuxKit to upload e.g. VMDK encoded files to AWS, it just blocks (seemingly forever).

I have a suspicion it's due to this line, that seems to hardcode a given format (even though AWS supports other formats and the raw file format is extremely inefficient). I.e.

importParams := &ec2.ImportSnapshotInput{
    Description: aws.String(fmt.Sprintf("LinuxKit: %s", name)),
    DiskContainer: &ec2.SnapshotDiskContainer{
        Description: aws.String(fmt.Sprintf("LinuxKit: %s disk", name)),
        Format:      aws.String("raw"),
        UserBucket: &ec2.UserBucket{
            S3Bucket: aws.String(bucket),
            S3Key:    aws.String(dst),
        },
    },
}

Anyway for now, I created a simple tool that automatically uploads the given image to AWS.

E.g.

$ ami-uploader upload --bucket my-s3-bucket --image ./my-source-image.vmdk --name my-aws-ami

colbyn avatar Jul 03 '21 04:07 colbyn