fivetran
Examine performance under concurrency
Realistic scenarios frequently involve multiple analysts using the same db at the same time. Examine performance with 6+ queries going at once. Under Redshift's default options this will likely result in issues (http://docs.aws.amazon.com/redshift/latest/dg/cm-c-defining-query-queues.html) that shouldn't be experienced by BigQuery and experienced to a lesser extent on Snowflake.
I expect Redshift's performance to be pretty linear under concurrency---that's what I've seen in other warehouses, and it makes sense given how MPP query planners work. But we should quantitatively evaluate the linearity of each warehouse and report it.
I think you'll be horrifyingly surprised when it comes to Redshift - maybe the dedicated storage resources per shard - it does seem particularly worse on dense storage.
In addition to concurrent readers, it would be good to have concurrent writers, representing an ETL process that is appending, deleting, and updating at the same time that you are querying.