Abdul-Raheman MouhamadSultane

Missing import for "ImproperlyConfigured" added from pydrill.exceptions

--spark-ec2-compressed option added.

7

## Description `--spark-ec2-compressed` option enables to precise a compressed version of spark-ec2. This option is an alternative to cloning spark-ec2 from GitHub. ## Accepted compression format .tar, .tar.gz, .tar.bz2, .tar.xz

Config file devel

8

# Configuration file `-c` or `--conf-file` option enables to use a YAML configuration file to simplify the way you interact with the cluster. ## Launch new cluster `./spark-ec2 --conf-file config.yml...

Elastic spark cluster

4

# Elastic Spark Cluster ## Prerequisite A running Spark cluster. ## add-slaves `./spark-ec2 -k key-name -i file.pem add-slaves cluster_name` This command adds a new slave to the cluster. To add...

s3a filesystem added to core-site for ephemeral-hdfs and persistent-hdfs

3

s3a is the successor of s3n file system, s3a offers higher performance and support of larger files. For more details: [https://wiki.apache.org/hadoop/AmazonS3](https://wiki.apache.org/hadoop/AmazonS3)