affinity icon indicating copy to clipboard operation
affinity copied to clipboard

Generalise AnalyticsDriver Prototype

Open michal-harish opened this issue 8 years ago • 1 comments

The steps involved in running the hadoop-less spark shell are quite configurable:

  1. shell.sh script can be generated - there are some env.variables which need to be linked to the AnalyticsShell object but this can be done in a patterned way
  2. shell-init.scala.sh scripts can be generated with configurable wildcard imports
  3. AnalyticsDriver class is the "face" of the shell
  4. AnalyticsDriver companion object can be completely provided by the spark module in general
  5. AnalyticsShell object can be generalised if configured well

Ideally most part under API and then some binding under each spark module

michal-harish avatar Jan 19 '18 17:01 michal-harish

Both shell and programmatic access to driver needs to be available in the general solution

michal-harish avatar Jan 25 '18 15:01 michal-harish