John Pugliesi

@MSeal Any update on the status of this? I don't fully understand papermill's architecture, and would like to understand why papermill's existing default engine cant run notebooks via the kernel...

@MSeal got it - and thanks for the quick response. For what it's worth, the Enterprise Gateway (EG) project recently merged a change that makes it simpler to execute EG...

@MSeal Ok, I just tried calling papermill with the `kernel-manager_class` specified, but it doesn't work. Here's the setup: --- a kernel called `spark_python_yarn_cluster` is available in the Gateway ``` ➜...

@mseal @golf-player @kevin-bates thanks for the additional context. I'm interested in getting more involved to contribute, but this is my first encounter with the jupyter internals, so I'm definitely still...

For what it's worth, we've open sourced a fully contained build and docker image for Spark 3.1.1 (with the kubernetes deps), Hadoop 3.2.0, Hive 2.3.7, and this glue client (largely...

As an update, [Spark 3.2 has been released](https://spark.apache.org/releases/spark-release-3-2-0.html) - are there any plans to support this?