[ZEPPELIN-4412] Added REST Api to shut down spark context for notebook

[amakaur]

What is this PR for?

This provides the ability to shut down interpreter for a specific notebook.

This makes it easier to release any allocated resources or close spark context when a paragraph is stuck on pending state.

This helps with resetting the state of the notebook.

What type of PR is it?

Feature

What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-4412

How should this be tested?

• Create a notebook with spark interpreter
• Make a call to http://[zeppelin-server]:[zeppelin-port]/api/notebook/close/[noteid]

Screenshots (if appropriate)

Questions:

• Does the licenses files need update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? No

[zjffdu]

I have a little concern about this PR. It looks like some workaround for issue of paragraph stuck in pending state, am I right ? @amakaur

[amakaur]

@zjffdu Yes, that's correct. We've ran into couple of issues with zeppelin 0.9 where this PR was necessary. Paragraph(s) would get stuck into a pending state so shutting down the interpreter would help bring back the notebook into working state.

[zjffdu]

@amakaur I think this is a bug of Zeppelin. Could you let me know when you will hit this kind of issue, is it reproducible ?

[amakaur]

@zjffdu its reproducible, but doesn't happen all of the time.

Currently we instantiate spark interpreter per note in isolated mode for each user. Change one of the spark interpreter settings from the ui and restart the interpreter.

After the restart the notebooks get stuck in pending state. This happens sometimes not all of the time.