Upgrade the spark-on-k8s-operator module to spark-operator-chart-1.1.15 #minor

[akumor]

This PR upgrades the spark-on-k8s-operator plugin with the goal of fixing support for defining the podSecurityContext for a SparkApplication created by executing a Spark task.

TL;DR

This upgrades the module to the spark-operator-chart-1.1.15 release.

Type

• [x] Bug Fix
• [ ] Feature
• [ ] Plugin

Are all requirements met?

• [x] Code completed
• [ ] Smoke tested
• [x] Unit tests added
• [x] Code documentation added
• [ ] Any pending items have an associated Issue

Complete description

I fixed the bug by upgrading the version of the spark-on-k8s-operator to the most recently tagged release spark-operator-chart-1.1.15 and updated the spark flyteplugin code to be compatible.

Tracking Issue

https://github.com/flyteorg/flyte/issues/2025

Follow-up issue

NA

[welcome[bot]]

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

• Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
• Sign off your commits (Reference: DCO Guide).

[codecov[bot]]

Codecov Report

Merging #232 (3a7f924) into master (6b72fd6) will increase coverage by 1.44%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #232      +/-   ##
==========================================
+ Coverage   62.50%   63.95%   +1.44%     
==========================================
  Files         142      142              
  Lines        8849     9365     +516     
==========================================
+ Hits         5531     5989     +458     
- Misses       2816     2852      +36     
- Partials      502      524      +22     

Flag	Coverage Δ
unittests	63.56% <100.00%> (+1.45%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
go/tasks/plugins/k8s/spark/spark.go	78.77% <100.00%> (+0.23%)	:arrow_up:
go/tasks/plugins/array/k8s/task.go	54.91% <0.00%> (-4.41%)	:arrow_down:
go/tasks/plugins/array/awsbatch/executor.go	39.09% <0.00%> (-1.48%)	:arrow_down:
...plugins/array/awsbatch/definition/job_def_cache.go	0.00% <0.00%> (ø)
go/tasks/plugins/k8s/pod/plugin.go	89.28% <0.00%> (+0.39%)	:arrow_up:
go/tasks/plugins/k8s/sagemaker/plugin.go	76.66% <0.00%> (+1.22%)	:arrow_up:
go/tasks/plugins/array/awsbatch/client.go	66.32% <0.00%> (+2.83%)	:arrow_up:
go/tasks/plugins/webapi/athena/plugin.go	21.76% <0.00%> (+3.91%)	:arrow_up:
go/tasks/plugins/array/awsbatch/transformer.go	83.75% <0.00%> (+4.88%)	:arrow_up:
... and 7 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6b72fd6...3a7f924. Read the comment docs.

[kumare3]

LGTM, @akumor weird that the coverage is broken