[K8S][HELM] Add Hadoop configuration support

[dnskr]

:mag: Description

Issue References 🔗

The PR adds basic support for Hadoop configuration files to be used by Apache Kyuubi, Apache Spark etc.

Describe Your Solution 🔧

The PR adds:

• Apache Hadoop related ConfigMap that has to be mounted to Kyuubi pods as files.
• HADOOP_CONF_DIR environment variable to Kyuubi pods.

Notes:

• The ConfigMap change restarts Kyuubi pods!
• Only core-site.xml and hdfs-site.xml files are added. Not sure yarn-site.xml, mapred-site.xml or other files should be added. Please, let me know if so.

Types of changes :bookmark:

• [ ] Bugfix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Install the chart with configured Spark and the following values:

# Hadoop configuration files
hadoopConf:
  # The value (templated string) is used for core-site.xml file
  # See https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/resources/core-default.xml and Hadoop documentation for more details
  coreSite: |
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
      <property>
        <name>test.core.site</name>
        <value>test-core-site</value>
      </property>
    </configuration>

  # The value (templated string) is used for hdfs-site.xml file
  # See https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml and Hadoop documentation for more details
  hdfsSite: |
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
      <property>
        <name>test.hdfs.site</name>
        <value>test-hdfs-site</value>
      </property>
    </configuration>

Run beeline from Kyuubi pod:

./bin/beeline -u 'jdbc:hive2://kyuubi-thrift-binary:10009' -n test-user

Check specified property values:

0: jdbc:hive2://kyuubi-thrift-binary:10009> set test.core.site;
+-----------------+-----------------+
|       key       |      value      |
+-----------------+-----------------+
| test.core.site  | test-core-site  |
+-----------------+-----------------+


0: jdbc:hive2://kyuubi-thrift-binary:10009> set test.hdfs.site;
+-----------------+-----------------+
|       key       |      value      |
+-----------------+-----------------+
| test.hdfs.site  | test-hdfs-site  |
+-----------------+-----------------+

---

Checklist 📝

• [x] This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (f67140e) 61.03% compared to head (1d66b3e) 60.99%.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #6046      +/-   ##
============================================
- Coverage     61.03%   60.99%   -0.05%     
  Complexity       23       23              
============================================
  Files           623      623              
  Lines         37162    37162              
  Branches       5035     5035              
============================================
- Hits          22682    22666      -16     
- Misses        12029    12033       +4     
- Partials       2451     2463      +12     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[dnskr]

Closed in favor to:

• https://github.com/apache/kyuubi/issues/6123
• https://github.com/apache/kyuubi/pull/6521