java exception during startup of opentsdb init container

[kgreczka9]

ISSUE The main pod opentsdb crashes afer some time/some amount of data processed. In our case it is regular - after about two weeks of usage. The opentsdb logs:

opentsdb-create-hbase-tables Checking if given compression is supported...
opentsdb-create-hbase-tables 2022-08-16 07:59:33,333 INFO  [main] metrics.MetricRegistries: Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesIm
opentsdb-create-hbase-tables 2022-08-16 07:59:33,350 INFO  [main] hfile.CacheConfig: Created cacheConfig: CacheConfig:disabled
opentsdb-create-hbase-tables 2022-08-16 07:59:33,523 INFO  [main] zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
opentsdb-create-hbase-tables 2022-08-16 07:59:33,538 INFO  [main] compress.CodecPool: Got brand-new compressor [.gz]
opentsdb-create-hbase-tables 2022-08-16 07:59:33,542 INFO  [main] compress.CodecPool: Got brand-new compressor [.gz]
opentsdb-create-hbase-tables 2022-08-16 07:59:33,727 INFO  [main] hfile.CacheConfig: Created cacheConfig: CacheConfig:disabled
opentsdb-create-hbase-tables 2022-08-16 07:59:33,741 INFO  [main] compress.CodecPool: Got brand-new decompressor [.gz]
opentsdb-create-hbase-tables SUCCESS
opentsdb-create-hbase-tables HBase script:
opentsdb-create-hbase-tables ----
opentsdb-create-hbase-tables create 'tsdb-uid',
opentsdb-create-hbase-tables {NAME => 'id', COMPRESSION => 'GZ', BLOOMFILTER => 'ROW'},
opentsdb-create-hbase-tables {NAME => 'name', COMPRESSION => 'GZ', BLOOMFILTER => 'ROW'}
opentsdb-create-hbase-tables create 'tsdb',
opentsdb-create-hbase-tables {NAME => 't', VERSIONS => 1, COMPRESSION => 'GZ', BLOOMFILTER => 'ROW'}
opentsdb-create-hbase-tables create 'tsdb-tree',
opentsdb-create-hbase-tables {NAME => 't', VERSIONS => 1, COMPRESSION => 'GZ', BLOOMFILTER => 'ROW'}
opentsdb-create-hbase-tables create 'tsdb-meta',
opentsdb-create-hbase-tables {NAME => 'name', COMPRESSION => 'GZ', BLOOMFILTER => 'ROW'}
opentsdb-create-hbase-tables ----
opentsdb-create-hbase-tables Checking if opentsdb tsdb-uid hbase table exists
opentsdb-create-hbase-tables java exception

PRECONDITIONS chart version - 0.1.7 custom values:

config: {"tsd.http.request.enable_chunked":true,"tsd.http.request.max_chunk":65535}
daemons: 1
hbase:
  enabled: true
  hbase:
    master:
      replicas:3
    regionServer:
      replicas: 3
      resources:
        requests:
          memory: 1Gi
          cpu: 100m
  hdfs:
    enabled: true
    dataNode:
      replicas: 3
    persistence:
      nameNode:
        enabled: true
        accessMode: ReadWriteOnce
        size: 5Gi
      dataNode:
        enabled: true
        accessMode: ReadWriteOnce
        size: 10Gi
  zookeeper:
    enabled: true
    replicaCount: 1
    affinity:
      podAntiAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - topologyKey: "kubernetes.io/hostname"
            labelSelector:
              matchLabels:
                release: zookeeper
    persistence:
      enabled: true
      accessMode: ReadWriteOnce
      size: 1Gi
resources:
  requests:
    memory: 1Gi
    cpu: 100m

STEPS TO REPRODUCE Install chart with custom values, use it as normal for a while (about a week or two depending on configuration)