[BUG] only one pair of ak/sk is allowed in multi mounts Dataset when using JindoRuntime

[pandada8]

What feature you'd like to add: Ability to specify different ak/sk for different mounts. You can do this for now, but only the last aksk will work. Why is this feature needed: Mount different bucket from different account.

[TrafalgarZZZ]

Would you like provide more information about it? In Fluid, you can set spec.mounts[*].encryptOptions for each mount, so AFAIK it is a supported feature.

[pandada8]

The current implementation of jindocache actually prevents us from doing this.

https://github.com/fluid-cloudnative/fluid/blob/6cc47b883366fbc6a1c99f5d275fc6fea733984a/pkg/ddc/jindocache/transform.go#L470-L494

Only single pair key/secret is set from the code

https://github.com/fluid-cloudnative/fluid/blob/6cc47b883366fbc6a1c99f5d275fc6fea733984a/pkg/ddc/jindocache/transform.go#L420-L427

the debug code however support set different access key / secret key from options, but it's hidden under the jindocache.internal.test flag

[pandada8]

How to Reproduce:

apiVersion: data.fluid.io/v1alpha1
kind: JindoRuntime
metadata:
  name: merged
  namespace: default-group
spec:
  replicas: 1
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        volumeType: emptyDir
        quota: 20Gi
        high: "0.99"
        low: "0.95"
---
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: merged
  namespace: default-group
spec:
  placement: "Shared"
  accessModes:
    - ReadWriteMany
  mounts:
    - mountPoint: oss://bucket-A/some-random-prefix/
      options:
        fs.oss.endpoint: oss-cn-beijing-internal.aliyuncs.com
      name: cache
      path: "/partA"
      readOnly: true
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: oss-secret-A
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: oss-secret-A
              key: fs.oss.accessKeySecret
    - mountPoint: oss://bucket-B/some-random-prefix/
      options:
        fs.oss.endpoint: oss-cn-beijing-internal.aliyuncs.com
      name: aegis
      path: "/partB"
      readOnly: true
      encryptOptions:
        - name: fs.oss.accessKeyId
          valueFrom:
            secretKeyRef:
              name: oss-secret-2
              key: fs.oss.accessKeyId
        - name: fs.oss.accessKeySecret
          valueFrom:
            secretKeyRef:
              name: oss-secret-2
              key: fs.oss.accessKeySecret
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-merged-mount
  namespace: default-group
spec:
  selector:
    matchLabels:
      app: test-merged-mount
  template:
    metadata:
      labels:
        app: test-merged-mount
    spec:
      containers:
      - name: test-merged-mount
        image: alpine
        command: ['sh', '-c', 'sleep infinity']
        resources:
          limits:
            memory: "128Mi"
            cpu: "500m"
        volumeMounts:
          - name: merged
            mountPath: /merged
      volumes:
        - persistentVolumeClaim:
            claimName: merged
          name: merged

Running in the pod

> ls /merged/
# hang
> ls /merged/partA/
# hang
> ls /merged/partB/
fileA fileB fileC

the generated jindo-fuse pod manifest contains only one reference to the secret, mounted at the /cache

...
    volumeMounts:
    - mountPath: /token/AccessKeyId
      name: jindofs-secret-token
      readOnly: true
      subPath: fs.oss.accessKeyId
    - mountPath: /token/AccessKeySecret
      name: jindofs-secret-token
      readOnly: true
      subPath: fs.oss.accessKeySecret
...
  volumes:
  - name: jindofs-secret-token
    secret:
      defaultMode: 420
      secretName: oss-secret-2

inside the jindo-fuse pod, only secret from the second mount can be found on the /token/

> ls /token/
AccessKeyId  AccessKeySecret

[TrafalgarZZZ]

@pandada8 Thanks for the detailed example for reproducing this issue. I think it's currently not supported in JindoRuntime. We will check it later how to support it.