Command: unpack module

[clebs]

Description

Kyma modules are released and distributed as OCI container images following the OCM format.

The CLI needs to provide utilities to unpack the modules and be able to read its contents.

Reasons

Allow developers to inspect the contents of module images

Attachments Implementation should be based on: https://github.com/gardener/component-cli

[kyma-stale-bot[bot]]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

[tobiscr]

@clebs : afaik we are not using CNUDIE for our Kyma modules anymore - is this still a valid task?

[clebs]

Hi @tobiscr, we are not using CNUDIE but OCM (its successor). I updated the description to make it clear.

The task is still relevant.

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically closed due to the lack of activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle stale

If you think that I work incorrectly, kindly raise an issue with the problem.

/close

[kyma-bot]

@kyma-bot: Closing this issue.

In response to this:

This issue or PR has been automatically closed due to the lack of activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle stale

If you think that I work incorrectly, kindly raise an issue with the problem.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Tomasz-Smelcerz-SAP]

The value of this feature is questionable now, as of recently we have some external tools that can perform the task.

Summary: The process of "unpacking" with an external tool requires a few manual steps, but can be easily automated with a simple bash script that needs just the jq for json parsing and some other common utlis (tar, gzip). The script will be simple because Kyma-CLI always produces modules (layers) in the same, predictable way. Considering that, I would postpone implementing that functionality in the Kyma-CLI until there's a real demand from our users to provide such a feature.

Details: There are two tools that I tested:

• skopeo
• oras

Both tools can pull and dump the artifact into a local directory. The oras tool is more promising as it is recommended by the docker team for the OCI artifacts management.

The latest available oras version, v1.0.0-rc.1, can do the following:

oras copy --to-oci-layout europe-west3-docker.pkg.dev/sap-kyma-jellyfish-dev/tsm-test/foo/component-descriptors/kyma-project.io/template-operator:0.0.21 ./download

Note: For this particular command to work, you need valid credentials to the registry I use (europe-west3-docker.pkg.dev/sap-kyma-jellyfish-dev) The command pulls an OCI artifact from the OCI registry into a local ./download directory. This specific OCI artifact is a Kyma module, pushed to the registry before with kyma alpha create module command. In the long URL you can recognize three parts of an OCM spec artifact name:

• europe-west3-docker.pkg.dev/sap-kyma-jellyfish-dev/tsm-test/foo (repository name)
• /kyma-project.io/template-operator (module name)
• 0.0.21 (module version)

This is how the contents of download/ directory look like after the oras copy command completes:

% tree download
download
├── blobs
│   └── sha256
│       ├── 4418e09ce610169f40f546206ab2616f2cb23d7f619348f175bd0ff9b7afb352
│       ├── 72fe2598b7846e37594ac83cd81450f1a668d523a5b9d4c8d4080b9519e277c7
│       ├── c68edd870840dacb8c3d52b953afaf1d3cf6f6c253f728eb9ab0ac2545eef241
│       ├── db86408caca4c94250d8291aa79655b84146f9cc45e0da49f05a52b3722d74a0
│       └── f30b04b95106383c7e280217b3e98a81c6aa9e581fb4bc2fbe4403273c673ed8
├── index.json
├── ingest
└── oci-layout

inspecting the type of the files provides a hint about their purpose:

% file download/blobs/sha256/*
download/blobs/sha256/4418e09ce610169f40f546206ab2616f2cb23d7f619348f175bd0ff9b7afb352: JSON data
download/blobs/sha256/72fe2598b7846e37594ac83cd81450f1a668d523a5b9d4c8d4080b9519e277c7: gzip compressed data, original size modulo 2^32 62464
download/blobs/sha256/c68edd870840dacb8c3d52b953afaf1d3cf6f6c253f728eb9ab0ac2545eef241: POSIX tar archive
download/blobs/sha256/db86408caca4c94250d8291aa79655b84146f9cc45e0da49f05a52b3722d74a0: ASCII text
download/blobs/sha256/f30b04b95106383c7e280217b3e98a81c6aa9e581fb4bc2fbe4403273c673ed8: JSON data

The download/index.json points to the OCI image manifest:

% cat download/index.json | jq .
{
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:f30b04b95106383c7e280217b3e98a81c6aa9e581fb4bc2fbe4403273c673ed8",
      "size": 657
    }
  ]
}

Reading out the manifest reveals the purpose of the layers:

% cat download/blobs/sha256/f30b04b95106383c7e280217b3e98a81c6aa9e581fb4bc2fbe4403273c673ed8 | jq .
{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.gardener.cloud.cnudie.component.config.v1+json",
    "digest": "sha256:4418e09ce610169f40f546206ab2616f2cb23d7f619348f175bd0ff9b7afb352",
    "size": 210
  },
  "layers": [
    {
      "mediaType": "application/vnd.gardener.cloud.cnudie.component-descriptor.v2+yaml+tar",
      "digest": "sha256:c68edd870840dacb8c3d52b953afaf1d3cf6f6c253f728eb9ab0ac2545eef241",
      "size": 2560
    },
    {
      "mediaType": "application/gzip",
      "digest": "sha256:72fe2598b7846e37594ac83cd81450f1a668d523a5b9d4c8d4080b9519e277c7",
      "size": 4352
    },
    {
      "mediaType": "application/octet-stream",
      "digest": "sha256:db86408caca4c94250d8291aa79655b84146f9cc45e0da49f05a52b3722d74a0",
      "size": 250
    }
  ]
}

The key to understand the purpose of the layer is the mediaType:

This is the component-descriptor media type. It gives us the name of the blob and a hint that it's a tar'red YAML:

{
      "mediaType": "application/vnd.gardener.cloud.cnudie.component-descriptor.v2+yaml+tar",
      "digest": "sha256:c68edd870840dacb8c3d52b953afaf1d3cf6f6c253f728eb9ab0ac2545eef241",
      "size": 2560
}

This is the "TAR+GZIP" contents of the bundled Helm Chart:

{
      "mediaType": "application/gzip",
      "digest": "sha256:72fe2598b7846e37594ac83cd81450f1a668d523a5b9d4c8d4080b9519e277c7",
      "size": 4352
},

Finally, this is a "config.yaml" of the module (neither encoded nor compressed):

    {
      "mediaType": "application/octet-stream",
      "digest": "sha256:db86408caca4c94250d8291aa79655b84146f9cc45e0da49f05a52b3722d74a0",
      "size": 250
    }

[janmedrek]

Since multiple tooling is available and the scenario is not on the basic path - we won't work on this issue.