java Java Model Generation Failed due to error message "The CustomResourceDefinition "xxx" is invalid: metadata.annotations: Too long: must have at most 262144 bytes"

Describe the bug Java Model Generation Failed due to the following error:

The CustomResourceDefinition "fleets.agones.dev" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Client Version e.g. 1.25.1

Kubernetes Version e.g. 1.25.1

Java Version e.g. Java 11

To Reproduce This bug is about crd-model-gen

Version: 1.06, 1.05 and 1.04

Steps to reproduce the behavior:

download the long CRD definitions (more than 262144 characters), eg: the attached CRD from agones fleets.agones.dev.yaml.zip
try to generate the Java models following the approach CRD Generation
You will see this error

Expected behavior The models can be generated successfully for long CRD definitions

Workaround We can workaround this issue by remove all the descriptions from the CRD to make it shorter, but we lost all the java docs after the generation.

KubeConfig If applicable, add a KubeConfig file with secrets redacted.

Server (please complete the following information):

OS: Linux
Environment [e.g. container]
Cloud [e.g. Azure]

Additional context Add any other context about the problem here.

Apr 07 '23 01:04 hippoz

This issue can be fixed by updating https://github.com/kubernetes-client/java/blob/34a7a2d19208bb140c0949054f14e67f00ad52d8/client-java-contrib/generate.sh#L52

from kubectl apply -f "$url" to kubectl create -f "$url"

Apr 07 '23 01:04 hippoz

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Jul 06 '23 02:07 k8s-triage-robot

Also hit this issue, not sure if the image has the latest generate script?

Also seems like the default image should move to ghcr.io/kubernetes-client/java/crd-model-gen to align with documentation https://github.com/kubernetes-client/java/blob/master/docs/generate-model-from-third-party-resources.md

❯ ./update.sh
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Creating cluster "kind" ...
 • Ensuring node image (kindest/node:v1.21.1) 🖼  ...
 ✓ Ensuring node image (kindest/node:v1.21.1) 🖼
 • Preparing nodes 📦   ...
 ✓ Preparing nodes 📦
 • Writing configuration 📜  ...
 ✓ Writing configuration 📜
 • Starting control-plane 🕹️  ...
 ✓ Starting control-plane 🕹️
 • Installing CNI 🔌  ...
 ✓ Installing CNI 🔌
 • Installing StorageClass 💾  ...
 ✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com created
customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com created
The CustomResourceDefinition "prometheuses.monitoring.coreos.com" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

Jan 02 '24 23:01 scprek

/remove-lifecycle stale

Jan 03 '24 14:01 scprek

This is the generate script in the latest image v1.0.6 it does not have the latest changes to the generate.sh file including https://github.com/kubernetes-client/java/pull/2670.

❯ docker run -ti ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 sh
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

/gen/openapi # cat /generate.sh
#!/bin/bash

# Copyright 2020 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This script orchestrates the code generation procedure. It is executed inside a crdgen
# container in order to minimize the environment dependencies on the host, being Docker only.

CRD_URLS=${CRD_URLS:-}
OUTPUT_DIR=${OUTPUT_DIR:-}
KUBERNETES_CRD_GROUP_PREFIX=${KUBERNETES_CRD_GROUP_PREFIX:-}
PACKAGE_NAME=${PACKAGE_NAME:-}

print_usage() {
  echo "Usage: generate Java model classes from CRDs" >& 2
  echo " -n: the prefix of the target CRD's api group to generate." >& 2
  echo " -p: the base package name of the generated java project. " >& 2
  echo " -o: output directory of the generated java project. " >& 2
  echo " -u: url location of the YAML manifest to install CRDs to a Kubernetes cluster. " >& 2
}

while getopts 'u:n:p:o:' flag; do
  case "${flag}" in
    u) CRD_URLS+=("${OPTARG}") ;;
    n) KUBERNETES_CRD_GROUP_PREFIX="${OPTARG}" ;;
    p) PACKAGE_NAME="${OPTARG}" ;;
    o) OUTPUT_DIR="${OPTARG}" ;;
    *) print_usage
       exit 1 ;;
  esac
done

set -e

# create a KinD cluster on the host
kind create cluster

# install CRDs to the KinD cluster and dump the swagger spec
for url in "${CRD_URLS[@]}"; do
  if [[ ! -z $url ]]; then
    kubectl apply -f "$url"
  fi
done

sleep 5
kubectl get --raw="/openapi/v2" > /tmp/swagger

echo "Verifying CRD installation.."
kubectl get crd -o name \
  | while read L
    do
      if [[ $(kubectl get $L -o jsonpath='{.status.conditions[?(@.type=="NonStructuralSchema")].status}') == "True" ]]; then
        echo "$L failed publishing openapi schema because it's attached non-structral-schema condition."
        kind delete cluster
        exit 1
      fi
      if [[ $(kubectl get $L -o jsonpath='{.spec.preserveUnknownFields}') == "true" ]]; then
        echo "$L failed publishing openapi schema because it explicitly disabled unknown fields pruning."
        kind delete cluster
        exit 1
      fi
      echo "$L successfully installed"
    done

# destroy the KinD cluster
kind delete cluster

# execute the generation script
bash java-crd-cmd.sh -n "${KUBERNETES_CRD_GROUP_PREFIX}" -p "${PACKAGE_NAME}" -l 2 -o "${OUTPUT_DIR}/gen" < /tmp/swagger

# only keep the model classes
mkdir -p "${OUTPUT_DIR}/src/main/java/${PACKAGE_NAME//.//}"
cp -r "${OUTPUT_DIR}/gen/src/main/java/${PACKAGE_NAME//.//}/models" "${OUTPUT_DIR}/src/main/java/${PACKAGE_NAME//.//}"
rm -rf "${OUTPUT_DIR}/gen"/gen/openapi

Jan 03 '24 14:01 scprek

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Apr 02 '24 15:04 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle rotten
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

May 02 '24 15:05 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen
Mark this issue as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Jun 01 '24 16:06 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue with /reopen

Mark this issue as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Jun 01 '24 16:06 k8s-ci-robot

As @scprek mentioned above, the latest version of crd-model-gen image, which is v1.0.6, is still not containing the fix. And the document is still pointing to v1.0.6.

This is inconvenient for the guys who have met the same problem. Here is a quick fix:

change the image from ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 to docker.io/apecloud/crd-model-gen:v1.0.7 .

That is, the whole cmd will be:

docker run \
  --rm \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$(pwd)":"$(pwd)" \
  -ti \
  --network host \
  docker.io/apecloud/crd-model-gen:v1.0.7 \
  /generate.sh \
  -u https://gist.githubusercontent.com/yue9944882/266fee8e95c2f15a93778263633e72ed/raw/be12c13379eeed13d2532cb65da61fffb19ee3e7/crontab-crd.yaml \
  -n com.example.stable \
  -p com.example.stable \
  -o "$(pwd)"

Attention: use the official image when they have updated it.

Jul 17 '24 05:07 free6om