kptdev
Share your "setter" use case to make KRM functions better
Call for help
We have received many issues about setters and other resource-specific KRM functions. We’d like to better help our users and understand your use cases. At the same time, we’d like to encourage more contributors to help improve the KRM functions together.
To make the discussion more transparent, if you encounter a setter issue or request a setter feature, please share your setter usage (and the friction issues if you have) by filling the questionnaire and reply in the comment.
Questionnaire
Q1: Are you using setters on metadata fields?
Answer: yes / no.
Which metadata fields are you using if the answer to Q1 is true
(please change - [ ] to - [x] if it applies)
- [ ] namespace
- [ ] name
- [ ] annotations
- [ ] labels
Can you use the following functions instead?
If the following functions do not fulfill your needs, please jump to Q4
| metadata | suggested functions | User Guide |
|---|---|---|
| namespace | https://gcr.io/kpt-fn/set-namespace:unstable | catalog.kpt.dev/set-namespace/v0.4 |
| name | https://gcr.io/kpt-fn/ensure-name-substring:unstable | catalog.kpt.dev/ensure-name-substring/v0.2 |
| annotations | https://gcr.io/kpt-fn/set-annotations:unstable | catalog.kpt.dev/set-annotations/v0.1 |
| name | https://gcr.io/kpt-fn/ensure-name-substring:unstable | catalog.kpt.dev/ensure-name-substring/v0.2 |
Check catalog.kpt.dev to find more options.
Q2: Are you using Google Cloud ConfigConnector resources?
See https://cloud.google.com/config-connector/docs/reference/overview Answer: yes / no.
Q3: Are you using setters for the GCP specific fields?
(please change - [ ] to - [x] if it applies)
- [ ] Project
- [ ] Folder
- [ ] Domain
- [ ] ContainerCluster
- [ ] ServiceAccount
- [ ] IAMPolicy
- [ ] Service
- [ ] WorkloadIdentityBinding
Do you have any other GCP fields that are frequently used?
If you have one GCP resource that uses setters and are referred to in multiple places, please add their apiVersion and kind below
Can you use the following curated GCP functions instead?
| GCP resources | suggested functions | User Guide |
|---|---|---|
| GCP Services | gcr.io/kpt-fn/enable-gcp-services:v0.1.0 | catalog.kpt.dev/enable-gcp-services/v0.1/ |
| ResourceHierarchy | gcr.io/kpt-fn/generate-folders:v0.1.1 | catalog.kpt.dev/generate-folders/v0.1/ |
| GCP Project | gcr.io/kpt-fn/set-project-id:v0.2.0 | catalog.kpt.dev/set-project-id/v0.2/ |
If the functions in the https://catalog.kpt.dev/?id=gcp-functio|ns do not fulfill the needs, please jump to Q4
Q4: Share your use case
Can you share the resources and their fieldpath that you apply the setters on?
apiVersion: # e.g. apps/v1
kind: # e.g. Deployment
jsonPath: # e.g. spec.template.spec.volumes[k=v].name
If the setter does not work well with your kpt packages or your function pipelines, can you share your kpt packages?
kpt pkg get <YOUR PKG>
kpt fn render
Problem summary
Expected output
Actual output
Q5: Are you using Porch or Porch UI?
Answer: yes / no.
Note: I have no idea how to fill-in this questionnaire other than copy&pasting some questions, which is a lot of work =)
Q1: Are you using setters on metadata fields?
Because of the "dont-use-apply-setters" warnings everywhere, we did something arguably worse: use starlark & apply-replacements.
Starlark usage for:
- constructing a value was not possible with apply-replacements (delimiter & index not sufficient)
- unique naming of resources. A lot of function config is needed to keep resources unique, when working with nested packages (application package nests: k8s-deployment-myapp, storage-bucket-myapp, cloudsql-myapp)
Which metadata fields are you using if the answer to Q1 is true
- [x] name
Can you use the following functions instead? ensure-name-substring would have to work with all resources, because everything needs to be unique. Now CustomResourceDefinition, Namespace and APIService are not updated.
We'd also like not to simply prepend or suffix every single resource, we just want them to be unique. Blindly mutating everything works, but also leads to ugly resource naming. We'd like far more control over what resources are named.
Q2: Are you using Google Cloud ConfigConnector resources? yes
Q3: Are you using setters for the GCP specific fields? Again starlark & apply-replacements.
- [x] ServiceAccount
- [x] IAMPolicy
- [x] Service
- [x] WorkloadIdentityBinding
Regarding Service, we had to create a generator function so that package authors can simply add a ToBeGeneratedService local CRD in their package, and the function in the root package with create 1 Service per api - otherwise multiple variants of e.g. a cloudsql package would result in 1. naming conflicts but also 2. Config Connector isn't smart enough to detect 2 resources that have different names, but are identical in what they instantiate, meaning you get into situations where with a single configuration, resources get deleted and re-created: https://github.com/GoogleCloudPlatform/k8s-config-connector/issues/687.
We deemed enable-gcp-services would not work, because of the decentralized nature (nested packages, and developed by different teams) of specifying ProjectServiceSet per package. That said, perhaps it would simply work and the function is smart enough to generate a consistent, deduplicated list of enabled Services. I'm not sure we tested this.
Do you have any other GCP fields that are frequently used?
IAMPolicyMember.metadata.name
IAMPolicyMember.spec.member
Q4: Share your use case
Problem summary
Using the value data.teamName from a uniquely named resource:
---
apiVersion: fn.kpt.bol.com/v1alpha1
kind: MyPackageContext
metadata:
name: unique-name-001.package-context.mydomain.com
annotations:
config.kubernetes.io/local-config: "true"
data:
teamName: example
With the value retrieved above, we'd like to change spec.member to prefix with group: and suffix it with @mydomain.com
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMPolicyMember
jsonPath: spec.member
apply-replacements will not be able to use the first resource as a source, due to its unpredictable name. Unless we also change its FunctionConfig. In hindsight, perhaps that would work with an ensure-name-substring setup with additionalNameFields - if the function supports mutating local-config that is.
The real problem here is lack of documentation that provides solutions to these issues. But perhaps our kpt usage is too exotic?
Q5: Are you using Porch or Porch UI? no
Call for help
We have received many issues about setters and other resource-specific KRM functions. We’d like to better help our users and understand your use cases. At the same time, we’d like to encourage more contributors to help improve the KRM functions together.
To make the discussion more transparent, if you encounter a setter issue or request a setter feature, please share your setter usage (and the friction issues if you have) by filling the questionnaire and reply in the comment.
Questionnaire
Q1: Are you using setters on metadata fields?
Yes
Which metadata fields are you using if the answer to Q1 is true
(please change
- [ ]to- [x]if it applies)
- [x] namespace
- [x] name
- [x] annotations
- [ ] labels
Can you use the following functions instead?
If the following functions do not fulfill your needs, please jump to Q4
In some cases, yes. In some cases, no. I will provide more details in Q4.
Q2: Are you using Google Cloud ConfigConnector resources?
See https://cloud.google.com/config-connector/docs/reference/overview Answer: yes / no.
Yes
Q3: Are you using setters for the GCP specific fields?
(please change
- [ ]to- [x]if it applies)
- [x] Project
- [x] Folder
- [ ] Domain
- [ ] ContainerCluster
- [x] ServiceAccount
- [x] IAMPolicy
- We use both external references (to legacy resources not defined in IaC) and via name and k8s namespace
- [ ] Service
- [x] WorkloadIdentityBinding
Do you have any other GCP fields that are frequently used?
If you have one GCP resource that uses setters and are referred to in multiple places, please add their apiVersion and kind below
Can you use the following curated GCP functions instead?
GCP resources suggested functions User Guide GCP Services gcr.io/kpt-fn/enable-gcp-services:v0.1.0 catalog.kpt.dev/enable-gcp-services/v0.1/ ResourceHierarchy gcr.io/kpt-fn/generate-folders:v0.1.1 catalog.kpt.dev/generate-folders/v0.1/ GCP Project gcr.io/kpt-fn/set-project-id:v0.2.0 catalog.kpt.dev/set-project-id/v0.2/ If the functions in the https://catalog.kpt.dev/?id=gcp-functio|ns do not fulfill the needs, please jump to Q4
We cannot use these functions well. Most of our work is based on GCP Provided blueprints. In fact, the original blueprints we followed for creating much of our infrastructure exclusively used setters. If these blueprints made examples of how the functions are supposed to be used, the intention might be clearer.
The issues for me * It is not clear how these functions work for packages which have resources in different namespaces and projects. * They are just using apply setters under the hood if i understand correctly. What value do these functions add over using setters? * We are a platform team who has explicit interest in managing these packages are understand and define the layout for these packages. Making everything incredibly dynamic and configurable does not provide us a huge value as we are not publishing packages for open source users to consume, but for our internal engineers to use.
Q4: Share your use case
We are a centralized platform team looking at moving all of our infrastructure to be defined in code. We (plus some other related teams) want to manage packages of discrete (a GCS bucket, super simple single API resources) and composite (Service, Pipeline, etc. more complex cross API) packages. We want to define the templates of packages for downstream users to define the setters of the specific resources, then handle the rendering and creation of those Configs via Config Controller.
Our hierarchy is nested for setters to inherently hierarchically into subpackages as they are created.
Can you share the resources and their fieldpath that you apply the setters on?
An overview of all the fields of ours that current have setters and how often
> for f in $(find .); do grep "kpt-set" $f 2>/dev/null; done | cut -d ':' -f1 | tr -s ' ' | sort | uniq -c
4 checkIntervalSec
3 clusterSecondaryRangeName
108 cnrm.cloud.google.com/project-id
13 description
20 displayName
4 dnsName
2 exportCustomRoutes
2 exportSubnetRoutesWithPublicIp
2 - external
97 external
2 gateways
3 gke.io/environment
3 gke.io/nodepool
13 googleServiceAccount
2 # grpcServiceName
4 healthyThreshold
3 identityNamespace
2 importCustomRoutes
2 importSubnetRoutesWithPublicIp
6 - ipCidrRange
5 ipCidrRange
3 issuer
19 kind
2 kind
11 location
3 masterIpv4CidrBlock
3 maxNodeCount
70 - member
27 member
1 #name
8 - name
328 name
1 #namespace
392 namespace
3 port
1 # portName
2 ports
1 portSpecification
1 purpose
6 rangeName
7 region
1 requestPath
5 resourceID
24 role
2 scope
3 # securityGroup
3 servicesSecondaryRangeName
3 tags
4 timeoutSec
4 unhealthyThreshold
If the setter does not work well with your kpt packages or your function pipelines, can you share your kpt packages?
Our packages are stored in a private repo.
Q5: Are you using Porch or Porch UI?
Not currently, though we would like to. Porch lacks support for Gerrit which is where we are currently publishing our packages.
