materialize
materialize copied to clipboard
make the materalize cr namespaced
Motivation
this allows us to move the backend configuration to a kubernetes secret, which this pr also does, and also remove all setup and cleanup of the data created by the database in favor of just having the credentials provided to us
Checklist
- [x] This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
- [x] This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
- [x] If this PR evolves an existing
$T ⇔ Proto$Tmapping (possibly in a backwards-incompatible way), then it is tagged with aT-protolabel. - [x] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
- [x] If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
the kubernetes yaml i'm using to test this now looks like this:
---
apiVersion: v1
kind: Namespace
metadata:
name: materialize
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: orchestratord
namespace: materialize
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: orchestratord
namespace: materialize
rules:
- apiGroups: [""]
resources:
- configmaps
- persistentvolumeclaims
- pods
- secrets
- serviceaccounts
- services
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["networking.k8s.io"]
resources:
- networkpolicies
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["rbac.authorization.k8s.io"]
resources:
- roles
- rolebindings
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["apps"]
resources:
- statefulsets
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["apiextensions.k8s.io"]
resources:
- customresourcedefinitions
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["materialize.cloud"]
resources:
- materializes
- materializes/status
- vpcendpoints
verbs:
- create
- update
- patch
- delete
- get
- list
- watch
- apiGroups: ["custom.metrics.k8s.io"]
resources:
- persistentvolumeclaims/kubelet_volume_stats_capacity_bytes
- persistentvolumeclaims/kubelet_volume_stats_used_bytes
verbs:
- get
- apiGroups: ["metrics.k8s.io"]
resources:
- pods
verbs:
- get
- list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: orchestratord
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: orchestratord
subjects:
- kind: ServiceAccount
name: orchestratord
namespace: materialize
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: orchestratord
namespace: materialize
spec:
replicas: 1
selector:
matchLabels:
materialize.cloud/app: orchestratord
template:
metadata:
namespace: materialize
labels:
materialize.cloud/app: orchestratord
spec:
serviceAccountName: orchestratord
containers:
- name: orchestratord
image: materialize/orchestratord:mzbuild-BSGIPDAFDCIMTKT2ECKNL7KBROG6BAPC
imagePullPolicy: IfNotPresent
args:
- "--cloud-provider=local"
- "--region=kind"
- "--local-development"
- "--environmentd-target-arch=amd64"
apiVersion: v1
kind: Namespace
metadata:
name: materialize-environment
---
apiVersion: v1
kind: Secret
metadata:
name: materialize-backend-12345678-1234-1234-1234-123456789012
namespace: materialize-environment
stringData:
metadata_backend_url: "postgres://materialize_12345678-1234-1234-1234-123456789012:vtk9E1Hr63f10SFwf08KAfMVnzCtafEk@cockroachdb-public.cockroachdb.svc.cluster.local:26257/materialize_12345678-1234-1234-1234-123456789012?sslmode=verify-full&sslrootcert_inline=-----BEGIN%20CERTIFICATE-----%0AMIIDJTCCAg2gAwIBAgIQWHu9BVDdXoemf3%2BNkhPWiDANBgkqhkiG9w0BAQsFADAr%0AMRIwEAYDVQQKEwlDb2Nrcm9hY2gxFTATBgNVBAMTDENvY2tyb2FjaCBDQTAeFw0y%0AMzA1MjcwNDQ0MDFaFw0yNTA1MjcwNDQ0MDFaMCsxEjAQBgNVBAoTCUNvY2tyb2Fj%0AaDEVMBMGA1UEAxMMQ29ja3JvYWNoIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A%0AMIIBCgKCAQEA0CR72MQDJhB%2BJsDGI4BO2ACpDajFUqmjVcpxwz0ZRz1v%2BqXmQjX0%0AHndZPkjTwu1mT6kBOqaiB9vamNaH2KMrC0mx8wBvqkNUh7JpJVxPvb9fXXoF%2BnCL%0ASev6rwOYh%2FQwKJsg8xto0aAShlY3JCdGP4AXt4K%2B5y3sZJ07cdpl9BvFVN17imQP%0AmB28NdmY0yMJsGHI1KzGYc3uFuwLk7DkZZvqflhHH8t2b3PEc3iDHkogYOD65gch%0AnggjelExVm4P7hd5BRnu6XLQhyhM4j85XDnAlYsxw%2BtjQhYpTlWtlFQaO2xoE9nt%0AZ6GUbod1x7m4%2FnyD3UattrUTlcJljEgSywIDAQABo0UwQzAOBgNVHQ8BAf8EBAMC%0AAuQwEgYDVR0TAQH%2FBAgwBgEB%2FwIBATAdBgNVHQ4EFgQUcR1tk1%2FAqWN5LuJauaRY%0AYeVCaqAwDQYJKoZIhvcNAQELBQADggEBACxjXBFoOcj5SlJPMj0wi8fVhhrx%2BQZO%0Ax8oB3Pk%2Fq%2BvxCINAyL%2B9NqlfevvYL9UxXNgs99otTRlJDQ%2Fyjgt9c8AKt%2BvG1Fsr%0AI7AQoxw4nfBqJUsAvCj4wspBBuLPkTziXSomScGpfGL%2B8204ESpbb8e%2Bjp7li7c9%0AfYwK8s2b2xgf45l5%2FDydnAELj8TPCCUUcczqcEPr5njfrPLlB6CR8h07Vh4PJCSC%0ACdH1Iu2FZ9NLpuy%2FxP9ylJxyOy53JN5w%2BDIAgqo4yxYHQXrEOXp71K5kEvI4OAfS%0ABhu2zkT5QXRhwVfutkvxyapik%2FJhV2LxbKlCLni15p19Sne6qKHplVo%3D%0A-----END%20CERTIFICATE-----"
persist_backend_url: "s3://minio:minio123@bucket/12345678-1234-1234-1234-123456789012?endpoint=http%3A%2F%2Fminio.minio.svc.cluster.local%3A9000®ion=minio"
---
apiVersion: materialize.cloud/v1alpha1
kind: Materialize
metadata:
name: 12345678-1234-1234-1234-123456789012
namespace: materialize-environment
spec:
environmentdImageRef: materialize/environmentd:testing