materialize icon indicating copy to clipboard operation
materialize copied to clipboard

make the materalize cr namespaced

Open doy-materialize opened this issue 1 year ago • 1 comments

Motivation

this allows us to move the backend configuration to a kubernetes secret, which this pr also does, and also remove all setup and cleanup of the data created by the database in favor of just having the credentials provided to us

Checklist

  • [x] This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • [x] This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • [x] If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • [x] If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).
  • [x] If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.

doy-materialize avatar Oct 18 '24 18:10 doy-materialize

the kubernetes yaml i'm using to test this now looks like this:

---
apiVersion: v1
kind: Namespace
metadata:
  name: materialize
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: orchestratord
  namespace: materialize
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: orchestratord
  namespace: materialize
rules:
- apiGroups: [""]
  resources:
  - configmaps
  - persistentvolumeclaims
  - pods
  - secrets
  - serviceaccounts
  - services
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["networking.k8s.io"]
  resources:
  - networkpolicies
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["rbac.authorization.k8s.io"]
  resources:
  - roles
  - rolebindings
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["apps"]
  resources:
  - statefulsets
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["apiextensions.k8s.io"]
  resources:
  - customresourcedefinitions
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["materialize.cloud"]
  resources:
  - materializes
  - materializes/status
  - vpcendpoints
  verbs:
  - create
  - update
  - patch
  - delete
  - get
  - list
  - watch
- apiGroups: ["custom.metrics.k8s.io"]
  resources:
  - persistentvolumeclaims/kubelet_volume_stats_capacity_bytes
  - persistentvolumeclaims/kubelet_volume_stats_used_bytes
  verbs:
  - get
- apiGroups: ["metrics.k8s.io"]
  resources:
  - pods
  verbs:
  - get
  - list
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: orchestratord
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: orchestratord
subjects:
- kind: ServiceAccount
  name: orchestratord
  namespace: materialize
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orchestratord
  namespace: materialize
spec:
  replicas: 1
  selector:
    matchLabels:
      materialize.cloud/app: orchestratord
  template:
    metadata:
      namespace: materialize
      labels:
        materialize.cloud/app: orchestratord
    spec:
      serviceAccountName: orchestratord
      containers:
      - name: orchestratord
        image: materialize/orchestratord:mzbuild-BSGIPDAFDCIMTKT2ECKNL7KBROG6BAPC
        imagePullPolicy: IfNotPresent
        args:
        - "--cloud-provider=local"
        - "--region=kind"
        - "--local-development"
        - "--environmentd-target-arch=amd64"
apiVersion: v1
kind: Namespace
metadata:
  name: materialize-environment
---
apiVersion: v1
kind: Secret
metadata:
  name: materialize-backend-12345678-1234-1234-1234-123456789012
  namespace: materialize-environment
stringData:
  metadata_backend_url: "postgres://materialize_12345678-1234-1234-1234-123456789012:vtk9E1Hr63f10SFwf08KAfMVnzCtafEk@cockroachdb-public.cockroachdb.svc.cluster.local:26257/materialize_12345678-1234-1234-1234-123456789012?sslmode=verify-full&sslrootcert_inline=-----BEGIN%20CERTIFICATE-----%0AMIIDJTCCAg2gAwIBAgIQWHu9BVDdXoemf3%2BNkhPWiDANBgkqhkiG9w0BAQsFADAr%0AMRIwEAYDVQQKEwlDb2Nrcm9hY2gxFTATBgNVBAMTDENvY2tyb2FjaCBDQTAeFw0y%0AMzA1MjcwNDQ0MDFaFw0yNTA1MjcwNDQ0MDFaMCsxEjAQBgNVBAoTCUNvY2tyb2Fj%0AaDEVMBMGA1UEAxMMQ29ja3JvYWNoIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A%0AMIIBCgKCAQEA0CR72MQDJhB%2BJsDGI4BO2ACpDajFUqmjVcpxwz0ZRz1v%2BqXmQjX0%0AHndZPkjTwu1mT6kBOqaiB9vamNaH2KMrC0mx8wBvqkNUh7JpJVxPvb9fXXoF%2BnCL%0ASev6rwOYh%2FQwKJsg8xto0aAShlY3JCdGP4AXt4K%2B5y3sZJ07cdpl9BvFVN17imQP%0AmB28NdmY0yMJsGHI1KzGYc3uFuwLk7DkZZvqflhHH8t2b3PEc3iDHkogYOD65gch%0AnggjelExVm4P7hd5BRnu6XLQhyhM4j85XDnAlYsxw%2BtjQhYpTlWtlFQaO2xoE9nt%0AZ6GUbod1x7m4%2FnyD3UattrUTlcJljEgSywIDAQABo0UwQzAOBgNVHQ8BAf8EBAMC%0AAuQwEgYDVR0TAQH%2FBAgwBgEB%2FwIBATAdBgNVHQ4EFgQUcR1tk1%2FAqWN5LuJauaRY%0AYeVCaqAwDQYJKoZIhvcNAQELBQADggEBACxjXBFoOcj5SlJPMj0wi8fVhhrx%2BQZO%0Ax8oB3Pk%2Fq%2BvxCINAyL%2B9NqlfevvYL9UxXNgs99otTRlJDQ%2Fyjgt9c8AKt%2BvG1Fsr%0AI7AQoxw4nfBqJUsAvCj4wspBBuLPkTziXSomScGpfGL%2B8204ESpbb8e%2Bjp7li7c9%0AfYwK8s2b2xgf45l5%2FDydnAELj8TPCCUUcczqcEPr5njfrPLlB6CR8h07Vh4PJCSC%0ACdH1Iu2FZ9NLpuy%2FxP9ylJxyOy53JN5w%2BDIAgqo4yxYHQXrEOXp71K5kEvI4OAfS%0ABhu2zkT5QXRhwVfutkvxyapik%2FJhV2LxbKlCLni15p19Sne6qKHplVo%3D%0A-----END%20CERTIFICATE-----"
  persist_backend_url: "s3://minio:minio123@bucket/12345678-1234-1234-1234-123456789012?endpoint=http%3A%2F%2Fminio.minio.svc.cluster.local%3A9000&region=minio"
---
apiVersion: materialize.cloud/v1alpha1
kind: Materialize
metadata:
  name: 12345678-1234-1234-1234-123456789012
  namespace: materialize-environment
spec:
  environmentdImageRef: materialize/environmentd:testing

doy-materialize avatar Oct 18 '24 18:10 doy-materialize