From 42fd7039e6ea64d2e8e924221b8029a7efaf3faa Mon Sep 17 00:00:00 2001 From: Bonusree Date: Thu, 15 Jan 2026 13:07:58 +0600 Subject: [PATCH 01/14] kafka gitops Signed-off-by: Bonusree --- docs/guides/kafka/gitops/_index.md | 10 + docs/guides/kafka/gitops/gitops.md | 908 +++++++++++++++++++++++++++ docs/guides/kafka/gitops/overview.md | 50 ++ 3 files changed, 968 insertions(+) create mode 100644 docs/guides/kafka/gitops/_index.md create mode 100644 docs/guides/kafka/gitops/gitops.md create mode 100644 docs/guides/kafka/gitops/overview.md diff --git a/docs/guides/kafka/gitops/_index.md b/docs/guides/kafka/gitops/_index.md new file mode 100644 index 000000000..165e948d7 --- /dev/null +++ b/docs/guides/kafka/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Gitops Kafka +menu: + docs_{{ .version }}: + identifier: kf-gitops-kafka + name: Gitops + parent: kf-kafka-guides + weight: 25 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/kafka/gitops/gitops.md b/docs/guides/kafka/gitops/gitops.md new file mode 100644 index 000000000..d9cd087f1 --- /dev/null +++ b/docs/guides/kafka/gitops/gitops.md @@ -0,0 +1,908 @@ +--- +title: GitOps KafkaQL +menu: + docs_{{ .version }}: + identifier: kf-using-gitops + name: GitOps KafkaQL + parent: kf-gitops-Kafka + weight: 15 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Kafka using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create Kafka database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/kafka](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/kafka) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create Kafka Database using GitOps + +### Create a Kafka GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: Standard + controller: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── Kafka.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is created in your cluster. + +Our `gitops` operator will create an actual `Kafka` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get kafka.gitops.kubedb.com,kafka.kubedb.com -n demo +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 2m56s + +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 2m56s +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` Kafka. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=kafka-prod' +NAME AGE +petset.apps.k8s.appscode.com/kafka-prod-broker 4m49s +petset.apps.k8s.appscode.com/kafka-prod-controller 4m47s + +NAME READY STATUS RESTARTS AGE +pod/kafka-prod-broker-0 1/1 Running 0 4m48s +pod/kafka-prod-broker-1 1/1 Running 0 4m42s +pod/kafka-prod-controller-0 1/1 Running 0 4m47s +pod/kafka-prod-controller-1 1/1 Running 0 4m40s + +NAME TYPE DATA AGE +secret/kafka-prod-auth kubernetes.io/basic-auth 2 4m51s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/kafka-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/kafka-prod kubedb.com/kafka 3.9.0 4m47s +``` + +## Update Kafka Database using GitOps + +### Scale Kafka Database Resources + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut + ``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `KafkaOpsRequest` to update the `Kafka` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ $ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 22h + +NAME TYPE STATUS AGE +Kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo Kafka-prod-broker-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1536Mi" + } +} +``` + +### Scale Kafka Replicas +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` KafkaOpsRequest to update the `Kafka` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 22h + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=kafka-prod' +NAME READY STATUS RESTARTS AGE +kafka-prod-broker-0 1/1 Running 0 34m +kafka-prod-broker-1 1/1 Running 0 33m +kafka-prod-broker-2 1/1 Running 0 33m +kafka-prod-controller-0 1/1 Running 0 32m +kafka-prod-controller-1 1/1 Running 0 31m +kafka-prod-controller-2 1/1 Running 0 31m +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Expand Kafka Volume + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` KafkaOpsRequest to update the `Kafka` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 23m + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=kafka-prod' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +kafka-prod-data-kafka-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure Kafka + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/kafka/configuration/kafka-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-kf-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /kf-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ └── Kafka.yaml +1 directories, 2 files +``` + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` KafkaOpsRequest to update the `Kafka` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 74m + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 24m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate Kafka Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, + +```bash +kubectl create secret generic kf-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=kafka \ +--from-literal=password=kafka-secret +secret/kf-rotate-auth created + +``` + + + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + authSecret: + kind: Secret + name: kf-rotate-auth + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` KafkaOpsRequest to update the `Kafka` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 7m11s + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 17h +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 28m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Kafka. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls kafka-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/Kafka-ca created +``` + +Now, Let's create an `Issuer` using the `Kafka-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: kf-issuer + namespace: demo +spec: + ca: + secretName: kafka-ca +``` + +Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ ├── kf-issuer.yaml +│ └── Kafka.yaml +1 directories, 4 files +``` + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` KafkaOpsRequest to update the `Kafka` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops,pods -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 75m + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Kafka. + +### Update Version + +List Kafka versions using `kubectl get Kafkaversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/kafka/update-version/versionupgrading/index.md). + +Let's choose `4.0.0` in this example. + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` KafkaOpsRequest to update the `Kafka` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,kafka,kfops -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 3h47m + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `Kafka`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get Kafka -n demo kafka-prod -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo kafka-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo kafka-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `Kafka.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Kafka +metadata: + name: kafka-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: kafka + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` KafkaOpsRequest to add the `Kafka` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get Kafkaes.gitops.kubedb.com,Kafkaes.kubedb.com,Kafkaopsrequest -n demo +NAME TYPE VERSION STATUS AGE +kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +kafka.gitops.kubedb.com/kafka-prod 5h12m + +NAME TYPE STATUS AGE +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 6d +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +kafkaopsrequest.ops.kubedb.com/kafka-prod-restart-ljpqih Restart Successful 3m51s +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `.spec.leaderElection` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn Kafka [GitOps](/docs/guides/kafka/concepts/Kafka-gitops.md)) +- Learn Kafka Scaling + - [Horizontal Scaling](/docs/guides/kafka/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/kafka/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/kafka/update-version/overview.md) +- Monitor your KafkaQL database with KubeDB using [built-in Prometheus](/docs/guides/kafka/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/kafka/gitops/overview.md b/docs/guides/kafka/gitops/overview.md new file mode 100644 index 000000000..c04150659 --- /dev/null +++ b/docs/guides/kafka/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: Kafka GitOps Overview +description: Kafka GitOps Overview +menu: + docs_{{ .version }}: + identifier: kf-gitops-overview + name: Overview + parent: kf-gitops-kafka + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for Kafka + +This guide will give you an overview of how KubeDB `gitops` operator works with Kafka databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Kafka databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Kafka](/docs/guides/kafka/concepts/kafka.md) + - [KafkaOpsRequest](/docs/guides/kafka/concepts/kafkaopsrequest.md) + + + +## Workflow GitOps with Kafka + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of Kafka
+ +
+ +1. **Define GitOps Kafka**: Create Custom Resource (CR) of kind `Kafka` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Kafka CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the KafkaGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing Kafka databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running Kafka using GitOps. From b056af0d7138958f7ccf91c3f64492eee5c98c5e Mon Sep 17 00:00:00 2001 From: Bonusree Date: Thu, 15 Jan 2026 13:20:14 +0600 Subject: [PATCH 02/14] url fixed Signed-off-by: Bonusree --- docs/guides/kafka/gitops/gitops.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/kafka/gitops/gitops.md b/docs/guides/kafka/gitops/gitops.md index d9cd087f1..509243a7d 100644 --- a/docs/guides/kafka/gitops/gitops.md +++ b/docs/guides/kafka/gitops/gitops.md @@ -707,7 +707,7 @@ kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansi ### Update Version -List Kafka versions using `kubectl get Kafkaversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/kafka/update-version/versionupgrading/index.md). +List Kafka versions using `kubectl get Kafkaversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/kafka/update-version/update-version.md). Let's choose `4.0.0` in this example. From 376a00224d98d07fb4cbe620b3605c69f3c006fb Mon Sep 17 00:00:00 2001 From: Bonusree Date: Thu, 15 Jan 2026 17:16:44 +0600 Subject: [PATCH 03/14] title Signed-off-by: Bonusree --- docs/guides/kafka/gitops/gitops.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/guides/kafka/gitops/gitops.md b/docs/guides/kafka/gitops/gitops.md index 509243a7d..bfcb9ef60 100644 --- a/docs/guides/kafka/gitops/gitops.md +++ b/docs/guides/kafka/gitops/gitops.md @@ -1,15 +1,17 @@ --- -title: GitOps KafkaQL +title: Kafka GitOps Guides +description: Kafka GitOps menu: docs_{{ .version }}: - identifier: kf-using-gitops - name: GitOps KafkaQL - parent: kf-gitops-Kafka - weight: 15 + identifier: kf-gitops-guides + name: Gitops Kafka + parent: kf-gitops-kafka + weight: 20 menu_name: docs_{{ .version }} section_menu_id: guides --- + > New to KubeDB? Please start [here](/docs/README.md). # GitOps Kafka using KubeDB GitOps Operator @@ -889,7 +891,6 @@ There are some other fields that will trigger `Restart` ops request. - `.spec.monitor` - `.spec.spec.archiver` - `.spec.remoteReplica` -- `.spec.leaderElection` - `spec.replication` - `.spec.standbyMode` - `.spec.streamingMode` From 3efed82e8237da0fbd8a350f86f98c90c095863a Mon Sep 17 00:00:00 2001 From: Bonusree Date: Thu, 15 Jan 2026 17:53:50 +0600 Subject: [PATCH 04/14] changed weight Signed-off-by: Bonusree --- docs/guides/kafka/autoscaler/_index.md | 2 +- docs/guides/kafka/cli/_index.md | 2 +- docs/guides/kafka/clustering/_index.md | 2 +- docs/guides/kafka/configuration/_index.md | 2 +- docs/guides/kafka/connectcluster/_index.md | 2 +- docs/guides/kafka/gitops/_index.md | 4 ++-- docs/guides/kafka/migration/_index.md | 2 +- docs/guides/kafka/monitoring/_index.md | 2 +- docs/guides/kafka/reconfigure-tls/_index.md | 2 +- docs/guides/kafka/reconfigure/_index.md | 2 +- docs/guides/kafka/restart/_index.md | 2 +- docs/guides/kafka/restproxy/_index.md | 2 +- docs/guides/kafka/rotate-auth/_index.md | 2 +- docs/guides/kafka/scaling/_index.md | 2 +- docs/guides/kafka/schemaregistry/_index.md | 2 +- docs/guides/kafka/tls/_index.md | 2 +- docs/guides/kafka/update-version/_index.md | 2 +- docs/guides/kafka/volume-expansion/_index.md | 2 +- 18 files changed, 19 insertions(+), 19 deletions(-) diff --git a/docs/guides/kafka/autoscaler/_index.md b/docs/guides/kafka/autoscaler/_index.md index 22a1e3830..c6d7e89ab 100644 --- a/docs/guides/kafka/autoscaler/_index.md +++ b/docs/guides/kafka/autoscaler/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-auto-scaling name: Autoscaling parent: kf-kafka-guides - weight: 46 + weight: 90 menu_name: docs_{{ .version }} --- \ No newline at end of file diff --git a/docs/guides/kafka/cli/_index.md b/docs/guides/kafka/cli/_index.md index 155f42086..e879f4f43 100755 --- a/docs/guides/kafka/cli/_index.md +++ b/docs/guides/kafka/cli/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-cli-kafka name: CLI parent: kf-kafka-guides - weight: 100 + weight: 95 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/clustering/_index.md b/docs/guides/kafka/clustering/_index.md index 21b81a743..6cab222d4 100755 --- a/docs/guides/kafka/clustering/_index.md +++ b/docs/guides/kafka/clustering/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-clustering name: Clustering parent: kf-kafka-guides - weight: 20 + weight: 25 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/configuration/_index.md b/docs/guides/kafka/configuration/_index.md index 81167c2af..1ef769c0d 100644 --- a/docs/guides/kafka/configuration/_index.md +++ b/docs/guides/kafka/configuration/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-configuration name: Custom Configuration parent: kf-kafka-guides - weight: 30 + weight: 35 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/connectcluster/_index.md b/docs/guides/kafka/connectcluster/_index.md index 7d833caaa..44991d14d 100644 --- a/docs/guides/kafka/connectcluster/_index.md +++ b/docs/guides/kafka/connectcluster/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-connectcluster-guides name: ConnectCluster parent: kf-kafka-guides - weight: 25 + weight: 40 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/gitops/_index.md b/docs/guides/kafka/gitops/_index.md index 165e948d7..098093afa 100644 --- a/docs/guides/kafka/gitops/_index.md +++ b/docs/guides/kafka/gitops/_index.md @@ -2,9 +2,9 @@ title: Gitops Kafka menu: docs_{{ .version }}: - identifier: kf-gitops-kafka + identifier: kf-gitops name: Gitops parent: kf-kafka-guides - weight: 25 + weight: 30 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/migration/_index.md b/docs/guides/kafka/migration/_index.md index 0f8b4d2e1..84e319492 100644 --- a/docs/guides/kafka/migration/_index.md +++ b/docs/guides/kafka/migration/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-migration-kafka name: Migration parent: kf-kafka-guides - weight: 27 + weight: 60 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/monitoring/_index.md b/docs/guides/kafka/monitoring/_index.md index 05429b728..16cc76ef2 100755 --- a/docs/guides/kafka/monitoring/_index.md +++ b/docs/guides/kafka/monitoring/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-monitoring-kafka name: Monitoring parent: kf-kafka-guides - weight: 50 + weight: 60 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/reconfigure-tls/_index.md b/docs/guides/kafka/reconfigure-tls/_index.md index 5b2552a6d..e4f77b489 100644 --- a/docs/guides/kafka/reconfigure-tls/_index.md +++ b/docs/guides/kafka/reconfigure-tls/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-reconfigure-tls name: Reconfigure TLS/SSL parent: kf-kafka-guides - weight: 46 + weight: 55 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/reconfigure/_index.md b/docs/guides/kafka/reconfigure/_index.md index 86d8888b6..7a9181a21 100644 --- a/docs/guides/kafka/reconfigure/_index.md +++ b/docs/guides/kafka/reconfigure/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-reconfigure name: Reconfigure parent: kf-kafka-guides - weight: 46 + weight: 45 menu_name: docs_{{ .version }} --- \ No newline at end of file diff --git a/docs/guides/kafka/restart/_index.md b/docs/guides/kafka/restart/_index.md index d0d4240b4..1199fe4ae 100644 --- a/docs/guides/kafka/restart/_index.md +++ b/docs/guides/kafka/restart/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-restart name: Restart parent: kf-kafka-guides - weight: 46 + weight: 75 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/restproxy/_index.md b/docs/guides/kafka/restproxy/_index.md index b02df6b52..1e1e16820 100644 --- a/docs/guides/kafka/restproxy/_index.md +++ b/docs/guides/kafka/restproxy/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-rest-proxy-guides name: RestProxy parent: kf-kafka-guides - weight: 25 + weight: 60 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/rotate-auth/_index.md b/docs/guides/kafka/rotate-auth/_index.md index 840907389..ed2cf8a38 100644 --- a/docs/guides/kafka/rotate-auth/_index.md +++ b/docs/guides/kafka/rotate-auth/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-rotate-auth name: Rotate Authentication parent: kf-kafka-guides - weight: 45 + weight: 85 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/scaling/_index.md b/docs/guides/kafka/scaling/_index.md index 98b83c710..48781a47d 100644 --- a/docs/guides/kafka/scaling/_index.md +++ b/docs/guides/kafka/scaling/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-scaling name: Scaling parent: kf-kafka-guides - weight: 43 + weight: 70 menu_name: docs_{{ .version }} --- \ No newline at end of file diff --git a/docs/guides/kafka/schemaregistry/_index.md b/docs/guides/kafka/schemaregistry/_index.md index 31f884753..64a121265 100644 --- a/docs/guides/kafka/schemaregistry/_index.md +++ b/docs/guides/kafka/schemaregistry/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-schema-registry-guides name: SchemaRegistry parent: kf-kafka-guides - weight: 25 + weight: 60 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/tls/_index.md b/docs/guides/kafka/tls/_index.md index 1b5fe9d94..d92320672 100755 --- a/docs/guides/kafka/tls/_index.md +++ b/docs/guides/kafka/tls/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-tls name: TLS/SSL Encryption parent: kf-kafka-guides - weight: 45 + weight: 50 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/kafka/update-version/_index.md b/docs/guides/kafka/update-version/_index.md index 08f8af5d4..2581cfd25 100644 --- a/docs/guides/kafka/update-version/_index.md +++ b/docs/guides/kafka/update-version/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-update-version name: UpdateVersion parent: kf-kafka-guides - weight: 42 + weight: 65 menu_name: docs_{{ .version }} --- \ No newline at end of file diff --git a/docs/guides/kafka/volume-expansion/_index.md b/docs/guides/kafka/volume-expansion/_index.md index 27c8e1f8b..dddfb1563 100644 --- a/docs/guides/kafka/volume-expansion/_index.md +++ b/docs/guides/kafka/volume-expansion/_index.md @@ -5,6 +5,6 @@ menu: identifier: kf-volume-expansion name: Volume Expansion parent: kf-kafka-guides - weight: 44 + weight: 80 menu_name: docs_{{ .version }} --- \ No newline at end of file From 748ea45119d8245336ccdad7a481187af21c976f Mon Sep 17 00:00:00 2001 From: Bonusree Date: Fri, 16 Jan 2026 15:33:51 +0600 Subject: [PATCH 05/14] organize front Signed-off-by: Bonusree --- docs/guides/druid/gitops/_index.md | 11 + docs/guides/druid/gitops/gitops.md | 906 +++++++++++++++++++++++++++ docs/guides/druid/gitops/overview.md | 50 ++ docs/guides/kafka/gitops/gitops.md | 1 - docs/guides/kafka/gitops/overview.md | 6 +- 5 files changed, 970 insertions(+), 4 deletions(-) create mode 100644 docs/guides/druid/gitops/_index.md create mode 100644 docs/guides/druid/gitops/gitops.md create mode 100644 docs/guides/druid/gitops/overview.md diff --git a/docs/guides/druid/gitops/_index.md b/docs/guides/druid/gitops/_index.md new file mode 100644 index 000000000..c7207d6eb --- /dev/null +++ b/docs/guides/druid/gitops/_index.md @@ -0,0 +1,11 @@ +--- +title: Druid Gitops +menu: + docs_{{ .version }}: + identifier: guides-druid-gitops + name: Gitops + parent: guides-druid + weight: 45 +menu_name: docs_{{ .version }} +--- + diff --git a/docs/guides/druid/gitops/gitops.md b/docs/guides/druid/gitops/gitops.md new file mode 100644 index 000000000..e974f667e --- /dev/null +++ b/docs/guides/druid/gitops/gitops.md @@ -0,0 +1,906 @@ +--- +title: Druid Gitops Guide +menu: + docs_{{ .version }}: + identifier: druid-gitops-guide + name: Druid Gitops + parent: guides-druid-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Druid using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create Druid database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/druid](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/druid) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create Druid Database using GitOps + +### Create a Druid GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: Standard + controller: + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── Druid.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is created in your cluster. + +Our `gitops` operator will create an actual `Druid` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get Druid.gitops.kubedb.com,Druid.kubedb.com -n demo +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 2m56s + +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 2m56s +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` Druid. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Druid-prod' +NAME AGE +petset.apps.k8s.appscode.com/Druid-prod-broker 4m49s +petset.apps.k8s.appscode.com/Druid-prod-controller 4m47s + +NAME READY STATUS RESTARTS AGE +pod/Druid-prod-broker-0 1/1 Running 0 4m48s +pod/Druid-prod-broker-1 1/1 Running 0 4m42s +pod/Druid-prod-controller-0 1/1 Running 0 4m47s +pod/Druid-prod-controller-1 1/1 Running 0 4m40s + +NAME TYPE DATA AGE +secret/Druid-prod-auth kubernetes.io/basic-auth 2 4m51s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/Druid-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/Druid-prod kubedb.com/Druid 3.9.0 4m47s +``` + +## Update Druid Database using GitOps + +### Scale Druid Database Resources + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut + ``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `DruidOpsRequest` to update the `Druid` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ $ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 22h + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo Druid-prod-broker-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1536Mi" + } +} +``` + +### Scale Druid Replicas +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` DruidOpsRequest to update the `Druid` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 22h + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Druid-prod' +NAME READY STATUS RESTARTS AGE +Druid-prod-broker-0 1/1 Running 0 34m +Druid-prod-broker-1 1/1 Running 0 33m +Druid-prod-broker-2 1/1 Running 0 33m +Druid-prod-controller-0 1/1 Running 0 32m +Druid-prod-controller-1 1/1 Running 0 31m +Druid-prod-controller-2 1/1 Running 0 31m +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Expand Druid Volume + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` DruidOpsRequest to update the `Druid` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 23m + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Druid-prod' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +Druid-prod-data-Druid-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +Druid-prod-data-Druid-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +Druid-prod-data-Druid-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +Druid-prod-data-Druid-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure Druid + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/Druid/configuration/Druid-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-kf-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /kf-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ └── Druid.yaml +1 directories, 2 files +``` + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` DruidOpsRequest to update the `Druid` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 74m + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 24m +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate Druid Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, + +```bash +kubectl create secret generic kf-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=Druid \ +--from-literal=password=Druid-secret +secret/kf-rotate-auth created + +``` + + + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + authSecret: + kind: Secret + name: kf-rotate-auth + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` DruidOpsRequest to update the `Druid` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 7m11s + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 17h +Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 28m +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Druid. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls Druid-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/Druid-ca created +``` + +Now, Let's create an `Issuer` using the `Druid-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: kf-issuer + namespace: demo +spec: + ca: + secretName: Druid-ca +``` + +Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ ├── kf-issuer.yaml +│ └── Druid.yaml +1 directories, 4 files +``` + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` DruidOpsRequest to update the `Druid` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops,pods -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 75m + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Druid. + +### Update Version + +List Druid versions using `kubectl get Druidversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Druid/update-version/update-version.md). + +Let's choose `4.0.0` in this example. + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` DruidOpsRequest to update the `Druid` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Druid,kfops -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 3h47m + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +Druidopsrequest.ops.kubedb.com/Druid-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `Druid`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get Druid -n demo Druid-prod -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo Druid-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Druid:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo Druid-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Druid:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `Druid.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Druid +metadata: + name: Druid-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Druid + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` DruidOpsRequest to add the `Druid` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get Druides.gitops.kubedb.com,Druides.kubedb.com,Druidopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Druid.kubedb.com/Druid-prod kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +Druid.gitops.kubedb.com/Druid-prod 5h12m + +NAME TYPE STATUS AGE +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 6d +Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Druidopsrequest.ops.kubedb.com/Druid-prod-restart-ljpqih Restart Successful 3m51s +Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +Druidopsrequest.ops.kubedb.com/Druid-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn Druid [GitOps](/docs/guides/Druid/concepts/Druid-gitops.md)) +- Learn Druid Scaling + - [Horizontal Scaling](/docs/guides/Druid/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/Druid/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/Druid/update-version/overview.md) +- Monitor your DruidQL database with KubeDB using [built-in Prometheus](/docs/guides/Druid/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/druid/gitops/overview.md b/docs/guides/druid/gitops/overview.md new file mode 100644 index 000000000..05e8c5433 --- /dev/null +++ b/docs/guides/druid/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: Druid GitOps Overview +description: Druid GitOps Overview +menu: + docs_{{ .version }}: + identifier: kf-gitops-overview + name: Overview + parent: kf-gitops-Druid + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for Druid + +This guide will give you an overview of how KubeDB `gitops` operator works with Druid databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Druid databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Druid](/docs/guides/druid/concepts/druid.md) + - [DruidOpsRequest](/docs/guides/druid/concepts/druidopsrequest.md) + + + +## Workflow GitOps with Druid + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of Druid
+ +
+ +1. **Define GitOps Druid**: Create Custom Resource (CR) of kind `Druid` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Druid CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the DruidGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing Druid databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running Druid using GitOps. diff --git a/docs/guides/kafka/gitops/gitops.md b/docs/guides/kafka/gitops/gitops.md index bfcb9ef60..e5b5b2d96 100644 --- a/docs/guides/kafka/gitops/gitops.md +++ b/docs/guides/kafka/gitops/gitops.md @@ -1,6 +1,5 @@ --- title: Kafka GitOps Guides -description: Kafka GitOps menu: docs_{{ .version }}: identifier: kf-gitops-guides diff --git a/docs/guides/kafka/gitops/overview.md b/docs/guides/kafka/gitops/overview.md index c04150659..cf959a479 100644 --- a/docs/guides/kafka/gitops/overview.md +++ b/docs/guides/kafka/gitops/overview.md @@ -1,11 +1,11 @@ --- -title: Kafka GitOps Overview -description: Kafka GitOps Overview +title: Kafka Gitops Overview +description: Kafka Gitops Overview menu: docs_{{ .version }}: identifier: kf-gitops-overview name: Overview - parent: kf-gitops-kafka + parent: kf-gitops weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides From eda323b67493ee6935e5116a244408d465c0d7ac Mon Sep 17 00:00:00 2001 From: Bonusree Date: Wed, 21 Jan 2026 15:05:03 +0600 Subject: [PATCH 06/14] Signed-off-by: Bonusree --- docs/guides/druid/gitops/_index.md | 11 - docs/guides/elasticsearch/gitops/_index.md | 10 + .../{druid => elasticsearch}/gitops/gitops.md | 360 +++++++++--------- .../gitops/overview.md | 30 +- 4 files changed, 206 insertions(+), 205 deletions(-) delete mode 100644 docs/guides/druid/gitops/_index.md create mode 100644 docs/guides/elasticsearch/gitops/_index.md rename docs/guides/{druid => elasticsearch}/gitops/gitops.md (71%) rename docs/guides/{druid => elasticsearch}/gitops/overview.md (63%) diff --git a/docs/guides/druid/gitops/_index.md b/docs/guides/druid/gitops/_index.md deleted file mode 100644 index c7207d6eb..000000000 --- a/docs/guides/druid/gitops/_index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Druid Gitops -menu: - docs_{{ .version }}: - identifier: guides-druid-gitops - name: Gitops - parent: guides-druid - weight: 45 -menu_name: docs_{{ .version }} ---- - diff --git a/docs/guides/elasticsearch/gitops/_index.md b/docs/guides/elasticsearch/gitops/_index.md new file mode 100644 index 000000000..a13bc5830 --- /dev/null +++ b/docs/guides/elasticsearch/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Elasticsearch Gitops +menu: + docs_{{ .version }}: + identifier: es-gitops + name: Elasticsearch Gitops + parent: es-elasticsearch-guides + weight: 65 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/druid/gitops/gitops.md b/docs/guides/elasticsearch/gitops/gitops.md similarity index 71% rename from docs/guides/druid/gitops/gitops.md rename to docs/guides/elasticsearch/gitops/gitops.md index e974f667e..e5b5b2d96 100644 --- a/docs/guides/druid/gitops/gitops.md +++ b/docs/guides/elasticsearch/gitops/gitops.md @@ -1,19 +1,21 @@ --- -title: Druid Gitops Guide +title: Kafka GitOps Guides menu: - docs_{{ .version }}: - identifier: druid-gitops-guide - name: Druid Gitops - parent: guides-druid-gitops - weight: 10 + docs_{{ .version }}: + identifier: kf-gitops-guides + name: Gitops Kafka + parent: kf-gitops-kafka + weight: 20 menu_name: docs_{{ .version }} section_menu_id: guides --- + + > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Druid using KubeDB GitOps Operator +# GitOps Kafka using KubeDB GitOps Operator -This guide will show you how to use `KubeDB` GitOps operator to create Druid database and manage updates using GitOps workflow. +This guide will show you how to use `KubeDB` GitOps operator to create Kafka database and manage updates using GitOps workflow. ## Before You Begin @@ -30,7 +32,7 @@ This guide will show you how to use `KubeDB` GitOps operator to create Druid dat $ kubectl create ns demo namespace/demo created ``` -> Note: YAML files used in this tutorial are stored in [docs/examples/druid](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/druid) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). +> Note: YAML files used in this tutorial are stored in [docs/examples/kafka](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/kafka) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). @@ -52,14 +54,14 @@ argocd app create kubedb --repo --path kubedb --dest-server https://k argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa ``` -## Create Druid Database using GitOps +## Create Kafka Database using GitOps -### Create a Druid GitOps CR +### Create a Kafka GitOps CR ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 3.9.0 @@ -90,58 +92,58 @@ Create a directory like below, ```bash $ tree . ├── kubedb - └── Druid.yaml + └── Kafka.yaml 1 directories, 1 files ``` -Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is created in your cluster. +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is created in your cluster. -Our `gitops` operator will create an actual `Druid` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. +Our `gitops` operator will create an actual `Kafka` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get Druid.gitops.kubedb.com,Druid.kubedb.com -n demo +$ kubectl get kafka.gitops.kubedb.com,kafka.kubedb.com -n demo NAME AGE -Druid.gitops.kubedb.com/Druid-prod 2m56s +kafka.gitops.kubedb.com/kafka-prod 2m56s NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 2m56s +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 2m56s ``` -List the resources created by `kubedb` operator created for `kubedb.com/v1` Druid. +List the resources created by `kubedb` operator created for `kubedb.com/v1` Kafka. ```bash -$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Druid-prod' +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=kafka-prod' NAME AGE -petset.apps.k8s.appscode.com/Druid-prod-broker 4m49s -petset.apps.k8s.appscode.com/Druid-prod-controller 4m47s +petset.apps.k8s.appscode.com/kafka-prod-broker 4m49s +petset.apps.k8s.appscode.com/kafka-prod-controller 4m47s NAME READY STATUS RESTARTS AGE -pod/Druid-prod-broker-0 1/1 Running 0 4m48s -pod/Druid-prod-broker-1 1/1 Running 0 4m42s -pod/Druid-prod-controller-0 1/1 Running 0 4m47s -pod/Druid-prod-controller-1 1/1 Running 0 4m40s +pod/kafka-prod-broker-0 1/1 Running 0 4m48s +pod/kafka-prod-broker-1 1/1 Running 0 4m42s +pod/kafka-prod-controller-0 1/1 Running 0 4m47s +pod/kafka-prod-controller-1 1/1 Running 0 4m40s NAME TYPE DATA AGE -secret/Druid-prod-auth kubernetes.io/basic-auth 2 4m51s +secret/kafka-prod-auth kubernetes.io/basic-auth 2 4m51s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/Druid-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s +service/kafka-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s NAME TYPE VERSION AGE -appbinding.appcatalog.appscode.com/Druid-prod kubedb.com/Druid 3.9.0 4m47s +appbinding.appcatalog.appscode.com/kafka-prod kubedb.com/kafka 3.9.0 4m47s ``` -## Update Druid Database using GitOps +## Update Kafka Database using GitOps -### Scale Druid Database Resources +### Scale Kafka Database Resources -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 3.9.0 @@ -150,7 +152,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1540Mi @@ -169,7 +171,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1540Mi @@ -188,25 +190,25 @@ spec: deletionPolicy: WipeOut ``` -Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the resource changes and create a `DruidOpsRequest` to update the `Druid` database. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the resource changes and create a `KafkaOpsRequest` to update the `Kafka` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,Druid,kfops -n demo +$ $ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 22h +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Druid.gitops.kubedb.com/Druid-prod 22h +kafka.gitops.kubedb.com/kafka-prod 22h NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +Kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, ```bash -$ kubectl get pod -n demo Druid-prod-broker-0 -o json | jq '.spec.containers[0].resources' +$ kubectl get pod -n demo Kafka-prod-broker-0 -o json | jq '.spec.containers[0].resources' { "limits": { "memory": "1536Mi" @@ -218,13 +220,13 @@ $ kubectl get pod -n demo Druid-prod-broker-0 -o json | jq '.spec.containers[0]. } ``` -### Scale Druid Replicas -Update the `Druid.yaml` with the following, +### Scale Kafka Replicas +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 3.9.0 @@ -233,7 +235,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1540Mi @@ -252,7 +254,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1540Mi @@ -271,44 +273,44 @@ spec: deletionPolicy: WipeOut ``` -Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. -Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` DruidOpsRequest to update the `Druid` database replicas. List the resources created by `gitops` operator in the `demo` namespace. +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` KafkaOpsRequest to update the `Kafka` database replicas. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops -n demo +$ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 22h +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Druid.gitops.kubedb.com/Druid-prod 22h +kafka.gitops.kubedb.com/kafka-prod 22h NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, ```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Druid-prod' +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=kafka-prod' NAME READY STATUS RESTARTS AGE -Druid-prod-broker-0 1/1 Running 0 34m -Druid-prod-broker-1 1/1 Running 0 33m -Druid-prod-broker-2 1/1 Running 0 33m -Druid-prod-controller-0 1/1 Running 0 32m -Druid-prod-controller-1 1/1 Running 0 31m -Druid-prod-controller-2 1/1 Running 0 31m +kafka-prod-broker-0 1/1 Running 0 34m +kafka-prod-broker-1 1/1 Running 0 33m +kafka-prod-broker-2 1/1 Running 0 33m +kafka-prod-controller-0 1/1 Running 0 32m +kafka-prod-controller-1 1/1 Running 0 31m +kafka-prod-controller-2 1/1 Running 0 31m ``` We can also scale down the replicas by updating the `replicas` fields. -### Expand Druid Volume +### Expand Kafka Volume -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 3.9.0 @@ -317,7 +319,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -336,7 +338,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -355,39 +357,39 @@ spec: deletionPolicy: WipeOut ``` -Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` DruidOpsRequest to update the `Druid` database volume. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` KafkaOpsRequest to update the `Kafka` database volume. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops -n demo +$ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 23m +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 23m NAME AGE -Druid.gitops.kubedb.com/Druid-prod 23m +kafka.gitops.kubedb.com/kafka-prod 23m NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Druidopsrequest.ops.kubedb.com/Druid-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m ``` After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, ```bash -$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Druid-prod' +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=kafka-prod' NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE -Druid-prod-data-Druid-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m -Druid-prod-data-Druid-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m -Druid-prod-data-Druid-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m -Druid-prod-data-Druid-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +kafka-prod-data-kafka-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m ``` -## Reconfigure Druid +## Reconfigure Kafka At first, we will create a secret containing `user.conf` file with required configuration settings. -To know more about this configuration file, check [here](/docs/guides/Druid/configuration/Druid-combined.md) +To know more about this configuration file, check [here](/docs/guides/kafka/configuration/kafka-combined.md) ```yaml apiVersion: v1 kind: Secret @@ -405,16 +407,16 @@ Now, we will add this file to `kubedb /kf-configuration.yaml`. $ tree . ├── kubedb │ ├── kf-configuration.yaml -│ └── Druid.yaml +│ └── Kafka.yaml 1 directories, 2 files ``` -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: configSecret: @@ -425,7 +427,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -444,7 +446,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -463,21 +465,21 @@ spec: deletionPolicy: WipeOut ``` -Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` DruidOpsRequest to update the `Druid` database configuration. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` KafkaOpsRequest to update the `Kafka` database configuration. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops -n demo +$ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 74m +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 74m NAME AGE -Druid.gitops.kubedb.com/Druid-prod 74m +kafka.gitops.kubedb.com/kafka-prod 74m NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 24m -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 24m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m ``` @@ -485,7 +487,7 @@ Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansi > We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. -### Rotate Druid Auth +### Rotate Kafka Auth To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. @@ -494,20 +496,20 @@ We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the fo ```bash kubectl create secret generic kf-rotate-auth -n demo \ --type=kubernetes.io/basic-auth \ ---from-literal=username=Druid \ ---from-literal=password=Druid-secret +--from-literal=username=kafka \ +--from-literal=password=kafka-secret secret/kf-rotate-auth created ``` -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: authSecret: @@ -521,7 +523,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -540,7 +542,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -559,22 +561,22 @@ spec: deletionPolicy: WipeOut ``` -Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the auth changes and create a `RotateAuth` DruidOpsRequest to update the `Druid` database auth. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` KafkaOpsRequest to update the `Kafka` database auth. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops -n demo +$ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 7m11s +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 7m11s NAME AGE -Druid.gitops.kubedb.com/Druid-prod 7m11s +kafka.gitops.kubedb.com/kafka-prod 7m11s NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 17h -Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 28m -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 17h +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 28m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h ``` @@ -583,7 +585,7 @@ Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansi We can add, rotate or remove TLS configuration using `gitops`. -To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Druid. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Kafka. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. - Start off by generating a ca certificates using openssl. @@ -599,14 +601,14 @@ writing new private key to './ca.key' - Now we are going to create a ca-secret using the certificate files that we have just generated. ```bash -$ kubectl create secret tls Druid-ca \ +$ kubectl create secret tls kafka-ca \ --cert=ca.crt \ --key=ca.key \ --namespace=demo -secret/Druid-ca created +secret/Kafka-ca created ``` -Now, Let's create an `Issuer` using the `Druid-ca` secret that we have just created. The `YAML` file looks like this: +Now, Let's create an `Issuer` using the `Kafka-ca` secret that we have just created. The `YAML` file looks like this: ```yaml apiVersion: cert-manager.io/v1 @@ -616,7 +618,7 @@ metadata: namespace: demo spec: ca: - secretName: Druid-ca + secretName: kafka-ca ``` Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, @@ -625,16 +627,16 @@ $ tree . ├── kubedb │ ├── kf-configuration.yaml │ ├── kf-issuer.yaml -│ └── Druid.yaml +│ └── Kafka.yaml 1 directories, 4 files ``` -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 3.9.0 @@ -643,7 +645,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -662,7 +664,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -681,41 +683,41 @@ spec: deletionPolicy: WipeOut ``` -Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` DruidOpsRequest to update the `Druid` database tls. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` KafkaOpsRequest to update the `Kafka` database tls. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops,pods -n demo +$ kubectl get kf,kafka,kfops,pods -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 3.9.0 Ready 41m +kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 41m NAME AGE -Druid.gitops.kubedb.com/Druid-prod 75m +kafka.gitops.kubedb.com/kafka-prod 75m NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 5d18h -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s -Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h ``` -> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Druid. +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Kafka. ### Update Version -List Druid versions using `kubectl get Druidversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Druid/update-version/update-version.md). +List Kafka versions using `kubectl get Kafkaversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/kafka/update-version/update-version.md). Let's choose `4.0.0` in this example. -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 4.0.0 @@ -724,7 +726,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -743,7 +745,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -762,50 +764,50 @@ spec: deletionPolicy: WipeOut ``` -Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the version changes and create a `VersionUpdate` DruidOpsRequest to update the `Druid` database version. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` KafkaOpsRequest to update the `Kafka` database version. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Druid,kfops -n demo +$ kubectl get kf,kafka,kfops -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 4.0.0 Ready 3h47m +kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 3h47m NAME AGE -Druid.gitops.kubedb.com/Druid-prod 3h47m +kafka.gitops.kubedb.com/kafka-prod 3h47m NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 5d22h -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m -Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h -Druidopsrequest.ops.kubedb.com/Druid-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h ``` -Now, we are going to verify whether the `Druid`, `PetSet` and it's `Pod` have updated with new image. Let's check, +Now, we are going to verify whether the `Kafka`, `PetSet` and it's `Pod` have updated with new image. Let's check, ```bash -$ kubectl get Druid -n demo Druid-prod -o=jsonpath='{.spec.version}{"\n"}' +$ kubectl get Kafka -n demo kafka-prod -o=jsonpath='{.spec.version}{"\n"}' 4.0.0 -$ kubectl get petset -n demo Druid-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Druid:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get petset -n demo kafka-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 -$ kubectl get pod -n demo Druid-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Druid:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get pod -n demo kafka-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 ``` ### Enable Monitoring If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). -Update the `Druid.yaml` with the following, +Update the `Kafka.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Druid +kind: Kafka metadata: - name: Druid-prod + name: kafka-prod namespace: demo spec: version: 4.0.0 @@ -814,7 +816,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -833,7 +835,7 @@ spec: podTemplate: spec: containers: - - name: Druid + - name: kafka resources: limits: memory: 1536Mi @@ -861,24 +863,24 @@ spec: deletionPolicy: WipeOut ``` -Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Druid` CR is updated in your cluster. +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the monitoring changes and create a `Restart` DruidOpsRequest to add the `Druid` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the monitoring changes and create a `Restart` KafkaOpsRequest to add the `Kafka` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get Druides.gitops.kubedb.com,Druides.kubedb.com,Druidopsrequest -n demo +$ kubectl get Kafkaes.gitops.kubedb.com,Kafkaes.kubedb.com,Kafkaopsrequest -n demo NAME TYPE VERSION STATUS AGE -Druid.kubedb.com/Druid-prod kubedb.com/v1 4.0.0 Ready 5h12m +kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 5h12m NAME AGE -Druid.gitops.kubedb.com/Druid-prod 5h12m +kafka.gitops.kubedb.com/kafka-prod 5h12m NAME TYPE STATUS AGE -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfigure-ukj41o Reconfigure Successful 6d -Druidopsrequest.ops.kubedb.com/Druid-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m -Druidopsrequest.ops.kubedb.com/Druid-prod-restart-ljpqih Restart Successful 3m51s -Druidopsrequest.ops.kubedb.com/Druid-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h -Druidopsrequest.ops.kubedb.com/Druid-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m -Druidopsrequest.ops.kubedb.com/Druid-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 6d +kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +kafkaopsrequest.ops.kubedb.com/kafka-prod-restart-ljpqih Restart Successful 3m51s +kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d ``` @@ -897,10 +899,10 @@ There are some other fields that will trigger `Restart` ops request. ## Next Steps -[//]: # (- Learn Druid [GitOps](/docs/guides/Druid/concepts/Druid-gitops.md)) -- Learn Druid Scaling - - [Horizontal Scaling](/docs/guides/Druid/scaling/horizontal-scaling/combined.md) - - [Vertical Scaling](/docs/guides/Druid/scaling/vertical-scaling/combined.md) -- Learn Version Update Ops Request and Constraints [here](/docs/guides/Druid/update-version/overview.md) -- Monitor your DruidQL database with KubeDB using [built-in Prometheus](/docs/guides/Druid/monitoring/using-builtin-prometheus.md). +[//]: # (- Learn Kafka [GitOps](/docs/guides/kafka/concepts/Kafka-gitops.md)) +- Learn Kafka Scaling + - [Horizontal Scaling](/docs/guides/kafka/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/kafka/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/kafka/update-version/overview.md) +- Monitor your KafkaQL database with KubeDB using [built-in Prometheus](/docs/guides/kafka/monitoring/using-builtin-prometheus.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/druid/gitops/overview.md b/docs/guides/elasticsearch/gitops/overview.md similarity index 63% rename from docs/guides/druid/gitops/overview.md rename to docs/guides/elasticsearch/gitops/overview.md index 05e8c5433..cf959a479 100644 --- a/docs/guides/druid/gitops/overview.md +++ b/docs/guides/elasticsearch/gitops/overview.md @@ -1,11 +1,11 @@ --- -title: Druid GitOps Overview -description: Druid GitOps Overview +title: Kafka Gitops Overview +description: Kafka Gitops Overview menu: docs_{{ .version }}: identifier: kf-gitops-overview name: Overview - parent: kf-gitops-Druid + parent: kf-gitops weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides @@ -13,20 +13,20 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Overview for Druid +# GitOps Overview for Kafka -This guide will give you an overview of how KubeDB `gitops` operator works with Druid databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for -managing Druid databases in Kubernetes. +This guide will give you an overview of how KubeDB `gitops` operator works with Kafka databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Kafka databases in Kubernetes. ## Before You Begin - You should be familiar with the following `KubeDB` concepts: - - [Druid](/docs/guides/druid/concepts/druid.md) - - [DruidOpsRequest](/docs/guides/druid/concepts/druidopsrequest.md) + - [Kafka](/docs/guides/kafka/concepts/kafka.md) + - [KafkaOpsRequest](/docs/guides/kafka/concepts/kafkaopsrequest.md) -## Workflow GitOps with Druid +## Workflow GitOps with Kafka The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. @@ -35,16 +35,16 @@ The following diagram shows how the `KubeDB` GitOps Operator used to sync with y GitOps Flow -
Fig: GitOps process of Druid
+
Fig: GitOps process of Kafka
-1. **Define GitOps Druid**: Create Custom Resource (CR) of kind `Druid` using the `gitops.kubedb.com/v1alpha1` API. +1. **Define GitOps Kafka**: Create Custom Resource (CR) of kind `Kafka` using the `gitops.kubedb.com/v1alpha1` API. 2. **Store in Git**: Push the CR to a Git repository. 3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. -4. **Create Database**: The GitOps operator creates a corresponding KubeDB Druid CR in the Kubernetes cluster to deploy the database. -5. **Handle Updates**: When you update the DruidGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Kafka CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the KafkaGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. -This flow makes managing Druid databases efficient, reliable, and fully integrated with GitOps practices. +This flow makes managing Kafka databases efficient, reliable, and fully integrated with GitOps practices. -In the next doc, we are going to show a step by step guide on running Druid using GitOps. +In the next doc, we are going to show a step by step guide on running Kafka using GitOps. From cd9376a28561f3f03bd0ab593c80b9b01860b0d0 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 27 Jan 2026 17:16:47 +0600 Subject: [PATCH 07/14] elasticsearch without tls Signed-off-by: Bonusree --- docs/guides/elasticsearch/gitops/_index.md | 2 +- docs/guides/elasticsearch/gitops/gitops.md | 957 ++++++++----------- docs/guides/elasticsearch/gitops/overview.md | 32 +- docs/guides/kafka/gitops/gitops.md | 6 +- docs/guides/mariadb/gitops/_index.md | 10 + docs/guides/mariadb/gitops/gitops.md | 910 ++++++++++++++++++ docs/guides/mariadb/gitops/overview.md | 50 + docs/guides/mongodb/gitops/_index.md | 10 + docs/guides/mongodb/gitops/gitops.md | 900 +++++++++++++++++ docs/guides/mongodb/gitops/overview.md | 50 + docs/guides/mssqlserver/gitops/_index.md | 10 + docs/guides/mssqlserver/gitops/gitops.md | 900 +++++++++++++++++ docs/guides/mssqlserver/gitops/overview.md | 50 + 13 files changed, 3317 insertions(+), 570 deletions(-) create mode 100644 docs/guides/mariadb/gitops/_index.md create mode 100644 docs/guides/mariadb/gitops/gitops.md create mode 100644 docs/guides/mariadb/gitops/overview.md create mode 100644 docs/guides/mongodb/gitops/_index.md create mode 100644 docs/guides/mongodb/gitops/gitops.md create mode 100644 docs/guides/mongodb/gitops/overview.md create mode 100644 docs/guides/mssqlserver/gitops/_index.md create mode 100644 docs/guides/mssqlserver/gitops/gitops.md create mode 100644 docs/guides/mssqlserver/gitops/overview.md diff --git a/docs/guides/elasticsearch/gitops/_index.md b/docs/guides/elasticsearch/gitops/_index.md index a13bc5830..cc82b5f6b 100644 --- a/docs/guides/elasticsearch/gitops/_index.md +++ b/docs/guides/elasticsearch/gitops/_index.md @@ -5,6 +5,6 @@ menu: identifier: es-gitops name: Elasticsearch Gitops parent: es-elasticsearch-guides - weight: 65 + weight: 70 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/elasticsearch/gitops/gitops.md b/docs/guides/elasticsearch/gitops/gitops.md index e5b5b2d96..e3a2e57d2 100644 --- a/docs/guides/elasticsearch/gitops/gitops.md +++ b/docs/guides/elasticsearch/gitops/gitops.md @@ -1,29 +1,34 @@ --- -title: Kafka GitOps Guides +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview menu: docs_{{ .version }}: - identifier: kf-gitops-guides - name: Gitops Kafka - parent: kf-gitops-kafka - weight: 20 + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides --- + > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Kafka using KubeDB GitOps Operator +# GitOps Elasticsearch using KubeDB GitOps Operator -This guide will show you how to use `KubeDB` GitOps operator to create Kafka database and manage updates using GitOps workflow. +This guide will show you how to use `KubeDB` GitOps operator to create Elasticsearch database and manage updates using GitOps woreslow. ## Before You Begin -- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, +you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). -- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation +process to enable `GitOps` operator. -- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes +cluster with the desired state defined in Git. ```bash $ kubectl create ns monitoring @@ -32,7 +37,7 @@ This guide will show you how to use `KubeDB` GitOps operator to create Kafka dat $ kubectl create ns demo namespace/demo created ``` -> Note: YAML files used in this tutorial are stored in [docs/examples/kafka](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/kafka) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). +> Note: YAML files used in this tutorial are stored in [docs/examples/elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). @@ -54,37 +59,27 @@ argocd app create kubedb --repo --path kubedb --dest-server https://k argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa ``` -## Create Kafka Database using GitOps +## Create Elasticsearch Database using GitOps -### Create a Kafka GitOps CR +### Create a Elasticsearch GitOps CR ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: Standard - controller: - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: Standard + version: xpack-8.2.3 + enableSSL: true + replicas: 2 storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi deletionPolicy: WipeOut ``` @@ -92,492 +87,383 @@ Create a directory like below, ```bash $ tree . ├── kubedb - └── Kafka.yaml + └── Elasticsearch.yaml 1 directories, 1 files ``` -Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is created in your cluster. +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is created in your cluster. -Our `gitops` operator will create an actual `Kafka` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. +Our `gitops` operator will create an actual `Elasticsearch` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kafka.gitops.kubedb.com,kafka.kubedb.com -n demo -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 2m56s +$ kubectl get Elasticsearch.gitops.kubedb.com,Elasticsearch.kubedb.com -n demo +NAME AGE +elasticsearch.gitops.kubedb.com/es-gitops 20m -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 2m56s +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-8.2.3 Ready 20m ``` -List the resources created by `kubedb` operator created for `kubedb.com/v1` Kafka. +List the resources created by `kubedb` operator created for `kubedb.com/v1` Elasticsearch. ```bash -$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=kafka-prod' -NAME AGE -petset.apps.k8s.appscode.com/kafka-prod-broker 4m49s -petset.apps.k8s.appscode.com/kafka-prod-controller 4m47s +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=es-gitops' +NAME AGE +petset.apps.k8s.appscode.com/es-gitops 20m + +NAME READY STATUS RESTARTS AGE +pod/es-gitops-0 1/1 Running 0 20m +pod/es-gitops-1 1/1 Running 0 19m + +NAME TYPE DATA AGE +secret/es-gitops-a1d7d4 Opaque 1 13m +secret/es-gitops-apm-system-cred kubernetes.io/basic-auth 2 20m +secret/es-gitops-auth kubernetes.io/basic-auth 2 20m +secret/es-gitops-beats-system-cred kubernetes.io/basic-auth 2 20m +secret/es-gitops-ca-cert kubernetes.io/tls 2 20m +secret/es-gitops-client-cert kubernetes.io/tls 3 20m +secret/es-gitops-config Opaque 1 20m +secret/es-gitops-http-cert kubernetes.io/tls 3 20m +secret/es-gitops-kibana-system-cred kubernetes.io/basic-auth 2 20m +secret/es-gitops-logstash-system-cred kubernetes.io/basic-auth 2 20m +secret/es-gitops-remote-monitoring-user-cred kubernetes.io/basic-auth 2 20m +secret/es-gitops-transport-cert kubernetes.io/tls 3 20m + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/es-gitops ClusterIP 10.43.235.79 9200/TCP 20m +service/es-gitops-master ClusterIP None 9300/TCP 20m +service/es-gitops-pods ClusterIP None 9200/TCP 20m + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/es-gitops kubedb.com/elasticsearch 8.2.3 20m +``` + +## Update Elasticsearch Database using GitOps +### Scale Elasticsearch Replicas +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: es-gitops + namespace: demo +spec: + version: xpack-8.2.3 + enableSSL: true + replicas: 3 + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` -NAME READY STATUS RESTARTS AGE -pod/kafka-prod-broker-0 1/1 Running 0 4m48s -pod/kafka-prod-broker-1 1/1 Running 0 4m42s -pod/kafka-prod-controller-0 1/1 Running 0 4m47s -pod/kafka-prod-controller-1 1/1 Running 0 4m40s +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Elasticsearch` database replicas. List the +resources created by `gitops` operator in the `demo` namespace. -NAME TYPE DATA AGE -secret/kafka-prod-auth kubernetes.io/basic-auth 2 4m51s +```bash +$ kubectl get es,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-8.2.3 Ready 64m -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/kafka-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-32p116 HorizontalScaling Successful 39m +``` -NAME TYPE VERSION AGE -appbinding.appcatalog.appscode.com/kafka-prod kubedb.com/kafka 3.9.0 4m47s +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=es-gitops' +NAME READY STATUS RESTARTS AGE +es-gitops-0 1/1 Running 0 36m +es-gitops-1 1/1 Running 0 16m +es-gitops-2 1/1 Running 0 15m ``` -## Update Kafka Database using GitOps -### Scale Kafka Database Resources +We can also scale down the replicas by updating the `replicas` fields. +### Scale Elasticsearch Database Resources -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path + version: xpack-8.2.3 + enableSSL: true + replicas: 3 + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi deletionPolicy: WipeOut ``` -Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Resource Requests and Limits are updated to `500m` CPU and `1Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the resource changes and create a `KafkaOpsRequest` to update the `Kafka` database. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Elasticsearch` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h +$ kubectl get es,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-8.2.3 Ready 64m -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 22h +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-injx1l HorizontalScaling Successful 15m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-x5mfy0 VerticalScaling Successful 39m -NAME TYPE STATUS AGE -Kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, ```bash -$ kubectl get pod -n demo Kafka-prod-broker-0 -o json | jq '.spec.containers[0].resources' +$ kubectl get pod -n demo es-gitops-0 -o json | jq '.spec.containers[0].resources' { "limits": { - "memory": "1536Mi" + "cpu": "500m", + "memory": "1Gi" }, "requests": { - "cpu": "500m", - "memory": "1536Mi" + "cpu": "100m", + "memory": "1Gi" } } -``` -### Scale Kafka Replicas -Update the `Kafka.yaml` with the following, -```yaml -apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka -metadata: - name: kafka-prod - namespace: demo -spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 3 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 3 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path - storageType: Durable - deletionPolicy: WipeOut ``` -Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. -Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` KafkaOpsRequest to update the `Kafka` database replicas. List the resources created by `gitops` operator in the `demo` namespace. - -```bash -$ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 22h -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 22h +### Expand Elasticsearch Volume -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -``` - -After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, -```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=kafka-prod' -NAME READY STATUS RESTARTS AGE -kafka-prod-broker-0 1/1 Running 0 34m -kafka-prod-broker-1 1/1 Running 0 33m -kafka-prod-broker-2 1/1 Running 0 33m -kafka-prod-controller-0 1/1 Running 0 32m -kafka-prod-controller-1 1/1 Running 0 31m -kafka-prod-controller-2 1/1 Running 0 31m -``` - -We can also scale down the replicas by updating the `replicas` fields. - -### Expand Kafka Volume - -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard + version: xpack-8.2.3 + enableSSL: true + replicas: 3 + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi deletionPolicy: WipeOut ``` -Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` KafkaOpsRequest to update the `Kafka` database volume. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Elasticsearch` database volume. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 23m +$ kubectl get es,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-8.2.3 Ready 3h1m -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 23m +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-32p116 HorizontalScaling Successful 157m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-x5mfy0 VerticalScaling Successful 157m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-sata37 VolumeExpansion Successful 38m -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -kafkaopsrequest.ops.kubedb.com/kafka-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m ``` After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, ```bash -$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=kafka-prod' -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE -kafka-prod-data-kafka-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m -kafka-prod-data-kafka-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m -kafka-prod-data-kafka-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m -kafka-prod-data-kafka-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m - +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=es-gitops' +NAME READY STATUS RESTARTS AGE +es-gitops-0 1/1 Running 0 36m +es-gitops-1 1/1 Running 0 16m +es-gitops-2 1/1 Running 0 15m ``` -## Reconfigure Kafka +## Reconfigure Elasticsearch At first, we will create a secret containing `user.conf` file with required configuration settings. -To know more about this configuration file, check [here](/docs/guides/kafka/configuration/kafka-combined.md) +To know more about this configuration file, check [here](/docs/guides/elasticsearch/configuration/combined-cluster/index.md) ```yaml apiVersion: v1 +stringData: + user.conf: | + max_connections=200 + shared_buffers=256MB kind: Secret metadata: - name: new-kf-combined-custom-config + name: es-configuration namespace: demo -stringData: - server.properties: |- - log.retention.hours=125 +type: Opaque ``` -Now, we will add this file to `kubedb /kf-configuration.yaml`. +Now, we will add this file to `kubedb /es-configuration.yaml`. ```bash $ tree . ├── kubedb -│ ├── kf-configuration.yaml -│ └── Kafka.yaml +│ ├── es-configuration.yaml +│ └── Elasticsearch.yaml 1 directories, 2 files ``` -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - configSecret: - name: new-kf-combined-custom-config - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard + version: xpack-8.2.3 + enableSSL: true + replicas: 3 + configuration: + secretName: es-configuration + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi deletionPolicy: WipeOut ``` -Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` KafkaOpsRequest to update the `Kafka` database configuration. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Elasticsearch` database configuration. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 74m - -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 74m - -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 24m -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m +$ kubectl get es,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-8.2.3 Ready 3h53m +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-32p116 HorizontalScaling Successful 3h29m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfigure-wj5qyx Reconfigure Successful 3m42s +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-lvh38k VerticalScaling Successful 99m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-sata37 VolumeExpansion Successful 90m ``` -> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. - -### Rotate Kafka Auth +### Rotate Elasticsearch Auth To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. -We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, +We will generate a secret named `es-rotate-auth` with the following content, ```bash -kubectl create secret generic kf-rotate-auth -n demo \ ---type=kubernetes.io/basic-auth \ ---from-literal=username=kafka \ ---from-literal=password=kafka-secret -secret/kf-rotate-auth created +$ kubectl create secret generic es-rotate-auth -n demo \ + --type=kubernetes.io/basic-auth \ + --from-literal=username=elastic \ + --from-literal=password=elasticsearch-secret +secret/es-rotate-auth created ``` -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: + version: xpack-8.2.3 + enableSSL: true + replicas: 3 authSecret: kind: Secret - name: kf-rotate-auth - configSecret: - name: new-kf-combined-custom-config - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard + name: es-rotate-auth + configuration: + secretName: es-configuration + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi deletionPolicy: WipeOut ``` -Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Change the `authSecret` field to `es-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the auth changes and create a `RotateAuth` KafkaOpsRequest to update the `Kafka` database auth. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Elasticsearch` database auth. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 7m11s - -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 7m11s - -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 17h -kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 28m -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h +$ kubectl get es,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-9.1.9 Ready 18m +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-wiezxp HorizontalScaling Successful 13m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfigure-i9puam Reconfigure Successful 13m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-rotate-auth-xisot8 RotateAuth Successful 4m38s +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-7dyz6e VerticalScaling Successful 13m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-pptef3 VolumeExpansion Successful 13m ``` @@ -585,7 +471,7 @@ kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansi We can add, rotate or remove TLS configuration using `gitops`. -To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Kafka. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. - Start off by generating a ca certificates using openssl. @@ -601,42 +487,42 @@ writing new private key to './ca.key' - Now we are going to create a ca-secret using the certificate files that we have just generated. ```bash -$ kubectl create secret tls kafka-ca \ +$ kubectl create secret tls Elasticsearch-ca \ --cert=ca.crt \ --key=ca.key \ --namespace=demo -secret/Kafka-ca created +secret/Elasticsearch-ca created ``` -Now, Let's create an `Issuer` using the `Kafka-ca` secret that we have just created. The `YAML` file looks like this: +Now, Let's create an `Issuer` using the `Elasticsearch-ca` secret that we have just created. The `YAML` file looks like this: ```yaml apiVersion: cert-manager.io/v1 kind: Issuer metadata: - name: kf-issuer + name: es-issuer namespace: demo spec: ca: - secretName: kafka-ca + secretName: Elasticsearch-ca ``` -Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +Let's add that to our `kubedb /es-issuer.yaml` file. File structure will look like this, ```bash $ tree . ├── kubedb -│ ├── kf-configuration.yaml -│ ├── kf-issuer.yaml -│ └── Kafka.yaml +│ ├── es-configuration.yaml +│ ├── es-issuer.yaml +│ └── Elasticsearch.yaml 1 directories, 4 files ``` -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: version: 3.9.0 @@ -645,7 +531,7 @@ spec: podTemplate: spec: containers: - - name: kafka + - name: Elasticsearch resources: limits: memory: 1536Mi @@ -664,7 +550,7 @@ spec: podTemplate: spec: containers: - - name: kafka + - name: Elasticsearch resources: limits: memory: 1536Mi @@ -683,204 +569,175 @@ spec: deletionPolicy: WipeOut ``` -Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` KafkaOpsRequest to update the `Kafka` database tls. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Elasticsearch` database tls. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,kafka,kfops,pods -n demo +$ kubectl get es,Elasticsearch,esops,pods -n demo NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 3.9.0 Ready 41m +Elasticsearch.kubedb.com/es-gitops kubedb.com/v1 3.9.0 Ready 41m NAME AGE -kafka.gitops.kubedb.com/kafka-prod 75m +Elasticsearch.gitops.kubedb.com/es-gitops 75m NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d18h -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s -kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h +Elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfigure-ukj41o Reconfigure Successful 5d18h +Elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Elasticsearchopsrequest.ops.kubedb.com/es-gitops-rotate-auth-43ris8 RotateAuth Successful 5d1h +Elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-41xthr VolumeExpansion Successful 5d19h ``` -> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Kafka. +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Elasticsearch. ### Update Version -List Kafka versions using `kubectl get Kafkaversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/kafka/update-version/update-version.md). +List Elasticsearch versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/elasticsearch/update-version/update-version.md). Let's choose `4.0.0` in this example. -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - version: 4.0.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard + version: xpack-9.2.3 + replicas: 3 + authSecret: + kind: Secret + name: es-rotate-auth + configuration: + secretName: es-configuration + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi deletionPolicy: WipeOut ``` -Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the version changes and create a `VersionUpdate` KafkaOpsRequest to update the `Kafka` database version. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Elasticsearch` database version. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,kafka,kfops -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 3h47m +$ kubectl get es,Elasticsearch,esops -n demo +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-9.2.3 Ready 64m -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 3h47m +NAME AGE +elasticsearch.gitops.kubedb.com/es-gitops 64m + +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-os8fon HorizontalScaling Successful 58m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfigure-hrbrs1 Reconfigure Successful 58m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-rotate-auth-s0tkka RotateAuth Successful 58m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-versionupdate-bn3rk4 UpdateVersion Successful 31m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-vhzkxl VerticalScaling Successful 58m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-bzqd4r VolumeExpansion Successful 58m -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 5d22h -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m -kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h -kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h ``` -Now, we are going to verify whether the `Kafka`, `PetSet` and it's `Pod` have updated with new image. Let's check, +Now, we are going to verify whether the `Elasticsearch`, `PetSet` and it's `Pod` have updated with new image. Let's check, ```bash -$ kubectl get Kafka -n demo kafka-prod -o=jsonpath='{.spec.version}{"\n"}' -4.0.0 +$ kubectl get Elasticsearch -n demo es-gitops -o=jsonpath='{.spec.version}{"\n"}' +xpack-9.2.3 -$ kubectl get petset -n demo kafka-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get petset -n demo es-gitops -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/elastic:9.2.3@sha256:714f35c53d333fef7e673079d05ce80440caf2c20ca3dc9b3e366728527760bb -$ kubectl get pod -n demo kafka-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/kafka:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get pod -n demo es-gitops-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/elastic:9.2.3@sha256:714f35c53d333fef7e673079d05ce80440caf2c20ca3dc9b3e366728527760bb ``` ### Enable Monitoring If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). -Update the `Kafka.yaml` with the following, +Update the `Elasticsearch.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Kafka +kind: Elasticsearch metadata: - name: kafka-prod + name: es-gitops namespace: demo spec: - version: 4.0.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: kafka - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard + version: xpack-9.2.3 + replicas: 3 + authSecret: + kind: Secret + name: es-rotate-auth + configuration: + secretName: es-configuration + podTemplate: + spec: + containers: + - name: elasticsearch + resources: + limits: + cpu: 500m + memory: 1Gi + requests: + cpu: 100m + memory: 1Gi storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + deletionPolicy: WipeOut monitor: agent: prometheus.io/operator prometheus: - exporter: - port: 9091 serviceMonitor: labels: release: prometheus interval: 10s - deletionPolicy: WipeOut ``` -Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Kafka` CR is updated in your cluster. +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the monitoring changes and create a `Restart` KafkaOpsRequest to add the `Kafka` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Elasticsearch` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get Kafkaes.gitops.kubedb.com,Kafkaes.kubedb.com,Kafkaopsrequest -n demo -NAME TYPE VERSION STATUS AGE -kafka.kubedb.com/kafka-prod kubedb.com/v1 4.0.0 Ready 5h12m +$ kubectl get Elasticsearches.gitops.kubedb.com,Elasticsearches.kubedb.com,Elasticsearchopsrequest -n demo +NAME AGE +elasticsearch.gitops.kubedb.com/es-gitops 106m -NAME AGE -kafka.gitops.kubedb.com/kafka-prod 5h12m +NAME VERSION STATUS AGE +elasticsearch.kubedb.com/es-gitops xpack-9.2.3 Ready 106m -NAME TYPE STATUS AGE -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfigure-ukj41o Reconfigure Successful 6d -kafkaopsrequest.ops.kubedb.com/kafka-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m -kafkaopsrequest.ops.kubedb.com/kafka-prod-restart-ljpqih Restart Successful 3m51s -kafkaopsrequest.ops.kubedb.com/kafka-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h -kafkaopsrequest.ops.kubedb.com/kafka-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m -kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d +NAME TYPE STATUS AGE +elasticsearchopsrequest.ops.kubedb.com/es-gitops-horizontalscaling-os8fon HorizontalScaling Successful 101m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-reconfigure-hrbrs1 Reconfigure Successful 101m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-restart-7v0e29 Restart Successful 15m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-rotate-auth-s0tkka RotateAuth Successful 101m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-versionupdate-bn3rk4 UpdateVersion Successful 73m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-verticalscaling-vhzkxl VerticalScaling Successful 101m +elasticsearchopsrequest.ops.kubedb.com/es-gitops-volumeexpansion-bzqd4r VolumeExpansion Successful 101m ``` @@ -899,10 +756,10 @@ There are some other fields that will trigger `Restart` ops request. ## Next Steps -[//]: # (- Learn Kafka [GitOps](/docs/guides/kafka/concepts/Kafka-gitops.md)) -- Learn Kafka Scaling - - [Horizontal Scaling](/docs/guides/kafka/scaling/horizontal-scaling/combined.md) - - [Vertical Scaling](/docs/guides/kafka/scaling/vertical-scaling/combined.md) -- Learn Version Update Ops Request and Constraints [here](/docs/guides/kafka/update-version/overview.md) -- Monitor your KafkaQL database with KubeDB using [built-in Prometheus](/docs/guides/kafka/monitoring/using-builtin-prometheus.md). -- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). +[//]: # (- Learn Elasticsearch [GitOps](/docs/guides/elasticsearch/concepts/Elasticsearch-gitops.md)) +- Learn Elasticsearch Scaling + - [Horizontal Scaling](/docs/guides/elasticsearch/scaling/horizontal/combined.md) + - [Vertical Scaling](/docs/guides/elasticsearch/scaling/vertical/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/elasticsearch/update-version/overview.md) +- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/elasticsearch/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md) \ No newline at end of file diff --git a/docs/guides/elasticsearch/gitops/overview.md b/docs/guides/elasticsearch/gitops/overview.md index cf959a479..bfa01c2ff 100644 --- a/docs/guides/elasticsearch/gitops/overview.md +++ b/docs/guides/elasticsearch/gitops/overview.md @@ -1,11 +1,11 @@ --- -title: Kafka Gitops Overview -description: Kafka Gitops Overview +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview menu: docs_{{ .version }}: - identifier: kf-gitops-overview + identifier: es-gitops-overview name: Overview - parent: kf-gitops + parent: es-gitops weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides @@ -13,20 +13,20 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Overview for Kafka +# GitOps Overview for Elasticsearch -This guide will give you an overview of how KubeDB `gitops` operator works with Kafka databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for -managing Kafka databases in Kubernetes. +This guide will give you an overview of how KubeDB `gitops` operator works with Elasticsearch databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Elasticsearch databases in Kubernetes. ## Before You Begin - You should be familiar with the following `KubeDB` concepts: - - [Kafka](/docs/guides/kafka/concepts/kafka.md) - - [KafkaOpsRequest](/docs/guides/kafka/concepts/kafkaopsrequest.md) + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) -## Workflow GitOps with Kafka +## Workflow GitOps with Elasticsearch The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. @@ -35,16 +35,16 @@ The following diagram shows how the `KubeDB` GitOps Operator used to sync with y GitOps Flow -
Fig: GitOps process of Kafka
+
Fig: GitOps process of Elasticsearch
-1. **Define GitOps Kafka**: Create Custom Resource (CR) of kind `Kafka` using the `gitops.kubedb.com/v1alpha1` API. +1. **Define GitOps Elasticsearch**: Create Custom Resource (CR) of kind `Elasticsearch` using the `gitops.kubedb.com/v1alpha1` API. 2. **Store in Git**: Push the CR to a Git repository. 3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. -4. **Create Database**: The GitOps operator creates a corresponding KubeDB Kafka CR in the Kubernetes cluster to deploy the database. -5. **Handle Updates**: When you update the KafkaGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Elasticsearch CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the ElasticsearchGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. -This flow makes managing Kafka databases efficient, reliable, and fully integrated with GitOps practices. +This flow makes managing Elasticsearch databases efficient, reliable, and fully integrated with GitOps practices. -In the next doc, we are going to show a step by step guide on running Kafka using GitOps. +In the next doc, we are going to show a step by step guide on running Elasticsearch using GitOps. diff --git a/docs/guides/kafka/gitops/gitops.md b/docs/guides/kafka/gitops/gitops.md index e5b5b2d96..881efb1dd 100644 --- a/docs/guides/kafka/gitops/gitops.md +++ b/docs/guides/kafka/gitops/gitops.md @@ -491,7 +491,7 @@ kafkaopsrequest.ops.kubedb.com/kafka-prod-volumeexpansion-41xthr VolumeExpansi To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. -We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, +We will create a secret named `kf-rotate-auth ` with the following content, ```bash kubectl create secret generic kf-rotate-auth -n demo \ @@ -515,8 +515,8 @@ spec: authSecret: kind: Secret name: kf-rotate-auth - configSecret: - name: new-kf-combined-custom-config + configuration: + secretName: new-kf-combined-custom-config version: 3.9.0 topology: broker: diff --git a/docs/guides/mariadb/gitops/_index.md b/docs/guides/mariadb/gitops/_index.md new file mode 100644 index 000000000..cc82b5f6b --- /dev/null +++ b/docs/guides/mariadb/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Elasticsearch Gitops +menu: + docs_{{ .version }}: + identifier: es-gitops + name: Elasticsearch Gitops + parent: es-elasticsearch-guides + weight: 70 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/mariadb/gitops/gitops.md b/docs/guides/mariadb/gitops/gitops.md new file mode 100644 index 000000000..ec2fdd8f5 --- /dev/null +++ b/docs/guides/mariadb/gitops/gitops.md @@ -0,0 +1,910 @@ +--- +title: MariaDB Gitops Overview +description: MariaDB Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + + + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps MariaDB using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create MariaDB database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/MariaDB](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/MariaDB) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create MariaDB Database using GitOps + +### Create a MariaDB GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: mariadb-gitops + namespace: demo +spec: + version: "10.5.23" + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── MariaDB.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is created in your cluster. + +Our `gitops` operator will create an actual `MariaDB` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get MariaDB.gitops.kubedb.com,MariaDB.kubedb.com -n demo +NAME AGE +mariadb.gitops.kubedb.com/mariadb-gitops 2m + +NAME VERSION STATUS AGE +mariadb.kubedb.com/mariadb-gitops 10.5.23 Ready 2m + +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` MariaDB. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=mariadb-gitops' +NAME AGE +petset.apps.k8s.appscode.com/mariadb-gitops 2m45s + +NAME READY STATUS RESTARTS AGE +pod/mariadb-gitops-0 1/1 Running 0 2m45s + +NAME TYPE DATA AGE +secret/mariadb-gitops-auth kubernetes.io/basic-auth 2 2m47s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/mariadb-gitops ClusterIP 10.43.132.16 3306/TCP 2m47s +service/mariadb-gitops-pods ClusterIP None 3306/TCP 2m47s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/mariadb-gitops kubedb.com/mariadb 10.5.23 2m45s + +``` + +## Update MariaDB Database using GitOps + +### Scale MariaDB Database Resources + +```shell +$ kubectl get pod -n demo mariadb-gitops-0 -o json | jq '.spec.containers[].resources' +{ + "limits": { + "memory": "1Gi" + }, + "requests": { + "cpu": "500m", + "memory": "1Gi" + } +} + +``` +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut + ``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `MariaDBOpsRequest` to update the `MariaDB` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ $ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 22h + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo MariaDB-prod-broker-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1536Mi" + } +} +``` + +### Scale MariaDB Replicas +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` MariaDBOpsRequest to update the `MariaDB` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 22h + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=MariaDB-prod' +NAME READY STATUS RESTARTS AGE +MariaDB-prod-broker-0 1/1 Running 0 34m +MariaDB-prod-broker-1 1/1 Running 0 33m +MariaDB-prod-broker-2 1/1 Running 0 33m +MariaDB-prod-controller-0 1/1 Running 0 32m +MariaDB-prod-controller-1 1/1 Running 0 31m +MariaDB-prod-controller-2 1/1 Running 0 31m +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Expand MariaDB Volume + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` MariaDBOpsRequest to update the `MariaDB` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 23m + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=MariaDB-prod' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +MariaDB-prod-data-MariaDB-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +MariaDB-prod-data-MariaDB-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +MariaDB-prod-data-MariaDB-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +MariaDB-prod-data-MariaDB-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure MariaDB + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/MariaDB/configuration/MariaDB-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-kf-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /kf-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ └── MariaDB.yaml +1 directories, 2 files +``` + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` MariaDBOpsRequest to update the `MariaDB` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 74m + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfigure-ukj41o Reconfigure Successful 24m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate MariaDB Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, + +```bash +kubectl create secret generic kf-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=MariaDB \ +--from-literal=password=MariaDB-secret +secret/kf-rotate-auth created + +``` + + + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + authSecret: + kind: Secret + name: kf-rotate-auth + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` MariaDBOpsRequest to update the `MariaDB` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 7m11s + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfigure-ukj41o Reconfigure Successful 17h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-rotate-auth-43ris8 RotateAuth Successful 28m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in MariaDB. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls MariaDB-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/MariaDB-ca created +``` + +Now, Let's create an `Issuer` using the `MariaDB-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: kf-issuer + namespace: demo +spec: + ca: + secretName: MariaDB-ca +``` + +Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ ├── kf-issuer.yaml +│ └── MariaDB.yaml +1 directories, 4 files +``` + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` MariaDBOpsRequest to update the `MariaDB` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops,pods -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 75m + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for MariaDB. + +### Update Version + +List MariaDB versions using `kubectl get MariaDBversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/MariaDB/update-version/update-version.md). + +Let's choose `4.0.0` in this example. + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` MariaDBOpsRequest to update the `MariaDB` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,MariaDB,kfops -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 3h47m + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `MariaDB`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get MariaDB -n demo MariaDB-prod -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo MariaDB-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MariaDB:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo MariaDB-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MariaDB:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `MariaDB.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MariaDB +metadata: + name: MariaDB-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: MariaDB + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MariaDB` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` MariaDBOpsRequest to add the `MariaDB` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get MariaDBes.gitops.kubedb.com,MariaDBes.kubedb.com,MariaDBopsrequest -n demo +NAME TYPE VERSION STATUS AGE +MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +MariaDB.gitops.kubedb.com/MariaDB-prod 5h12m + +NAME TYPE STATUS AGE +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfigure-ukj41o Reconfigure Successful 6d +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-restart-ljpqih Restart Successful 3m51s +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn MariaDB [GitOps](/docs/guides/MariaDB/concepts/MariaDB-gitops.md)) +- Learn MariaDB Scaling + - [Horizontal Scaling](/docs/guides/MariaDB/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/MariaDB/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/MariaDB/update-version/overview.md) +- Monitor your MariaDBQL database with KubeDB using [built-in Prometheus](/docs/guides/MariaDB/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mariadb/gitops/overview.md b/docs/guides/mariadb/gitops/overview.md new file mode 100644 index 000000000..80ab6a2fc --- /dev/null +++ b/docs/guides/mariadb/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: MariaDB Gitops Overview +description: MariaDB Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for MariaDB + +This guide will give you an overview of how KubeDB `gitops` operator works with MariaDB databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing MariaDB databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [MariaDB](/docs/guides/mariadb/concepts/mariadb/index.md) + - [MariaDBOpsRequest](/docs/guides/mariadb/concepts/opsrequest/index.md) + + + +## Workflow GitOps with MariaDB + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of MariaDB
+ +
+ +1. **Define GitOps MariaDB**: Create Custom Resource (CR) of kind `MariaDB` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB MariaDB CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the MariaDBGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing MariaDB databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running MariaDB using GitOps. diff --git a/docs/guides/mongodb/gitops/_index.md b/docs/guides/mongodb/gitops/_index.md new file mode 100644 index 000000000..cc82b5f6b --- /dev/null +++ b/docs/guides/mongodb/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Elasticsearch Gitops +menu: + docs_{{ .version }}: + identifier: es-gitops + name: Elasticsearch Gitops + parent: es-elasticsearch-guides + weight: 70 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/mongodb/gitops/gitops.md b/docs/guides/mongodb/gitops/gitops.md new file mode 100644 index 000000000..332c4f29d --- /dev/null +++ b/docs/guides/mongodb/gitops/gitops.md @@ -0,0 +1,900 @@ +--- +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + + + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Elasticsearch using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create Elasticsearch database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/Elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create Elasticsearch Database using GitOps + +### Create a Elasticsearch GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: es-gitops + namespace: demo +spec: + version: xpack-8.2.3 + enableSSL: true + replicas: 2 + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── Elasticsearch.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is created in your cluster. + +Our `gitops` operator will create an actual `Elasticsearch` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get Elasticsearch.gitops.kubedb.com,Elasticsearch.kubedb.com -n demo +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 2m56s + +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 2m56s +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` Elasticsearch. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME AGE +petset.apps.k8s.appscode.com/Elasticsearch-prod-broker 4m49s +petset.apps.k8s.appscode.com/Elasticsearch-prod-controller 4m47s + +NAME READY STATUS RESTARTS AGE +pod/Elasticsearch-prod-broker-0 1/1 Running 0 4m48s +pod/Elasticsearch-prod-broker-1 1/1 Running 0 4m42s +pod/Elasticsearch-prod-controller-0 1/1 Running 0 4m47s +pod/Elasticsearch-prod-controller-1 1/1 Running 0 4m40s + +NAME TYPE DATA AGE +secret/Elasticsearch-prod-auth kubernetes.io/basic-auth 2 4m51s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/Elasticsearch-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/Elasticsearch-prod kubedb.com/Elasticsearch 3.9.0 4m47s +``` + +## Update Elasticsearch Database using GitOps + +### Scale Elasticsearch Database Resources + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut + ``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Elasticsearch` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ $ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1536Mi" + } +} +``` + +### Scale Elasticsearch Replicas +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Elasticsearch` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME READY STATUS RESTARTS AGE +Elasticsearch-prod-broker-0 1/1 Running 0 34m +Elasticsearch-prod-broker-1 1/1 Running 0 33m +Elasticsearch-prod-broker-2 1/1 Running 0 33m +Elasticsearch-prod-controller-0 1/1 Running 0 32m +Elasticsearch-prod-controller-1 1/1 Running 0 31m +Elasticsearch-prod-controller-2 1/1 Running 0 31m +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Expand Elasticsearch Volume + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Elasticsearch` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 23m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +Elasticsearch-prod-data-Elasticsearch-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure Elasticsearch + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/Elasticsearch/configuration/Elasticsearch-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-kf-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /kf-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ └── Elasticsearch.yaml +1 directories, 2 files +``` + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Elasticsearch` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 74m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 24m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate Elasticsearch Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, + +```bash +kubectl create secret generic kf-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=Elasticsearch \ +--from-literal=password=Elasticsearch-secret +secret/kf-rotate-auth created + +``` + + + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + authSecret: + kind: Secret + name: kf-rotate-auth + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Elasticsearch` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 7m11s + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 28m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls Elasticsearch-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/Elasticsearch-ca created +``` + +Now, Let's create an `Issuer` using the `Elasticsearch-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: kf-issuer + namespace: demo +spec: + ca: + secretName: Elasticsearch-ca +``` + +Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ ├── kf-issuer.yaml +│ └── Elasticsearch.yaml +1 directories, 4 files +``` + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Elasticsearch` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops,pods -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 75m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Elasticsearch. + +### Update Version + +List Elasticsearch versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Elasticsearch/update-version/update-version.md). + +Let's choose `4.0.0` in this example. + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Elasticsearch` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 3h47m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `Elasticsearch`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get Elasticsearch -n demo Elasticsearch-prod -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo Elasticsearch-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Elasticsearch` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get Elasticsearches.gitops.kubedb.com,Elasticsearches.kubedb.com,Elasticsearchopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 5h12m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-restart-ljpqih Restart Successful 3m51s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn Elasticsearch [GitOps](/docs/guides/Elasticsearch/concepts/Elasticsearch-gitops.md)) +- Learn Elasticsearch Scaling + - [Horizontal Scaling](/docs/guides/Elasticsearch/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/Elasticsearch/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/Elasticsearch/update-version/overview.md) +- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/Elasticsearch/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/gitops/overview.md b/docs/guides/mongodb/gitops/overview.md new file mode 100644 index 000000000..bfa01c2ff --- /dev/null +++ b/docs/guides/mongodb/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for Elasticsearch + +This guide will give you an overview of how KubeDB `gitops` operator works with Elasticsearch databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Elasticsearch databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + + + +## Workflow GitOps with Elasticsearch + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of Elasticsearch
+ +
+ +1. **Define GitOps Elasticsearch**: Create Custom Resource (CR) of kind `Elasticsearch` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Elasticsearch CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the ElasticsearchGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing Elasticsearch databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running Elasticsearch using GitOps. diff --git a/docs/guides/mssqlserver/gitops/_index.md b/docs/guides/mssqlserver/gitops/_index.md new file mode 100644 index 000000000..cc82b5f6b --- /dev/null +++ b/docs/guides/mssqlserver/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Elasticsearch Gitops +menu: + docs_{{ .version }}: + identifier: es-gitops + name: Elasticsearch Gitops + parent: es-elasticsearch-guides + weight: 70 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/mssqlserver/gitops/gitops.md b/docs/guides/mssqlserver/gitops/gitops.md new file mode 100644 index 000000000..332c4f29d --- /dev/null +++ b/docs/guides/mssqlserver/gitops/gitops.md @@ -0,0 +1,900 @@ +--- +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + + + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Elasticsearch using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create Elasticsearch database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/Elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create Elasticsearch Database using GitOps + +### Create a Elasticsearch GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: es-gitops + namespace: demo +spec: + version: xpack-8.2.3 + enableSSL: true + replicas: 2 + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── Elasticsearch.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is created in your cluster. + +Our `gitops` operator will create an actual `Elasticsearch` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get Elasticsearch.gitops.kubedb.com,Elasticsearch.kubedb.com -n demo +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 2m56s + +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 2m56s +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` Elasticsearch. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME AGE +petset.apps.k8s.appscode.com/Elasticsearch-prod-broker 4m49s +petset.apps.k8s.appscode.com/Elasticsearch-prod-controller 4m47s + +NAME READY STATUS RESTARTS AGE +pod/Elasticsearch-prod-broker-0 1/1 Running 0 4m48s +pod/Elasticsearch-prod-broker-1 1/1 Running 0 4m42s +pod/Elasticsearch-prod-controller-0 1/1 Running 0 4m47s +pod/Elasticsearch-prod-controller-1 1/1 Running 0 4m40s + +NAME TYPE DATA AGE +secret/Elasticsearch-prod-auth kubernetes.io/basic-auth 2 4m51s + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/Elasticsearch-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/Elasticsearch-prod kubedb.com/Elasticsearch 3.9.0 4m47s +``` + +## Update Elasticsearch Database using GitOps + +### Scale Elasticsearch Database Resources + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut + ``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Elasticsearch` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ $ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1536Mi" + } +} +``` + +### Scale Elasticsearch Replicas +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1540Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 3 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: local-path + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Elasticsearch` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME READY STATUS RESTARTS AGE +Elasticsearch-prod-broker-0 1/1 Running 0 34m +Elasticsearch-prod-broker-1 1/1 Running 0 33m +Elasticsearch-prod-broker-2 1/1 Running 0 33m +Elasticsearch-prod-controller-0 1/1 Running 0 32m +Elasticsearch-prod-controller-1 1/1 Running 0 31m +Elasticsearch-prod-controller-2 1/1 Running 0 31m +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Expand Elasticsearch Volume + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Elasticsearch` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 23m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +Elasticsearch-prod-data-Elasticsearch-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +Elasticsearch-prod-data-Elasticsearch-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure Elasticsearch + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/Elasticsearch/configuration/Elasticsearch-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-kf-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /kf-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ └── Elasticsearch.yaml +1 directories, 2 files +``` + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Elasticsearch` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 74m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 24m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate Elasticsearch Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the following content, + +```bash +kubectl create secret generic kf-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=Elasticsearch \ +--from-literal=password=Elasticsearch-secret +secret/kf-rotate-auth created + +``` + + + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + authSecret: + kind: Secret + name: kf-rotate-auth + configSecret: + name: new-kf-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Elasticsearch` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 7m11s + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 28m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls Elasticsearch-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/Elasticsearch-ca created +``` + +Now, Let's create an `Issuer` using the `Elasticsearch-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: kf-issuer + namespace: demo +spec: + ca: + secretName: Elasticsearch-ca +``` + +Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── kf-configuration.yaml +│ ├── kf-issuer.yaml +│ └── Elasticsearch.yaml +1 directories, 4 files +``` + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Elasticsearch` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops,pods -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 75m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Elasticsearch. + +### Update Version + +List Elasticsearch versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Elasticsearch/update-version/update-version.md). + +Let's choose `4.0.0` in this example. + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Elasticsearch` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get kf,Elasticsearch,kfops -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 3h47m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `Elasticsearch`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get Elasticsearch -n demo Elasticsearch-prod -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo Elasticsearch-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `Elasticsearch.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Elasticsearch +metadata: + name: Elasticsearch-prod + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Elasticsearch + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Elasticsearch` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get Elasticsearches.gitops.kubedb.com,Elasticsearches.kubedb.com,Elasticsearchopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 5h12m + +NAME TYPE STATUS AGE +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-restart-ljpqih Restart Successful 3m51s +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn Elasticsearch [GitOps](/docs/guides/Elasticsearch/concepts/Elasticsearch-gitops.md)) +- Learn Elasticsearch Scaling + - [Horizontal Scaling](/docs/guides/Elasticsearch/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/Elasticsearch/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/Elasticsearch/update-version/overview.md) +- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/Elasticsearch/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mssqlserver/gitops/overview.md b/docs/guides/mssqlserver/gitops/overview.md new file mode 100644 index 000000000..bfa01c2ff --- /dev/null +++ b/docs/guides/mssqlserver/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: Elasticsearch Gitops Overview +description: Elasticsearch Gitops Overview +menu: + docs_{{ .version }}: + identifier: es-gitops-overview + name: Overview + parent: es-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for Elasticsearch + +This guide will give you an overview of how KubeDB `gitops` operator works with Elasticsearch databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Elasticsearch databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) + - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + + + +## Workflow GitOps with Elasticsearch + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of Elasticsearch
+ +
+ +1. **Define GitOps Elasticsearch**: Create Custom Resource (CR) of kind `Elasticsearch` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Elasticsearch CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the ElasticsearchGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing Elasticsearch databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running Elasticsearch using GitOps. From 3645011896475b3f2b7deed5a5e6956936923a8c Mon Sep 17 00:00:00 2001 From: Bonusree Date: Mon, 16 Feb 2026 17:02:41 +0600 Subject: [PATCH 08/14] gitops demo Signed-off-by: Bonusree --- .../using-config-file.md.bak | 177 +++ docs/guides/mysql/gitops/_index.md | 10 + docs/guides/mysql/gitops/gitops.md | 1031 +++++++++++++++++ docs/guides/mysql/gitops/overview.md | 43 + docs/guides/redis/gitops/_index.md | 10 + docs/guides/redis/gitops/gitops.md | 886 ++++++++++++++ docs/guides/redis/gitops/overview.md | 50 + .../configuration/config-file/index.md.bak | 248 ++++ 8 files changed, 2455 insertions(+) create mode 100644 docs/guides/ignite/custom-configuration/using-config-file.md.bak create mode 100755 docs/guides/mysql/gitops/_index.md create mode 100644 docs/guides/mysql/gitops/gitops.md create mode 100644 docs/guides/mysql/gitops/overview.md create mode 100644 docs/guides/redis/gitops/_index.md create mode 100644 docs/guides/redis/gitops/gitops.md create mode 100644 docs/guides/redis/gitops/overview.md create mode 100644 docs/guides/singlestore/configuration/config-file/index.md.bak diff --git a/docs/guides/ignite/custom-configuration/using-config-file.md.bak b/docs/guides/ignite/custom-configuration/using-config-file.md.bak new file mode 100644 index 000000000..68140892b --- /dev/null +++ b/docs/guides/ignite/custom-configuration/using-config-file.md.bak @@ -0,0 +1,177 @@ +--- +title: Run Ignite with Custom Configuration +menu: + docs_{{ .version }}: + identifier: ig-using-config-file-configuration + name: Customize Configurations + parent: ig-custom-configuration + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Using Custom Configuration File + +KubeDB supports providing custom configuration for Ignite. This tutorial will show you how to use KubeDB to run Ignite with custom configuration. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + + $ kubectl get ns demo + NAME STATUS AGE + demo Active 5s + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/ignite](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/ignite) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +Ignite does not allow to configuration via any file. However, configuration parameters can be set as arguments while starting the ignite docker image. To keep similarity with other KubeDB supported databases which support configuration through a config file, KubeDB has added an additional executable script on top of the official ignite docker image. This script parses the configuration file then set them as arguments of ignite binary. + +To know more about configuring Ignite server see [here](https://ignite.apache.org/docs/ignite3/latest/administrators-guide/config/node-config). + +At first, you have to create a custom configuration file and provide its name in `spec.configuration.secretName`. The operator reads this Secret internally and applies the configuration automatically. + + +In this tutorial, we will enable Ignite's authentication via secret. + +Create a secret with custom configuration file: +```yaml +apiVersion: v1 +stringData: + node-configuration.xml: | + + + + + + + + +kind: Secret +metadata: + name: ignite-configuration + namespace: demo + resourceVersion: "4505" +``` +Here, `authenticationEnabled's` default value is `false`. In this secret, we make the value `true`. + +```bash + $ kubectl apply -f ignite-configuration.yaml +secret/ignite-configuration created +``` + +Let's get the ignite-configuration `secret` with custom configuration: + +```yaml +$ kubectl get secret -n demo ignite-configuration -o yaml +apiVersion: v1 +data: + node-configuration.xml: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiPz4KPGJlYW5zIHhtbG5zPSJodHRwOi8vd3d3LnNwcmluZ2ZyYW1ld29yay5vcmcvc2NoZW1hL2JlYW5zIgogICB4bWxuczp4c2k9Imh0dHA6Ly93d3cudzMub3JnLzIwMDEvWE1MU2NoZW1hLWluc3RhbmNlIgogICB4c2k6c2NoZW1hTG9jYXRpb249Imh0dHA6Ly93d3cuc3ByaW5nZnJhbWV3b3JrLm9yZy9zY2hlbWEvYmVhbnMKICAgICAgICAgICAgICAgICAgICAgICBodHRwOi8vd3d3LnNwcmluZ2ZyYW1ld29yay5vcmcvc2NoZW1hL2JlYW5zL3NwcmluZy1iZWFucy0zLjAueHNkIj4KPCEtLSBZb3VyIElnbml0ZSBDb25maWd1cmF0aW9uIC0tPgo8YmVhbiBjbGFzcz0ib3JnLmFwYWNoZS5pZ25pdGUuY29uZmlndXJhdGlvbi5JZ25pdGVDb25maWd1cmF0aW9uIj4KCiAgICA8cHJvcGVydHkgbmFtZT0iYXV0aGVudGljYXRpb25FbmFibGVkIiB2YWx1ZT0idHJ1ZSIvPgoKPC9iZWFuPgo8L2JlYW5zPgo= +kind: Secret +metadata: + annotations: + kubectl.kubernetes.io/last-applied-configuration: | + {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"ignite-configuration","namespace":"demo","resourceVersion":"4505"},"stringData":{"node-configuration.xml":"\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cbeans xmlns=\"http://www.springframework.org/schema/beans\"\n xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"\n xsi:schemaLocation=\"http://www.springframework.org/schema/beans\n http://www.springframework.org/schema/beans/spring-beans-3.0.xsd\"\u003e\n\u003c!-- Your Ignite Configuration --\u003e\n\u003cbean class=\"org.apache.ignite.configuration.IgniteConfiguration\"\u003e\n\n \u003cproperty name=\"authenticationEnabled\" value=\"true\"/\u003e\n\n\u003c/bean\u003e\n\u003c/beans\u003e\n"}} + creationTimestamp: "2025-06-02T09:37:05Z" + name: ignite-configuration + namespace: demo + resourceVersion: "1391127" + uid: 57f2a44c-d6b1-4571-bb91-fd68b3048306 +type: Opaque +``` + +Now, create Ignite crd specifying `spec.configuration.secretName` field. + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/ignite/configuration/custom-ignite.yaml +ignite.kubedb.com/custom-ignite created +``` + +Below is the YAML for the Ignite crd we just created. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Ignite +metadata: + name: custom-ignite + namespace: demo +spec: + replicas: 3 + version: 2.17.0 + configuration: + secretName: ignite-configuration + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Now, wait a few minutes. KubeDB operator will create necessary petset, services etc. If everything goes well, we will see that a pod with the name `custom-ignite-0` has been created. + +Check if the database is ready + +```bash +$ kubectl get ig -n demo +NAME VERSION STATUS AGE +custom-ignite 2.17.0 Ready 17m +``` + +Now, we will check if the database has started with the custom configuration we have provided. +We will connect to `custom-ignite-0` pod: + +```bash +$ kubectl exec -it -n demo ignite-quickstart-0 -c ignite -- bash +[ignite@ignite-quickstart-0 config]$ cat /ignite/config/node-configuration.xml + + + + +... +... + + + +``` + +Here, we can see `authenticationEnabled's` value is `true`. + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl patch -n demo ig/custom-ignite -p '{"spec":{"deletionPolicy":"WipeOut"}}' --type="merge" +kubectl delete -n demo ig/custom-ignite + +kubectl delete -n demo secret ignite-configuration +kubectl delete ns demo +``` + +If you would like to uninstall KubeDB operator, please follow the steps [here](/docs/setup/README.md). + +## Next Steps + +- Learn how to use KubeDB to run a Ignite server [here](/docs/guides/ignite/README.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mysql/gitops/_index.md b/docs/guides/mysql/gitops/_index.md new file mode 100755 index 000000000..80ccea816 --- /dev/null +++ b/docs/guides/mysql/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: GitOps PostgreSQL +menu: + docs_{{ .version }}: + identifier: pg-gitops-postgres + name: GitOps + parent: pg-postgres-guides + weight: 40 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/mysql/gitops/gitops.md b/docs/guides/mysql/gitops/gitops.md new file mode 100644 index 000000000..2f56baef2 --- /dev/null +++ b/docs/guides/mysql/gitops/gitops.md @@ -0,0 +1,1031 @@ +--- +title: GitOps MySQL +menu: + docs_{{ .version }}: + identifier: my-using-gitops + name: GitOps MySQL + parent: my-gitops-MySQL + weight: 15 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps MySQL using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create MySQL database and manage updates using GitOps workflow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/MySQL](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/MySQL) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create MySQL Database using GitOps + +### Create a MySQL GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + replicas: 3 + version: "16.6" + storageType: Durable + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 1Gi + requests: + cpu: 500m + memory: 1Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── MySQL.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is created in your cluster. + +Our `gitops` operator will create an actual `MySQL` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com -n demo +NAME AGE +mysql.gitops.kubedb.com/my-gitops 172m + +NAME VERSION STATUS AGE +mysql.kubedb.com/my-gitops 9.1.0 Ready 172m +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` MySQL. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=my-gitops' +NAME AGE +petset.apps.k8s.appscode.com/my-gitops 174m + +NAME READY STATUS RESTARTS AGE +pod/my-gitops-0 2/2 Running 0 133m +pod/my-gitops-1 2/2 Running 0 129m +pod/my-gitops-2 2/2 Running 0 132m + +NAME TYPE DATA AGE +secret/my-gitops-auth kubernetes.io/basic-auth 2 174m + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/my-gitops ClusterIP 10.43.39.209 3306/TCP 174m +service/my-gitops-pods ClusterIP None 3306/TCP 174m +service/my-gitops-standby ClusterIP 10.43.212.101 3306/TCP 174m + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/my-gitops kubedb.com/mysql 9.1.0 174m +``` + +## Update MySQL Database using GitOps + +### Scale MySQL Database Resources + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + replicas: 3 + version: "16.6" + storageType: Durable + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `MySQLOpsRequest` to update the `MySQL` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +mysql.gitops.kubedb.com/my-gitops 178m + +NAME VERSION STATUS AGE +mysql.kubedb.com/my-gitops 9.1.0 Ready 178m + +NAME TYPE STATUS AGE +mysqlopsrequest.ops.kubedb.com/my-gitops-verticalscaling-mw8s6j VerticalScaling Successful 144m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo my-gitops-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "cpu": "1", + "memory": "1536Mi" + }, + "requests": { + "cpu": "500m", + "memory": "1Gi" + } +} + +``` + +### Scale MySQL Replicas +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + replicas: 5 + version: "16.6" + storageType: Durable + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Update the `replicas` to `5`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` MySQLOpsRequest to update the `MySQL` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +mysql.gitops.kubedb.com/my-gitops 178m + +NAME VERSION STATUS AGE +mysql.kubedb.com/my-gitops 9.1.0 Ready 178m + +NAME TYPE STATUS AGE +mysqlopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-s9lyzc HorizontalScaling Successful 117m +mysqlopsrequest.ops.kubedb.com/my-gitops-verticalscaling-mw8s6j VerticalScaling Successful 144m + +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=my-gitops' +NAME READY STATUS RESTARTS AGE +my-gitops-0 2/2 Running 0 144m +my-gitops-1 2/2 Running 0 140m +my-gitops-2 2/2 Running 0 143m + +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Exapand MySQL Volume + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + replicas: 5 + version: "16.6" + storageType: Durable + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `10Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` MySQLOpsRequest to update the `MySQL` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +MySQL.gitops.kubedb.com/my-gitops 27m + +NAME VERSION STATUS AGE +MySQL.kubedb.com/my-gitops 16.6 Ready 27m + +NAME TYPE STATUS AGE +MySQLopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-wvxu5x HorizontalScaling Successful 6m +MySQLopsrequest.ops.kubedb.com/my-gitops-verticalscaling-i0kr1l VerticalScaling Successful 13m +MySQLopsrequest.ops.kubedb.com/my-gitops-volumeexpansion-2j5x5g VolumeExpansion Progressing 2s +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=my-gitops' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +data-my-gitops-0 Bound pvc-061f3622-234f-4f91-b4d1-b81aa8739503 10Gi RWO standard 30m +data-my-gitops-1 Bound pvc-045fc563-fb4e-416c-a9c2-b20c96532978 10Gi RWO standard 30m +data-my-gitops-2 Bound pvc-a0f1d8fd-a677-4407-80b1-104b9f7b4cd1 10Gi RWO standard 30m +data-my-gitops-3 Bound pvc-060b6fab-0c2d-4935-b31b-2866be68dd6f 10Gi RWO standard 8m58s +data-my-gitops-4 Bound pvc-8149b579-a40f-4cd8-ac37-6a2401fd7807 10Gi RWO standard 8m23s +``` + +## Reconfigure MySQL + +Before, create opsrequest parameters were, +```shell +kubectl exec -it -n demo my-gitops-0 -- bash +Defaulted container "mysql" out of: mysql, mysql-coordinator, mysql-init (init) +bash-5.1$ mysql -uroot -p$MYSQL_ROOT_PASSWORD +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 277 +Server version: 9.1.0 MySQL Community Server - GPL + +Copyright (c) 2000, 2024, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> show variables like 'max_connections'; ++-----------------+-------+ +| Variable_name | Value | ++-----------------+-------+ +| max_connections | 151 | ++-----------------+-------+ +1 row in set (0.01 sec) + +mysql> show variables like 'read_buffer_size'; ++------------------+--------+ +| Variable_name | Value | ++------------------+--------+ +| read_buffer_size | 131072 | ++------------------+--------+ +1 row in set (0.00 sec) + +``` +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/MySQL/configuration/using-config-file.md) +```yaml +apiVersion: v1 +stringData: + user.conf: | + max_connections=200 + shared_buffers=256MB +kind: Secret +metadata: + name: my-configuration + namespace: demo +type: Opaque +``` + +Now, we will add this file to `kubedb/my-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── my-configuration.yaml +│ └── MySQL.yaml +1 directories, 2 files +``` + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + configSecret: + name: my-configuration + replicas: 5 + version: "16.6" + storageType: Durable + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` MySQLOpsRequest to update the `MySQL` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +MySQL.gitops.kubedb.com/my-gitops 36m + +NAME VERSION STATUS AGE +MySQL.kubedb.com/my-gitops 16.6 Ready 36m + +NAME TYPE STATUS AGE +MySQLopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-wvxu5x HorizontalScaling Successful 15m +MySQLopsrequest.ops.kubedb.com/my-gitops-reconfigure-i4r23j Reconfigure Progressing 1s +MySQLopsrequest.ops.kubedb.com/my-gitops-verticalscaling-i0kr1l VerticalScaling Successful 23m +``` + +After Ops Request becomes `Succesful`, lets check these parameters, + +```bash +$ kubectl exec -it -n demo my-gitops-0 -- bash +Defaulted container "mysql" out of: mysql, mysql-coordinator, mysql-init (init) +bash-5.1$ mysql -uroot -p$MYSQL_ROOT_PASSWORD +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 3115 +Server version: 9.1.0 MySQL Community Server - GPL + +Copyright (c) 2000, 2024, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> show variables like 'max_connections'; ++-----------------+-------+ +| Variable_name | Value | ++-----------------+-------+ +| max_connections | 200 | ++-----------------+-------+ +1 row in set (0.00 sec) + +mysql> show variables like 'read_buffer_size'; ++------------------+---------+ +| Variable_name | Value | ++------------------+---------+ +| read_buffer_size | 1048576 | ++------------------+---------+ +1 row in set (0.01 sec) +``` +You can check the other pods same way. +So we have configured custom parameters. + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate MySQL Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will do that using gitops, create the file `kubedb/my-auth.yaml` with the following content, + +```yaml +apiVersion: v1 +data: + password: cGdwYXNzd29yZA== + username: cG9zdGdyZXM= +kind: Secret +metadata: + name: my-rotate-auth + namespace: demo +type: kubernetes.io/basic-auth +``` + +File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── my-auth.yaml +│ ├── my-configuration.yaml +│ └── MySQL.yaml +1 directories, 3 files +``` + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + version: "9.1.0" + replicas: 3 + authSecret: + kind: Secret + name: mysql-quickstart-auth-user + configuration: + secretName: my-configuration + topology: + mode: GroupReplication + group: + mode: Single-Primary + podTemplate: + spec: + containers: + - name: mysql + resources: + limits: + cpu: 1000m + memory: 1.5Gi + requests: + cpu: 500m + memory: 1Gi + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `my-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` MySQLOpsRequest to update the `MySQL` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +mysql.gitops.kubedb.com/my-gitops 5h2m + +NAME VERSION STATUS AGE +mysql.kubedb.com/my-gitops 9.1.0 Ready 5h2m + +NAME TYPE STATUS AGE +mysqlopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-s9lyzc HorizontalScaling Successful 4h1m +mysqlopsrequest.ops.kubedb.com/my-gitops-reconfigure-q4yrg0 Reconfigure Successful 4h28m +mysqlopsrequest.ops.kubedb.com/my-gitops-rotate-auth-5lg5o3 RotateAuth Successful 10m +mysqlopsrequest.ops.kubedb.com/my-gitops-verticalscaling-mw8s6j VerticalScaling Successful 4h28m + +``` + +After Ops Request becomes `Successful`, We can validate the changes connecting MySQL with new credentials. +```bash +$ kubectl exec -it -n demo my-gitops-0 -c mysql -- bash +bash-5.1$ mysql -uroot -p"Mysql2" +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 347 +Server version: 9.1.0 MySQL Community Server - GPL + +Copyright (c) 2000, 2024, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> SHOW DATABASES; ++--------------------+ +| Database | ++--------------------+ +| information_schema | +| kubedb_system | +| mysql | +| performance_schema | +| sys | ++--------------------+ +5 rows in set (0.00 sec) + +``` + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in MySQL. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls mysql-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/mysql-ca created +``` + +Now, Let's create an `Issuer` using the `MySQL-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: my-issuer + namespace: demo +spec: + ca: + secretName: MySQL-ca +``` + +Let's add that to our `kubedb/my-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── my-auth.yaml +│ ├── my-configuration.yaml +│ ├── my-issuer.yaml +│ └── MySQL.yaml +1 directories, 4 files +``` + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + version: "9.1.0" + replicas: 3 + authSecret: + kind: Secret + name: mysql-quickstart-auth-user + configuration: + secretName: my-configuration + topology: + mode: GroupReplication + group: + mode: Single-Primary + requireSSL: true + tls: + issuerRef: + apiGroup: cert-manager.io + kind: Issuer + name: my-issuer + certificates: + - alias: server + subject: + organizations: + - kubedb:server + dnsNames: + - localhost + ipAddresses: + - "127.0.0.1" + podTemplate: + spec: + containers: + - name: mysql + resources: + limits: + cpu: 1000m + memory: 1.5Gi + requests: + cpu: 500m + memory: 1Gi + storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` MySQLOpsRequest to update the `MySQL` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +mysql.gitops.kubedb.com/my-gitops 6h7m + +NAME VERSION STATUS AGE +mysql.kubedb.com/my-gitops 9.1.0 Ready 6h7m + +NAME TYPE STATUS AGE +mysqlopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-s9lyzc HorizontalScaling Successful 5h6m +mysqlopsrequest.ops.kubedb.com/my-gitops-reconfigure-q4yrg0 Reconfigure Successful 5h33m +mysqlopsrequest.ops.kubedb.com/my-gitops-reconfiguretls-39ir77 ReconfigureTLS Successful 4m39s +mysqlopsrequest.ops.kubedb.com/my-gitops-rotate-auth-5lg5o3 RotateAuth Successful 74m +mysqlopsrequest.ops.kubedb.com/my-gitops-verticalscaling-mw8s6j VerticalScaling Successful 5h33m + +``` + +After Ops Request becomes `Successful`, We can validate the changes connecting MySQL with new credentials. +```bash +$ kubectl exec -it -n demo my-gitops-0 -c mysql -- bash +bash-5.1$ mysql -uroot -p"Mysql2" +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 347 +Server version: 9.1.0 MySQL Community Server - GPL + +Copyright (c) 2000, 2024, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> SHOW DATABASES; ++--------------------+ +| Database | ++--------------------+ +| information_schema | +| kubedb_system | +| mysql | +| performance_schema | +| sys | ++--------------------+ +5 rows in set (0.00 sec) + +mysql> command terminated with exit code 137 +banusree@bonusree-datta-PC ~ [SIGKILL]> kubectl exec -it -n demo my-gitops-0 -c mysql -- bash +bash-5.1$ ls /etc/mysql/certs/ +ca.crt client.crt client.key server.crt server.key +bash-5.1$ mysql -u${MYSQL_ROOT_USERNAME} -p{MYSQL_ROOT_PASSWORD} +mysql: [Warning] Using a password on the command line interface can be insecure. +ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES) +bash-5.1$ mysql -uroot -p"Mysql2" +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 201 +Server version: 9.1.0 MySQL Community Server - GPL + +Copyright (c) 2000, 2024, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> SHOW VARIABLES LIKE '%ssl%'; ++---------------------------------------------------+-----------------------------+ +| Variable_name | Value | ++---------------------------------------------------+-----------------------------+ +| admin_ssl_ca | | +| admin_ssl_capath | | +| admin_ssl_cert | | +| admin_ssl_cipher | | +| admin_ssl_crl | | +| admin_ssl_crlpath | | +| admin_ssl_key | | +| clone_ssl_ca | /etc/mysql/certs/ca.crt | +| clone_ssl_cert | /etc/mysql/certs/server.crt | +| clone_ssl_key | /etc/mysql/certs/server.key | +| group_replication_recovery_ssl_ca | /etc/mysql/certs/ca.crt | +| group_replication_recovery_ssl_capath | | +| group_replication_recovery_ssl_cert | /etc/mysql/certs/server.crt | +| group_replication_recovery_ssl_cipher | | +| group_replication_recovery_ssl_crl | | +| group_replication_recovery_ssl_crlpath | | +| group_replication_recovery_ssl_key | /etc/mysql/certs/server.key | +| group_replication_recovery_ssl_verify_server_cert | OFF | +| group_replication_recovery_use_ssl | ON | +| group_replication_ssl_mode | VERIFY_CA | +| mysqlx_ssl_ca | | +| mysqlx_ssl_capath | | +| mysqlx_ssl_cert | | +| mysqlx_ssl_cipher | | +| mysqlx_ssl_crl | | +| mysqlx_ssl_crlpath | | +| mysqlx_ssl_key | | +| performance_schema_show_processlist | OFF | +| ssl_ca | /etc/mysql/certs/ca.crt | +| ssl_capath | /etc/mysql/certs | +| ssl_cert | /etc/mysql/certs/server.crt | +| ssl_cipher | | +| ssl_crl | | +| ssl_crlpath | | +| ssl_fips_mode | OFF | +| ssl_key | /etc/mysql/certs/server.key | +| ssl_session_cache_mode | ON | +| ssl_session_cache_timeout | 300 | ++---------------------------------------------------+-----------------------------+ +38 rows in set (0.00 sec) + +mysql> SHOW VARIABLES LIKE '%require_secure_transport%'; ++--------------------------+-------+ +| Variable_name | Value | ++--------------------------+-------+ +| require_secure_transport | OFF | ++--------------------------+-------+ +1 row in set (0.00 sec) + + +``` + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for MySQL. + +### Update Version + +List MySQL versions using `kubectl get MySQLversion` and choose desired version that is compatible for umyrade from current version. Check the version constraints and ops request [here](/docs/guides/MySQL/update-version/versionumyrading/index.md). + +Let's choose `17.4` in this example. + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + authSecret: + kind: Secret + name: my-rotate-auth + configSecret: + name: my-configuration + replicas: 5 + version: "17.4" + storageType: Durable + sslMode: verify-full + tls: + issuerRef: + apiGroup: cert-manager.io + name: my-issuer + kind: Issuer + certificates: + - alias: server + subject: + organizations: + - kubedb:server + dnsNames: + - localhost + ipAddresses: + - "127.0.0.1" + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` MySQLOpsRequest to update the `MySQL` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +MySQL.gitops.kubedb.com/my-gitops 3h25m + +NAME VERSION STATUS AGE +MySQL.kubedb.com/my-gitops 16.6 Ready 3h25m + +NAME TYPE STATUS AGE +MySQLopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-wvxu5x HorizontalScaling Successful 3h3m +MySQLopsrequest.ops.kubedb.com/my-gitops-reconfigure-i4r23j Reconfigure Successful 168m +MySQLopsrequest.ops.kubedb.com/my-gitops-reconfiguretls-91fseg ReconfigureTLS Successful 7m33s +MySQLopsrequest.ops.kubedb.com/my-gitops-rotate-auth-zot83x RotateAuth Successful 161m +MySQLopsrequest.ops.kubedb.com/my-gitops-versionupdate-1wxgt9 UpdateVersion Progressing 4s +MySQLopsrequest.ops.kubedb.com/my-gitops-verticalscaling-i0kr1l VerticalScaling Successful 3h11m +``` + + +Now, we are going to verify whether the `MySQL`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get MySQL -n demo my-gitops -o=jsonpath='{.spec.version}{"\n"}' +17.4 + +$ kubectl get petset -n demo my-gitops -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MySQL:17.4-alpine + +$ kubectl get pod -n demo my-gitops-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MySQL:17.4-alpine +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `MySQL.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: MySQL +metadata: + name: my-gitops + namespace: demo +spec: + authSecret: + kind: Secret + name: my-rotate-auth + configSecret: + name: my-configuration + replicas: 5 + version: "17.4" + storageType: Durable + sslMode: verify-full + tls: + issuerRef: + apiGroup: cert-manager.io + name: my-issuer + kind: Issuer + certificates: + - alias: server + subject: + organizations: + - kubedb:server + dnsNames: + - localhost + ipAddresses: + - "127.0.0.1" + podTemplate: + spec: + containers: + - name: MySQL + resources: + limits: + memory: 2Gi + requests: + cpu: 700m + memory: 2Gi + monitor: + agent: prometheus.io/operator + prometheus: + serviceMonitor: + labels: + release: prometheus + interval: 10s + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MySQL` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` MySQLOpsRequest to add the `MySQL` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get mysql.gitops.kubedb.com,mysql.kubedb.com,MySQLopsrequest -n demo +NAME AGE +MySQL.gitops.kubedb.com/my-gitops 3h34m + +NAME VERSION STATUS AGE +MySQL.kubedb.com/my-gitops 16.6 NotReady 3h34m + +NAME TYPE STATUS AGE +MySQLopsrequest.ops.kubedb.com/my-gitops-horizontalscaling-wvxu5x HorizontalScaling Successful 3h13m +MySQLopsrequest.ops.kubedb.com/my-gitops-reconfigure-i4r23j Reconfigure Successful 177m +MySQLopsrequest.ops.kubedb.com/my-gitops-reconfiguretls-91fseg ReconfigureTLS Successful 16m +MySQLopsrequest.ops.kubedb.com/my-gitops-restart-nhjk9u Restart Progressing 2s +MySQLopsrequest.ops.kubedb.com/my-gitops-rotate-auth-zot83x RotateAuth Successful 170m +MySQLopsrequest.ops.kubedb.com/my-gitops-versionupdate-1wxgt9 UpdateVersion Successful 9m30s +MySQLopsrequest.ops.kubedb.com/my-gitops-verticalscaling-i0kr1l VerticalScaling Successful 3h21m +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `.spec.leaderElection` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +- Learn MySQL [GitOps](/docs/guides/MySQL/concepts/MySQL-gitops.md) +- Learn MySQL Scaling + - [Horizontal Scaling](/docs/guides/MySQL/scaling/horizontal-scaling/overview/index.md) + - [Vertical Scaling](/docs/guides/MySQL/scaling/vertical-scaling/overview/index.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/MySQL/update-version/versionumyrading/index.md) +- Monitor your MySQL database with KubeDB using [built-in Prometheus](/docs/guides/MySQL/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mysql/gitops/overview.md b/docs/guides/mysql/gitops/overview.md new file mode 100644 index 000000000..bccfd07fc --- /dev/null +++ b/docs/guides/mysql/gitops/overview.md @@ -0,0 +1,43 @@ +--- +title: MySQL GitOps Overview +description: MySQL GitOps Overview +menu: + docs_{{ .version }}: + identifier: my-gitops-overview + name: Overview + parent: my-gitops-MySQL + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for MySQL + +This guide will give you an overview of how KubeDB `gitops` operator works with MySQL databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for managing MySQL databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [MySQL](/docs/guides/mysql/concepts/database/index.md) + - [MySQLOpsRequest](/docs/guides/mysql/concepts/opsrequest/index.md) + +## Workflow GitOps with MySQL + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + +
+ GitOps Flow +
Fig: GitOps process of MySQL
+
+ +1. **Define GitOps MySQL**: Create Custom Resource (CR) of kind `MySQL` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB MySQL CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the MySQLGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing MySQL databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running MySQL using GitOps. diff --git a/docs/guides/redis/gitops/_index.md b/docs/guides/redis/gitops/_index.md new file mode 100644 index 000000000..098093afa --- /dev/null +++ b/docs/guides/redis/gitops/_index.md @@ -0,0 +1,10 @@ +--- +title: Gitops Kafka +menu: + docs_{{ .version }}: + identifier: kf-gitops + name: Gitops + parent: kf-kafka-guides + weight: 30 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/redis/gitops/gitops.md b/docs/guides/redis/gitops/gitops.md new file mode 100644 index 000000000..1eafdcc0e --- /dev/null +++ b/docs/guides/redis/gitops/gitops.md @@ -0,0 +1,886 @@ +--- +title: Redis GitOps Guides +menu: + docs_{{ .version }}: + identifier: rd-gitops-guides + name: Gitops Redis + parent: rd-gitops-Redis + weight: 20 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Redis using KubeDB GitOps Operator + +This guide will show you how to use `KubeDB` GitOps operator to create Redis database and manage updates using GitOps worrdlow. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Install `KubeDB` operator in your cluster following the steps [here](/docs/setup/README.md). Pass `--set kubedb-crd-manager.installGitOpsCRDs=true` in the kubedb installation process to enable `GitOps` operator. + +- You need to install GitOps tools like `ArgoCD` or `FluxCD` and configure with your Git Repository to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. + + ```bash + $ kubectl create ns monitoring + namespace/monitoring created + + $ kubectl create ns demo + namespace/demo created + ``` +> Note: YAML files used in this tutorial are stored in [docs/examples/Redis](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Redis) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). + +## Creating Apps via CLI + +### For Public Repository +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace +``` + +### For Private Repository +#### Using HTTPS +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --username --password +``` + +#### Using SSH +```bash +argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa +``` + +## Create Redis Database using GitOps + +### Create a Redis GitOps CR +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 8.0.4 + mode: Cluster + cluster: + shards: 3 + replicas: 2 + storageType: Durable + storage: + resources: + requests: + storage: "1Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis + resources: + requests: + cpu: "100m" + memory: "100Mi" + deletionPolicy: WipeOut +``` + +Create a directory like below, +```bash +$ tree . +├── kubedb + └── Redis.yaml +1 directories, 1 files +``` + +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is created in your cluster. + +Our `gitops` operator will create an actual `Redis` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. + + +```bash +$ kubectl get redis.gitops.kubedb.com,redis.kubedb.com -n demo +NAME AGE +redis.gitops.kubedb.com/rd-gitops 8m46s + +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 8m46s + +``` + +List the resources created by `kubedb` operator created for `kubedb.com/v1` Redis. + +```bash +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=rd-gitops' +NAME AGE +petset.apps.k8s.appscode.com/rd-gitops-shard0 9m38s +petset.apps.k8s.appscode.com/rd-gitops-shard1 9m36s +petset.apps.k8s.appscode.com/rd-gitops-shard2 9m34s + +NAME READY STATUS RESTARTS AGE +pod/rd-gitops-shard0-0 1/1 Running 0 9m37s +pod/rd-gitops-shard0-1 1/1 Running 0 9m14s +pod/rd-gitops-shard1-0 1/1 Running 0 9m35s +pod/rd-gitops-shard1-1 1/1 Running 0 9m14s +pod/rd-gitops-shard2-0 1/1 Running 0 9m34s +pod/rd-gitops-shard2-1 1/1 Running 0 9m14s + +NAME TYPE DATA AGE +secret/rd-gitops-9a6ce0 Opaque 1 9m40s +secret/rd-gitops-auth kubernetes.io/basic-auth 2 65m + +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/rd-gitops ClusterIP 10.43.193.228 6379/TCP 9m40s +service/rd-gitops-pods ClusterIP None 6379/TCP,16379/TCP 9m40s + +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/rd-gitops kubedb.com/redis 8.0.4 9m34s + +``` + +## Update Redis Database using GitOps +### Scale Redis Replicas + + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 8.0.4 + mode: Cluster + cluster: + shards: 4 + replicas: 3 + storageType: Durable + storage: + resources: + requests: + storage: "1Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis + resources: + requests: + cpu: "100m" + memory: "100Mi" + deletionPolicy: WipeOut + ``` +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` RedisOpsRequest to update the `Redis` database replicas. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 12m + +NAME AGE +redis.gitops.kubedb.com/rd-gitops 12m + +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-1nblpl HorizontalScaling Successful 10m + +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, +```bash +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=rd-gitops' +NAME READY STATUS RESTARTS AGE +rd-gitops-shard0-0 1/1 Running 0 16m +rd-gitops-shard0-1 1/1 Running 0 16m +rd-gitops-shard0-2 1/1 Running 0 13m +rd-gitops-shard1-0 1/1 Running 0 16m +rd-gitops-shard1-1 1/1 Running 0 16m +rd-gitops-shard1-2 1/1 Running 0 12m +rd-gitops-shard2-0 1/1 Running 0 16m +rd-gitops-shard2-1 1/1 Running 0 16m +rd-gitops-shard2-2 1/1 Running 0 12m +rd-gitops-shard3-0 1/1 Running 0 14m +rd-gitops-shard3-1 1/1 Running 0 14m +rd-gitops-shard3-2 1/1 Running 0 11m + +``` + +We can also scale down the replicas by updating the `replicas` fields. + +### Scale Redis Database Resources + +before Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, +```bash +$ kubectl get pod -n demo rd-gitops-shard0-0 -o json | jq '.spec.containers[0].resources' +{ + "requests": { + "cpu": "100m", + "memory": "100Mi" + } +} + +``` + + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 8.0.4 + mode: Cluster + cluster: + shards: 4 + replicas: 3 + storageType: Durable + storage: + resources: + requests: + storage: "1Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis + resources: + requests: + cpu: "200m" + memory: "200Mi" + deletionPolicy: WipeOut +``` + + +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the resource changes and create a `RedisOpsRequest` to update the `Redis` database. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 28m + +NAME AGE +redis.gitops.kubedb.com/rd-gitops 28m + +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-1nblpl HorizontalScaling Successful 26m +redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-or3shk VerticalScaling Successful 10m + +``` + + + +### Expand Redis Volume + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` RedisOpsRequest to update the `Redis` database volume. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 23m + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 23m + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 19m +``` + +After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, +```bash +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=rd-gitops' +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +rd-gitops-data-rd-gitops-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +rd-gitops-data-rd-gitops-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +rd-gitops-data-rd-gitops-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +rd-gitops-data-rd-gitops-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m + +``` + +## Reconfigure Redis + +At first, we will create a secret containing `user.conf` file with required configuration settings. +To know more about this configuration file, check [here](/docs/guides/Redis/configuration/Redis-combined.md) +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: new-rd-combined-custom-config + namespace: demo +stringData: + server.properties: |- + log.retention.hours=125 +``` + +Now, we will add this file to `kubedb /rd-configuration.yaml`. + +```bash +$ tree . +├── kubedb +│ ├── rd-configuration.yaml +│ └── Redis.yaml +1 directories, 2 files +``` + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + configSecret: + name: new-rd-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` RedisOpsRequest to update the `Redis` database configuration. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 74m + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 74m + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 24m +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 70m + +``` + + + +> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. + +### Rotate Redis Auth + +To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. + +We will create a secret named `rd-rotate-auth ` with the following content, + +```bash +kubectl create secret generic rd-rotate-auth -n demo \ +--type=kubernetes.io/basic-auth \ +--from-literal=username=Redis \ +--from-literal=password=Redis-secret +secret/rd-rotate-auth created + +``` + + + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + authSecret: + kind: Secret + name: rd-rotate-auth + configuration: + secretName: new-rd-combined-custom-config + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Change the `authSecret` field to `rd-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` RedisOpsRequest to update the `Redis` database auth. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 7m11s + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 7m11s + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 17h +Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 28m +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 17h + +``` + + +### TLS configuration + +We can add, rotate or remove TLS configuration using `gitops`. + +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Redis. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. + +- Start off by generating a ca certificates using openssl. + +```bash +$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ./ca.key -out ./ca.crt -subj "/CN=ca/O=kubedb" +Generating a RSA private key +................+++++ +........................+++++ +writing new private key to './ca.key' +----- +``` + +- Now we are going to create a ca-secret using the certificate files that we have just generated. + +```bash +$ kubectl create secret tls Redis-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/Redis-ca created +``` + +Now, Let's create an `Issuer` using the `Redis-ca` secret that we have just created. The `YAML` file looks like this: + +```yaml +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: rd-issuer + namespace: demo +spec: + ca: + secretName: Redis-ca +``` + +Let's add that to our `kubedb /rd-issuer.yaml` file. File structure will look like this, +```bash +$ tree . +├── kubedb +│ ├── rd-configuration.yaml +│ ├── rd-issuer.yaml +│ └── Redis.yaml +1 directories, 4 files +``` + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 3.9.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` RedisOpsRequest to update the `Redis` database tls. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,Redis,rdops,pods -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 41m + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 75m + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 5d18h +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 5d1h +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 5d19h + +``` + + +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Redis. + +### Update Version + +List Redis versions using `kubectl get Redisversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Redis/update-version/update-version.md). + +Let's choose `4.0.0` in this example. + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + deletionPolicy: WipeOut +``` + +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` RedisOpsRequest to update the `Redis` database version. List the resources created by `gitops` operator in the `demo` namespace. + +```bash +$ kubectl get rd,redis,redisopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 4.0.0 Ready 3h47m + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 3h47m + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 5d22h +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 5d6h +Redisopsrequest.ops.kubedb.com/rd-gitops-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +``` + + +Now, we are going to verify whether the `Redis`, `PetSet` and it's `Pod` have updated with new image. Let's check, + +```bash +$ kubectl get Redis -n demo rd-gitops -o=jsonpath='{.spec.version}{"\n"}' +4.0.0 + +$ kubectl get petset -n demo rd-gitops-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Redis:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 + +$ kubectl get pod -n demo rd-gitops-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Redis:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +``` + +### Enable Monitoring + +If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). + +Update the `Redis.yaml` with the following, +```yaml +apiVersion: gitops.kubedb.com/v1alpha1 +kind: Redis +metadata: + name: rd-gitops + namespace: demo +spec: + version: 4.0.0 + topology: + broker: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + controller: + podTemplate: + spec: + containers: + - name: Redis + resources: + limits: + memory: 1536Mi + requests: + cpu: 500m + memory: 1536Mi + replicas: 2 + storage: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi + storageClassName: Standard + storageType: Durable + monitor: + agent: prometheus.io/operator + prometheus: + exporter: + port: 9091 + serviceMonitor: + labels: + release: prometheus + interval: 10s + deletionPolicy: WipeOut +``` + +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. + +Now, `gitops` operator will detect the monitoring changes and create a `Restart` RedisOpsRequest to add the `Redis` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +```bash +$ kubectl get Redises.gitops.kubedb.com,Redises.kubedb.com,Redisopsrequest -n demo +NAME TYPE VERSION STATUS AGE +Redis.kubedb.com/rd-gitops kubedb.com/v1 4.0.0 Ready 5h12m + +NAME AGE +Redis.gitops.kubedb.com/rd-gitops 5h12m + +NAME TYPE STATUS AGE +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 6d +Redisopsrequest.ops.kubedb.com/rd-gitops-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Redisopsrequest.ops.kubedb.com/rd-gitops-restart-ljpqih Restart Successful 3m51s +Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 5d7h +Redisopsrequest.ops.kubedb.com/rd-gitops-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 6d + +``` + +Verify the monitoring is enabled by checking the prometheus targets. + +There are some other fields that will trigger `Restart` ops request. +- `.spec.monitor` +- `.spec.spec.archiver` +- `.spec.remoteReplica` +- `spec.replication` +- `.spec.standbyMode` +- `.spec.streamingMode` +- `.spec.enforceGroup` +- `.spec.sslMode` etc. + + +## Next Steps + +[//]: # (- Learn Redis [GitOps](/docs/guides/Redis/concepts/Redis-gitops.md)) +- Learn Redis Scaling + - [Horizontal Scaling](/docs/guides/Redis/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/Redis/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/Redis/update-version/overview.md) +- Monitor your RedisQL database with KubeDB using [built-in Prometheus](/docs/guides/Redis/monitoring/using-builtin-prometheus.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/redis/gitops/overview.md b/docs/guides/redis/gitops/overview.md new file mode 100644 index 000000000..cf959a479 --- /dev/null +++ b/docs/guides/redis/gitops/overview.md @@ -0,0 +1,50 @@ +--- +title: Kafka Gitops Overview +description: Kafka Gitops Overview +menu: + docs_{{ .version }}: + identifier: kf-gitops-overview + name: Overview + parent: kf-gitops + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# GitOps Overview for Kafka + +This guide will give you an overview of how KubeDB `gitops` operator works with Kafka databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Kafka databases in Kubernetes. + +## Before You Begin + +- You should be familiar with the following `KubeDB` concepts: + - [Kafka](/docs/guides/kafka/concepts/kafka.md) + - [KafkaOpsRequest](/docs/guides/kafka/concepts/kafkaopsrequest.md) + + + +## Workflow GitOps with Kafka + +The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. + + +
+ + GitOps Flow + +
Fig: GitOps process of Kafka
+ +
+ +1. **Define GitOps Kafka**: Create Custom Resource (CR) of kind `Kafka` using the `gitops.kubedb.com/v1alpha1` API. +2. **Store in Git**: Push the CR to a Git repository. +3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Kafka CR in the Kubernetes cluster to deploy the database. +5. **Handle Updates**: When you update the KafkaGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. + +This flow makes managing Kafka databases efficient, reliable, and fully integrated with GitOps practices. + +In the next doc, we are going to show a step by step guide on running Kafka using GitOps. diff --git a/docs/guides/singlestore/configuration/config-file/index.md.bak b/docs/guides/singlestore/configuration/config-file/index.md.bak new file mode 100644 index 000000000..5199faa00 --- /dev/null +++ b/docs/guides/singlestore/configuration/config-file/index.md.bak @@ -0,0 +1,248 @@ +--- +title: Run SingleStore with Custom Configuration +menu: + docs_{{ .version }}: + identifier: guides-sdb-configuration-using-config-file + name: Config File + parent: guides-sdb-configuration + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Using Custom Configuration File + +KubeDB supports providing custom configuration for SingleStore. This tutorial will show you how to use KubeDB to run a SingleStore database with custom configuration. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + + $ kubectl get ns demo + NAME STATUS AGE + demo Active 5s + ``` + +> Note: YAML files used in this tutorial are stored in [docs/guides/singlestore/configuration/config-file/yamls](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/guides/singlestore/configuration/config-file/yamls) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +SingleStore allows to configure database via configuration file. The default configuration for SingleStore can be found in `/var/lib/memsql/instance/memsql.cnf` file. When SingleStore starts, it will look for custom configuration file in `/etc/memsql/conf.d` directory. If configuration file exist, SingleStore instance will use combined startup setting from both `/var/lib/memsql/instance/memsql.cnf` and `*.cnf` files in `/etc/memsql/conf.d` directory. This custom configuration will overwrite the existing default one. To know more about configuring SingleStore see [here](https://docs.singlestore.com/db/v8.7/reference/configuration-reference/cluster-config-files/singlestore-server-config-files/). + +At first, you have to create a custom configuration file and provide its name in `spec.configuration.secretName`. The operator reads this Secret internally and applies the configuration automatically. + +In this tutorial, we will configure [max_connections](https://docs.singlestore.com/db/v8.7/reference/configuration-reference/engine-variables/list-of-engine-variables/#in-depth-variable-definitions) and [read_buffer_size](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_read_buffer_size) via a custom config file. We will use configMap as volume source. + +## Create SingleStore License Secret + +We need SingleStore License to create SingleStore Database. So, Ensure that you have acquired a license and then simply pass the license by secret. + +```bash +$ kubectl create secret generic -n demo license-secret \ + --from-literal=username=license \ + --from-literal=password='your-license-set-here' +secret/license-secret created +``` + +## Custom Configuration + +At first, let's create `sdb-config.cnf` file setting `max_connections` and `read_buffer_size` parameters. + +```bash +cat < sdb-config.cnf +[server] +max_connections = 250 +read_buffer_size = 1048576 +EOF + +$ cat sdb-config.cnf +[server] +max_connections = 250 +read_buffer_size = 122880 +``` + +Now, create a secret with this configuration file. + +```bash +$ kubectl create secret generic -n demo sdb-configuration --from-file=./sdb-config.cnf +configmap/sdb-configuration created +``` + +Verify the secret has the configuration file. + +```yaml +$ kubectl get secret -n demo sdb-configuration -o yaml +apiVersion: v1 +data: + sdb-config.cnf: W3NlcnZlcl0KbWF4X2Nvbm5lY3Rpb25zID0gMjUwCnJlYWRfYnVmZmVyX3NpemUgPSAxMjI4ODAK +kind: Secret +metadata: + creationTimestamp: "2024-10-02T12:54:35Z" + name: sdb-configuration + namespace: demo + resourceVersion: "99627" + uid: c2621d8e-ebca-4300-af05-0180512ce031 +type: Opaque + + +``` + +Now, create SingleStore crd specifying `spec.topology.aggregator.configSecret` and `spec.topology.leaf.configSecret` field. + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/singlestore/configuration/config-file/yamls/sdb-custom.yaml +singlestore.kubedb.com/custom-sdb created +``` + +Below is the YAML for the SingleStore crd we just created. + +```yaml +apiVersion: kubedb.com/v1alpha2 +kind: Singlestore +metadata: + name: custom-sdb + namespace: demo +spec: + version: "8.7.10" + topology: + aggregator: + replicas: 2 + configuration: + secretName: sdb-configuration + podTemplate: + spec: + containers: + - name: singlestore + resources: + limits: + memory: "2Gi" + cpu: "600m" + requests: + memory: "2Gi" + cpu: "600m" + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + leaf: + replicas: 2 + configuration: + secretName: sdb-configuration + podTemplate: + spec: + containers: + - name: singlestore + resources: + limits: + memory: "2Gi" + cpu: "600m" + requests: + memory: "2Gi" + cpu: "600m" + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + licenseSecret: + kind: Secret + name: license-secret + storageType: Durable + deletionPolicy: WipeOut +``` + +Now, wait a few minutes. KubeDB operator will create necessary PVC, petset, services, secret etc. + +Check that the petset's pod is running + +```bash +$ kubectl get pod -n demo +NAME READY STATUS RESTARTS AGE +custom-sdb-aggregator-0 2/2 Running 0 94s +custom-sdb-aggregator-1 2/2 Running 0 88s +custom-sdb-leaf-0 2/2 Running 0 91s +custom-sdb-leaf-1 2/2 Running 0 86s + +$ kubectl get sdb -n demo +NAME TYPE VERSION STATUS AGE +custom-sdb kubedb.com/v1alpha2 8.7.10 Ready 4m29s + +``` + +We can see the database is in ready phase so it can accept conncetion. + +Now, we will check if the database has started with the custom configuration we have provided. + +> Read the comment written for the following commands. They contain the instructions and explanations of the commands. + +```bash +# Connceting to the database +$ kubectl exec -it -n demo custom-sdb-aggregator-0 -- bash +Defaulted container "singlestore" out of: singlestore, singlestore-coordinator, singlestore-init (init) +[memsql@custom-sdb-aggregator-0 /]$ memsql -uroot -p$ROOT_PASSWORD +singlestore-client: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 208 +Server version: 5.7.32 SingleStoreDB source distribution (compatible; MySQL Enterprise & MySQL Commercial) + +Copyright (c) 2000, 2022, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +# value of `max_conncetions` is same as provided +singlestore> show variables like 'max_connections'; ++-----------------+-------+ +| Variable_name | Value | ++-----------------+-------+ +| max_connections | 250 | ++-----------------+-------+ +1 row in set (0.00 sec) + +# value of `read_buffer_size` is same as provided +singlestore> show variables like 'read_buffer_size'; ++------------------+--------+ +| Variable_name | Value | ++------------------+--------+ +| read_buffer_size | 122880 | ++------------------+--------+ +1 row in set (0.00 sec) + +singlestore> exit +Bye + +``` +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl patch -n demo my/custom-sdb -p '{"spec":{"deletionPolicy":"WipeOut"}}' --type="merge" +kubectl delete -n demo my/custom-sdb +kubectl delete ns demo +``` + +If you would like to uninstall KubeDB operator, please follow the steps [here](/docs/setup/README.md). + +## Next Steps +- [Quickstart SingleStore](/docs/guides/singlestore/quickstart/quickstart.md) with KubeDB Operator. +- Detail concepts of [singlestore object](/docs/guides/singlestore/concepts/singlestore.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). From 9b27df0b5e819829aafd6f1fb24a653fd7ca504d Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 24 Feb 2026 12:11:58 +0600 Subject: [PATCH 09/14] maria Signed-off-by: Bonusree --- .../reconfigure_tls/elasticsearch.md | 2 +- docs/guides/mariadb/gitops/gitops.md | 164 +++++---------- .../using-config-file.md.bak | 182 +++++++++++++++++ .../using-config-file/index.md.bak | 193 ++++++++++++++++++ 4 files changed, 432 insertions(+), 109 deletions(-) create mode 100644 docs/guides/memcached/custom-configuration/using-config-file.md.bak create mode 100644 docs/guides/percona-xtradb/configuration/using-config-file/index.md.bak diff --git a/docs/guides/elasticsearch/reconfigure_tls/elasticsearch.md b/docs/guides/elasticsearch/reconfigure_tls/elasticsearch.md index 421fb1f60..afb4e3d19 100644 --- a/docs/guides/elasticsearch/reconfigure_tls/elasticsearch.md +++ b/docs/guides/elasticsearch/reconfigure_tls/elasticsearch.md @@ -140,7 +140,7 @@ spec: Let's apply the `YAML` file: ```bash -$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconfigure-tls/Elasticsearch-issuer.yaml +$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/elasticsearch/reconficure-tls/Elasticsearch-issuer.yaml issuer.cert-manager.io/es-issuer created ``` diff --git a/docs/guides/mariadb/gitops/gitops.md b/docs/guides/mariadb/gitops/gitops.md index ec2fdd8f5..8d9125eea 100644 --- a/docs/guides/mariadb/gitops/gitops.md +++ b/docs/guides/mariadb/gitops/gitops.md @@ -67,6 +67,7 @@ metadata: namespace: demo spec: version: "10.5.23" + replicas: 3 storageType: Durable storage: storageClassName: longhorn @@ -145,50 +146,30 @@ Update the `MariaDB.yaml` with the following, apiVersion: gitops.kubedb.com/v1alpha1 kind: MariaDB metadata: - name: MariaDB-prod + name: mariadb-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: MariaDB - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path - controller: - podTemplate: - spec: - containers: - - name: MariaDB - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce + version: "10.5.23" + replicas: 5 + storageType: Durable + podTemplate: + spec: + containers: + - name: mariadb resources: + limits: + cpu: 500m + memory: 1.2Gi requests: - storage: 1Gi - storageClassName: local-path - storageType: Durable + cpu: 100m + memory: 1.2Gi + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi deletionPolicy: WipeOut ``` @@ -197,29 +178,31 @@ Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit Now, `gitops` operator will detect the resource changes and create a `MariaDBOpsRequest` to update the `MariaDB` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,MariaDB,kfops -n demo -NAME TYPE VERSION STATUS AGE -MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 22h +$ kubectl get md,mariadbopsrequest -n demo +NAME VERSION STATUS AGE +mariadb.kubedb.com/mariadb-gitops 10.5.23 Ready 39m -NAME AGE -MariaDB.gitops.kubedb.com/MariaDB-prod 22h +NAME TYPE STATUS AGE +mariadbopsrequest.ops.kubedb.com/mariadb-gitops-horizontalscaling-g0l4rg HorizontalScaling Successful 22m +mariadbopsrequest.ops.kubedb.com/mariadb-gitops-verticalscaling-umbwjh VerticalScaling Successful 8m16s -NAME TYPE STATUS AGE -MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, ```bash -$ kubectl get pod -n demo MariaDB-prod-broker-0 -o json | jq '.spec.containers[0].resources' +$ kubectl get pod -n demo mariadb-gitops-0 -o json | jq '.spec.containers[0].resources' { "limits": { - "memory": "1536Mi" + "cpu": "500m", + "memory": "1288490188800m" }, "requests": { - "cpu": "500m", - "memory": "1536Mi" + "cpu": "100m", + "memory": "1288490188800m" } } + + ``` ### Scale MariaDB Replicas @@ -228,50 +211,19 @@ Update the `MariaDB.yaml` with the following, apiVersion: gitops.kubedb.com/v1alpha1 kind: MariaDB metadata: - name: MariaDB-prod + name: mariadb-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: MariaDB - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 3 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path - controller: - podTemplate: - spec: - containers: - - name: MariaDB - resources: - limits: - memory: 1540Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 3 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 1Gi - storageClassName: local-path + version: "10.5.23" + replicas: 5 storageType: Durable + storage: + storageClassName: longhorn + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi deletionPolicy: WipeOut ``` @@ -279,28 +231,24 @@ Update the `replicas` to `3`. Commit the changes and push to your Git repository Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` MariaDBOpsRequest to update the `MariaDB` database replicas. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,MariaDB,kfops -n demo -NAME TYPE VERSION STATUS AGE -MariaDB.kubedb.com/MariaDB-prod kubedb.com/v1 3.9.0 Ready 22h +$ kubectl get md,mariadbopsrequest -n demo +NAME VERSION STATUS AGE +mariadb.kubedb.com/mariadb-gitops 10.5.23 Ready 26m -NAME AGE -MariaDB.gitops.kubedb.com/MariaDB-prod 22h +NAME TYPE STATUS AGE +mariadbopsrequest.ops.kubedb.com/mariadb-gitops-horizontalscaling-g0l4rg HorizontalScaling Successful 9m2s -NAME TYPE STATUS AGE -MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -MariaDBopsrequest.ops.kubedb.com/MariaDB-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, ```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=MariaDB-prod' -NAME READY STATUS RESTARTS AGE -MariaDB-prod-broker-0 1/1 Running 0 34m -MariaDB-prod-broker-1 1/1 Running 0 33m -MariaDB-prod-broker-2 1/1 Running 0 33m -MariaDB-prod-controller-0 1/1 Running 0 32m -MariaDB-prod-controller-1 1/1 Running 0 31m -MariaDB-prod-controller-2 1/1 Running 0 31m +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=mariadb-gitops' +NAME READY STATUS RESTARTS AGE +mariadb-gitops-0 2/2 Running 0 26m +mariadb-gitops-1 2/2 Running 0 26m +mariadb-gitops-2 2/2 Running 0 26m +mariadb-gitops-3 2/2 Running 0 9m29s +mariadb-gitops-4 2/2 Running 0 8m29s ``` We can also scale down the replicas by updating the `replicas` fields. diff --git a/docs/guides/memcached/custom-configuration/using-config-file.md.bak b/docs/guides/memcached/custom-configuration/using-config-file.md.bak new file mode 100644 index 000000000..5f1387bd2 --- /dev/null +++ b/docs/guides/memcached/custom-configuration/using-config-file.md.bak @@ -0,0 +1,182 @@ +--- +title: Run Memcached with Custom Configuration +menu: + docs_{{ .version }}: + identifier: mc-using-config-file-configuration + name: Customize Configurations + parent: custom-configuration + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Using Custom Configuration File + +KubeDB supports providing custom configuration for Memcached. This tutorial will show you how to use KubeDB to run Memcached with custom configuration. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + + $ kubectl get ns demo + NAME STATUS AGE + demo Active 5s + ``` + +> Note: YAML files used in this tutorial are stored in [docs/examples/memcached](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/memcached) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +Memcached does not allows to configuration via any file. However, configuration parameters can be set as arguments while starting the memcached docker image. To keep similarity with other KubeDB supported databases which support configuration through a config file, KubeDB has added an additional executable script on top of the official memcached docker image. This script parses the configuration file then set them as arguments of memcached binary. + +To know more about configuring Memcached server see [here](https://github.com/memcached/memcached/wiki/ConfiguringServer). + +At first, you have to create a custom configuration file and provide its name in `spec.configuration.secretName`. The operator reads this Secret internally and applies the configuration automatically. + +In this tutorial, we will configure [max_connections](https://github.com/memcached/memcached/blob/ee171109b3afe1f30ff053166d205768ce635342/doc/protocol.txt#L672) and [limit_maxbytes](https://github.com/memcached/memcached/blob/ee171109b3afe1f30ff053166d205768ce635342/doc/protocol.txt#L720) via secret. + +Create a secret with custom configuration file: +```yaml +apiVersion: v1 +stringData: + memcached.conf: | + --conn-limit=500 + --memory-limit=128 +kind: Secret +metadata: + name: mc-configuration + namespace: demo + resourceVersion: "4505" +``` +Here, --con-limit means max simultaneous connections which is default value is 1024. +and --memory-limit means item memory in megabytes which default value is 64. + +```bash + $ kubectl apply -f mc-configuration.yaml +secret/mc-configuration created +``` + +Let's get the mc-configuration `secret` with custom configuration: + +```yaml +$ kubectl get secret -n demo mc-configuration -o yaml +apiVersion: v1 +data: + memcached.conf: LS1jb25uLWxpbWl0PTUwMAotLW1lbW9yeS1saW1pdD01MTIK +kind: Secret +metadata: + annotations: + kubectl.kubernetes.io/last-applied-configuration: | + {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"mc-configuration","namespace":"demo","resourceVersion":"4505"},"stringData":{"memcached.conf":"--conn-limit=500\n--memory-limit=512\n"}} + creationTimestamp: "2024-08-26T12:19:54Z" + name: mc-configuration + namespace: demo + resourceVersion: "4580860" + uid: 02d41fc0-590e-44d1-ae95-2ee8f9632d36 +type: Opaque +``` + +Now, create Memcached crd specifying `spec.configuration.secretName` field. + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/examples/memcached/configuration/mc-custom.yaml +memcached.kubedb.com/custom-memcached created +``` + +Below is the YAML for the Memcached crd we just created. + +```yaml +apiVersion: kubedb.com/v1 +kind: Memcached +metadata: + name: custom-memcached + namespace: demo +spec: + replicas: 1 + version: "1.6.22" + configuration: + secretName: mc-configuration + podTemplate: + spec: + containers: + - name: memcached + resources: + limits: + cpu: 500m + memory: 128Mi + requests: + cpu: 250m + memory: 64Mi + deletionPolicy: WipeOut +``` + +Now, wait a few minutes. KubeDB operator will create necessary petset, services etc. If everything goes well, we will see that a pod with the name `custom-memcached-0` has been created. + +Check if the database is ready + +```bash +$ kubectl get mc -n demo +NAME VERSION STATUS AGE +custom-memcached 1.6.22 Ready 17m +``` + +Now, we will check if the database has started with the custom configuration we have provided. We will use [stats](https://github.com/memcached/memcached/wiki/ConfiguringServer#inspecting-running-configuration) command to check the configuration. + +We will connect to `custom-memcached-0` pod from local-machine using port-frowarding. + +```bash +$ kubectl port-forward -n demo custom-memcached-0 11211 +Forwarding from 127.0.0.1:11211 -> 11211 +Forwarding from [::1]:11211 -> 11211 +``` + +Now, connect to the memcached server from a different terminal through `telnet`. + +```bash +$ telnet 127.0.0.1 11211 +Trying 127.0.0.1... +Connected to 127.0.0.1. +Escape character is '^]'. +stats +... +STAT max_connections 500 +... +STAT limit_maxbytes 134217728 +... +END +``` + +Here, `limit_maxbytes` is represented in bytes. + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl patch -n demo mc/custom-memcached -p '{"spec":{"deletionPolicy":"WipeOut"}}' --type="merge" +kubectl delete -n demo mc/custom-memcached + +kubectl patch -n demo drmn/custom-memcached -p '{"spec":{"wipeOut":true}}' --type="merge" +kubectl delete -n demo drmn/custom-memcached + +kubectl delete -n demo secret mc-configuration + +kubectl delete ns demo +``` + +If you would like to uninstall KubeDB operator, please follow the steps [here](/docs/setup/README.md). + +## Next Steps + +- Learn how to use KubeDB to run a Memcached server [here](/docs/guides/memcached/README.md). +- Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/percona-xtradb/configuration/using-config-file/index.md.bak b/docs/guides/percona-xtradb/configuration/using-config-file/index.md.bak new file mode 100644 index 000000000..0f1d95a8e --- /dev/null +++ b/docs/guides/percona-xtradb/configuration/using-config-file/index.md.bak @@ -0,0 +1,193 @@ +--- +title: Run PerconaXtraDB with Custom Configuration +menu: + docs_{{ .version }}: + identifier: guides-perconaxtradb-configuration-usingconfigfile + name: Config File + parent: guides-perconaxtradb-configuration + weight: 10 +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +> New to KubeDB? Please start [here](/docs/README.md). + +# Using Custom Configuration File + +KubeDB supports providing custom configuration for PerconaXtraDB. This tutorial will show you how to use KubeDB to run a PerconaXtraDB database with custom configuration. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +- Now, install KubeDB cli on your workstation and KubeDB operator in your cluster following the steps [here](/docs/setup/README.md). + +- To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. + + ```bash + $ kubectl create ns demo + namespace/demo created + + $ kubectl get ns demo + NAME STATUS AGE + demo Active 5s + ``` + +> Note: YAML files used in this tutorial are stored in [here](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/guides/percona-xtradb/configuration/using-config-file/examples) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). + +## Overview + +PerconaXtraDB allows to configure database via configuration file. The default configuration for PerconaXtraDB can be found in `/etc/my.cnf` file. KubeDB adds a new custom configuration directory `/etc/mysql/custom.conf.d` if it's enabled. When PerconaXtraDB starts, it will look for custom configuration file in `/etc/mysql/custom.conf.d` directory. If configuration file exist, PerconaXtraDB instance will use combined startup setting from both `/etc/my.cnf` and `*.cnf` files in `/etc/mysql/conf.d` and `/etc/mysql/custom.conf.d` directory. This custom configuration will overwrite the existing default one. + +At first, you have to create a custom configuration file and provide its name in `spec.configuration.secretName`. The operator reads this Secret internally and applies the configuration automatically. + +In this tutorial, we will configure [max_connections](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_max_connections/) and [read_buffer_size](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_read_buffer_size) via a custom config file. We will use Secret. + +## Custom Configuration + +At first, let's create `px-config.cnf` file setting `max_connections` and `read_buffer_size` parameters. + +```bash +cat < px-config.cnf +[mysqld] +max_connections = 200 +read_buffer_size = 1048576 +EOF + +$ cat px-config.cnf +[mysqld] +max_connections = 200 +read_buffer_size = 1048576 +``` + +Here, `read_buffer_size` is set to 1MB in bytes. + +Now, create a Secret with this configuration file. + +```bash +$ kubectl create secret generic -n demo px-configuration --from-file=./px-config.cnf +secret/md-configuration created +``` + +Verify the Secret has the configuration file. + +```bash +$ kubectl get secret -n demo px-configuration -o yaml +apiVersion: v1 +stringData: + px-config.cnf: | + [mysqld] + max_connections = 200 + read_buffer_size = 1048576 +kind: Secret +metadata: + name: px-configuration + namespace: demo + ... +``` + +Now, create PerconaXtraDB crd specifying `spec.configuration.secretName` field. + +```bash +$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/percona-xtradb/configuration/using-config-file/examples/px-custom.yaml +mysql.kubedb.com/custom-mysql created +``` + +Below is the YAML for the PerconaXtraDB crd we just created. + +```yaml +apiVersion: kubedb.com/v1 +kind: PerconaXtraDB +metadata: + name: sample-pxc + namespace: demo +spec: + version: "8.0.40" + configuration: + secretName: px-configuration + storageType: Durable + storage: + storageClassName: "standard" + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + deletionPolicy: WipeOut + +``` + +Now, wait a few minutes. KubeDB operator will create necessary PVC, petset, services, secret etc. If everything goes well, we will see that a pod with the name `sample-pxc-0` has been created. + +Check that the petset's pod is running + +```bash +$ kubectl get pod -n demo +NAME READY STATUS RESTARTS AGE +sample-pxc-0 2/2 Running 0 75m +sample-pxc-1 2/2 Running 0 95m +sample-pxc-2 2/2 Running 0 95m + +$ kubectl get perconaxtradb -n demo +NAME VERSION STATUS AGE +NAME VERSION STATUS AGE +sample-pxc 8.0.40 Ready 96m +``` + +We can see the database is in ready phase so it can accept connection. + +Now, we will check if the database has started with the custom configuration we have provided. + +> Read the comment written for the following commands. They contain the instructions and explanations of the commands. + +```bash +# Connecting to the database +$ kubectl exec -it -n demo sample-pxc-0 -- bash +Defaulted container "perconaxtradb" out of: perconaxtradb, px-coordinator, px-init (init) +bash-4.4$ mysql -u${MYSQL_ROOT_USERNAME} -p${MYSQL_ROOT_PASSWORD} +mysql: [Warning] Using a password on the command line interface can be insecure. +Welcome to the MySQL monitor. Commands end with ; or \g. +Your MySQL connection id is 1390 +Server version: 8.0.40-16.1 Percona XtraDB Cluster (GPL), Release rel16, Revision b141904, WSREP version 26.4.3 + +Copyright (c) 2009-2021 Percona LLC and/or its affiliates +Copyright (c) 2000, 2021, Oracle and/or its affiliates. + +Oracle is a registered trademark of Oracle Corporation and/or its +affiliates. Other names may be trademarks of their respective +owners. + +Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. + +mysql> show variables like 'max_connections'; ++-----------------+-------+ +| Variable_name | Value | ++-----------------+-------+ +| max_connections | 200 | ++-----------------+-------+ +1 row in set (0.01 sec) + + +# value of `read_buffer_size` is same as provided +mysql> show variables like 'read_buffer_size'; ++------------------+---------+ +| Variable_name | Value | ++------------------+---------+ +| read_buffer_size | 1048576 | ++------------------+---------+ +1 row in set (0.001 sec) + +mysql> exit +Bye +``` + +## Cleaning up + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +$ kubectl delete perconaxtradb -n demo sample-pxc +perconaxtradb.kubedb.com "sample-pxc" deleted +$ kubectl delete ns demo +namespace "demo" deleted +``` From d78227d571364f9a5f5b945bb0cf41803bd63add Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 3 Mar 2026 17:24:20 +0600 Subject: [PATCH 10/14] mssql some changes Signed-off-by: Bonusree --- docs/guides/mssqlserver/gitops/gitops.md | 346 +++++++++++----------- docs/guides/mysql/rotate-auth/overview.md | 2 +- 2 files changed, 174 insertions(+), 174 deletions(-) diff --git a/docs/guides/mssqlserver/gitops/gitops.md b/docs/guides/mssqlserver/gitops/gitops.md index 332c4f29d..0e940cbe7 100644 --- a/docs/guides/mssqlserver/gitops/gitops.md +++ b/docs/guides/mssqlserver/gitops/gitops.md @@ -1,6 +1,6 @@ --- -title: Elasticsearch Gitops Overview -description: Elasticsearch Gitops Overview +title: Mssqlserver Gitops Overview +description: Mssqlserver Gitops Overview menu: docs_{{ .version }}: identifier: es-gitops-overview @@ -15,9 +15,9 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Elasticsearch using KubeDB GitOps Operator +# GitOps Mssqlserver using KubeDB GitOps Operator -This guide will show you how to use `KubeDB` GitOps operator to create Elasticsearch database and manage updates using GitOps workflow. +This guide will show you how to use `KubeDB` GitOps operator to create Mssqlserver database and manage updates using GitOps workflow. ## Before You Begin @@ -34,7 +34,7 @@ This guide will show you how to use `KubeDB` GitOps operator to create Elasticse $ kubectl create ns demo namespace/demo created ``` -> Note: YAML files used in this tutorial are stored in [docs/examples/Elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). +> Note: YAML files used in this tutorial are stored in [docs/examples/Mssqlserver](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Mssqlserver) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). @@ -56,12 +56,12 @@ argocd app create kubedb --repo --path kubedb --dest-server https://k argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa ``` -## Create Elasticsearch Database using GitOps +## Create Mssqlserver Database using GitOps -### Create a Elasticsearch GitOps CR +### Create a Mssqlserver GitOps CR ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: name: es-gitops namespace: demo @@ -84,58 +84,58 @@ Create a directory like below, ```bash $ tree . ├── kubedb - └── Elasticsearch.yaml + └── Mssqlserver.yaml 1 directories, 1 files ``` -Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is created in your cluster. +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is created in your cluster. -Our `gitops` operator will create an actual `Elasticsearch` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. +Our `gitops` operator will create an actual `Mssqlserver` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get Elasticsearch.gitops.kubedb.com,Elasticsearch.kubedb.com -n demo +$ kubectl get Mssqlserver.gitops.kubedb.com,Mssqlserver.kubedb.com -n demo NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 2m56s +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 2m56s NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 2m56s +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 2m56s ``` -List the resources created by `kubedb` operator created for `kubedb.com/v1` Elasticsearch. +List the resources created by `kubedb` operator created for `kubedb.com/v1` Mssqlserver. ```bash -$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Mssqlserver-prod' NAME AGE -petset.apps.k8s.appscode.com/Elasticsearch-prod-broker 4m49s -petset.apps.k8s.appscode.com/Elasticsearch-prod-controller 4m47s +petset.apps.k8s.appscode.com/Mssqlserver-prod-broker 4m49s +petset.apps.k8s.appscode.com/Mssqlserver-prod-controller 4m47s NAME READY STATUS RESTARTS AGE -pod/Elasticsearch-prod-broker-0 1/1 Running 0 4m48s -pod/Elasticsearch-prod-broker-1 1/1 Running 0 4m42s -pod/Elasticsearch-prod-controller-0 1/1 Running 0 4m47s -pod/Elasticsearch-prod-controller-1 1/1 Running 0 4m40s +pod/Mssqlserver-prod-broker-0 1/1 Running 0 4m48s +pod/Mssqlserver-prod-broker-1 1/1 Running 0 4m42s +pod/Mssqlserver-prod-controller-0 1/1 Running 0 4m47s +pod/Mssqlserver-prod-controller-1 1/1 Running 0 4m40s NAME TYPE DATA AGE -secret/Elasticsearch-prod-auth kubernetes.io/basic-auth 2 4m51s +secret/Mssqlserver-prod-auth kubernetes.io/basic-auth 2 4m51s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/Elasticsearch-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s +service/Mssqlserver-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s NAME TYPE VERSION AGE -appbinding.appcatalog.appscode.com/Elasticsearch-prod kubedb.com/Elasticsearch 3.9.0 4m47s +appbinding.appcatalog.appscode.com/Mssqlserver-prod kubedb.com/Mssqlserver 3.9.0 4m47s ``` -## Update Elasticsearch Database using GitOps +## Update Mssqlserver Database using GitOps -### Scale Elasticsearch Database Resources +### Scale Mssqlserver Database Resources -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 3.9.0 @@ -144,7 +144,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1540Mi @@ -163,7 +163,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1540Mi @@ -182,25 +182,25 @@ spec: deletionPolicy: WipeOut ``` -Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Elasticsearch` database. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Mssqlserver` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,Elasticsearch,kfops -n demo +$ $ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 22h NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, ```bash -$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.containers[0].resources' +$ kubectl get pod -n demo Mssqlserver-prod-broker-0 -o json | jq '.spec.containers[0].resources' { "limits": { "memory": "1536Mi" @@ -212,13 +212,13 @@ $ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.contai } ``` -### Scale Elasticsearch Replicas -Update the `Elasticsearch.yaml` with the following, +### Scale Mssqlserver Replicas +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 3.9.0 @@ -227,7 +227,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1540Mi @@ -246,7 +246,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1540Mi @@ -265,44 +265,44 @@ spec: deletionPolicy: WipeOut ``` -Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Elasticsearch` database replicas. List the resources created by `gitops` operator in the `demo` namespace. +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Mssqlserver` database replicas. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 22h NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, ```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Mssqlserver-prod' NAME READY STATUS RESTARTS AGE -Elasticsearch-prod-broker-0 1/1 Running 0 34m -Elasticsearch-prod-broker-1 1/1 Running 0 33m -Elasticsearch-prod-broker-2 1/1 Running 0 33m -Elasticsearch-prod-controller-0 1/1 Running 0 32m -Elasticsearch-prod-controller-1 1/1 Running 0 31m -Elasticsearch-prod-controller-2 1/1 Running 0 31m +Mssqlserver-prod-broker-0 1/1 Running 0 34m +Mssqlserver-prod-broker-1 1/1 Running 0 33m +Mssqlserver-prod-broker-2 1/1 Running 0 33m +Mssqlserver-prod-controller-0 1/1 Running 0 32m +Mssqlserver-prod-controller-1 1/1 Running 0 31m +Mssqlserver-prod-controller-2 1/1 Running 0 31m ``` We can also scale down the replicas by updating the `replicas` fields. -### Expand Elasticsearch Volume +### Expand Mssqlserver Volume -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 3.9.0 @@ -311,7 +311,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -330,7 +330,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -349,39 +349,39 @@ spec: deletionPolicy: WipeOut ``` -Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Elasticsearch` database volume. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Mssqlserver` database volume. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 23m +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 23m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 23m +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 23m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m ``` After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, ```bash -$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Mssqlserver-prod' NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE -Elasticsearch-prod-data-Elasticsearch-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m +Mssqlserver-prod-data-Mssqlserver-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +Mssqlserver-prod-data-Mssqlserver-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +Mssqlserver-prod-data-Mssqlserver-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +Mssqlserver-prod-data-Mssqlserver-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m ``` -## Reconfigure Elasticsearch +## Reconfigure Mssqlserver At first, we will create a secret containing `user.conf` file with required configuration settings. -To know more about this configuration file, check [here](/docs/guides/Elasticsearch/configuration/Elasticsearch-combined.md) +To know more about this configuration file, check [here](/docs/guides/Mssqlserver/configuration/Mssqlserver-combined.md) ```yaml apiVersion: v1 kind: Secret @@ -399,16 +399,16 @@ Now, we will add this file to `kubedb /kf-configuration.yaml`. $ tree . ├── kubedb │ ├── kf-configuration.yaml -│ └── Elasticsearch.yaml +│ └── Mssqlserver.yaml 1 directories, 2 files ``` -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: configSecret: @@ -419,7 +419,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -438,7 +438,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -457,21 +457,21 @@ spec: deletionPolicy: WipeOut ``` -Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Elasticsearch` database configuration. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Mssqlserver` database configuration. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 74m +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 74m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 74m +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 74m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 24m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfigure-ukj41o Reconfigure Successful 24m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m ``` @@ -479,7 +479,7 @@ Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr > We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. -### Rotate Elasticsearch Auth +### Rotate Mssqlserver Auth To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. @@ -488,20 +488,20 @@ We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the fo ```bash kubectl create secret generic kf-rotate-auth -n demo \ --type=kubernetes.io/basic-auth \ ---from-literal=username=Elasticsearch \ ---from-literal=password=Elasticsearch-secret +--from-literal=username=Mssqlserver \ +--from-literal=password=Mssqlserver-secret secret/kf-rotate-auth created ``` -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: authSecret: @@ -515,7 +515,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -534,7 +534,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -553,22 +553,22 @@ spec: deletionPolicy: WipeOut ``` -Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Elasticsearch` database auth. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Mssqlserver` database auth. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 7m11s +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 7m11s NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 7m11s +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 7m11s NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 17h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 28m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfigure-ukj41o Reconfigure Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-rotate-auth-43ris8 RotateAuth Successful 28m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h ``` @@ -577,7 +577,7 @@ Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr We can add, rotate or remove TLS configuration using `gitops`. -To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Mssqlserver. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. - Start off by generating a ca certificates using openssl. @@ -593,14 +593,14 @@ writing new private key to './ca.key' - Now we are going to create a ca-secret using the certificate files that we have just generated. ```bash -$ kubectl create secret tls Elasticsearch-ca \ +$ kubectl create secret tls Mssqlserver-ca \ --cert=ca.crt \ --key=ca.key \ --namespace=demo -secret/Elasticsearch-ca created +secret/Mssqlserver-ca created ``` -Now, Let's create an `Issuer` using the `Elasticsearch-ca` secret that we have just created. The `YAML` file looks like this: +Now, Let's create an `Issuer` using the `Mssqlserver-ca` secret that we have just created. The `YAML` file looks like this: ```yaml apiVersion: cert-manager.io/v1 @@ -610,7 +610,7 @@ metadata: namespace: demo spec: ca: - secretName: Elasticsearch-ca + secretName: Mssqlserver-ca ``` Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, @@ -619,16 +619,16 @@ $ tree . ├── kubedb │ ├── kf-configuration.yaml │ ├── kf-issuer.yaml -│ └── Elasticsearch.yaml +│ └── Mssqlserver.yaml 1 directories, 4 files ``` -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 3.9.0 @@ -637,7 +637,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -656,7 +656,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -675,41 +675,41 @@ spec: deletionPolicy: WipeOut ``` -Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Elasticsearch` database tls. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Mssqlserver` database tls. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops,pods -n demo +$ kubectl get kf,Mssqlserver,kfops,pods -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 41m +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 41m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 75m +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 75m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d18h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h ``` -> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Elasticsearch. +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Mssqlserver. ### Update Version -List Elasticsearch versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Elasticsearch/update-version/update-version.md). +List Mssqlserver versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Mssqlserver/update-version/update-version.md). Let's choose `4.0.0` in this example. -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 4.0.0 @@ -718,7 +718,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -737,7 +737,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -756,50 +756,50 @@ spec: deletionPolicy: WipeOut ``` -Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Elasticsearch` database version. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Mssqlserver` database version. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 3h47m +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 4.0.0 Ready 3h47m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 3h47m +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 3h47m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d22h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h ``` -Now, we are going to verify whether the `Elasticsearch`, `PetSet` and it's `Pod` have updated with new image. Let's check, +Now, we are going to verify whether the `Mssqlserver`, `PetSet` and it's `Pod` have updated with new image. Let's check, ```bash -$ kubectl get Elasticsearch -n demo Elasticsearch-prod -o=jsonpath='{.spec.version}{"\n"}' +$ kubectl get Mssqlserver -n demo Mssqlserver-prod -o=jsonpath='{.spec.version}{"\n"}' 4.0.0 -$ kubectl get petset -n demo Elasticsearch-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get petset -n demo Mssqlserver-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Mssqlserver:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 -$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get pod -n demo Mssqlserver-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/Mssqlserver:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 ``` ### Enable Monitoring If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). -Update the `Elasticsearch.yaml` with the following, +Update the `Mssqlserver.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: Mssqlserver metadata: - name: Elasticsearch-prod + name: Mssqlserver-prod namespace: demo spec: version: 4.0.0 @@ -808,7 +808,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -827,7 +827,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: Mssqlserver resources: limits: memory: 1536Mi @@ -855,24 +855,24 @@ spec: deletionPolicy: WipeOut ``` -Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Mssqlserver` CR is updated in your cluster. -Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Elasticsearch` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Mssqlserver` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. ```bash $ kubectl get Elasticsearches.gitops.kubedb.com,Elasticsearches.kubedb.com,Elasticsearchopsrequest -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 5h12m +Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 4.0.0 Ready 5h12m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 5h12m +Mssqlserver.gitops.kubedb.com/Mssqlserver-prod 5h12m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 6d -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-restart-ljpqih Restart Successful 3m51s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfigure-ukj41o Reconfigure Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-restart-ljpqih Restart Successful 3m51s +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d ``` @@ -891,10 +891,10 @@ There are some other fields that will trigger `Restart` ops request. ## Next Steps -[//]: # (- Learn Elasticsearch [GitOps](/docs/guides/Elasticsearch/concepts/Elasticsearch-gitops.md)) -- Learn Elasticsearch Scaling - - [Horizontal Scaling](/docs/guides/Elasticsearch/scaling/horizontal-scaling/combined.md) - - [Vertical Scaling](/docs/guides/Elasticsearch/scaling/vertical-scaling/combined.md) -- Learn Version Update Ops Request and Constraints [here](/docs/guides/Elasticsearch/update-version/overview.md) -- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/Elasticsearch/monitoring/using-builtin-prometheus.md). +[//]: # (- Learn Mssqlserver [GitOps](/docs/guides/Mssqlserver/concepts/Mssqlserver-gitops.md)) +- Learn Mssqlserver Scaling + - [Horizontal Scaling](/docs/guides/Mssqlserver/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/Mssqlserver/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/Mssqlserver/update-version/overview.md) +- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/Mssqlserver/monitoring/using-builtin-prometheus.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mysql/rotate-auth/overview.md b/docs/guides/mysql/rotate-auth/overview.md index 51fd22409..37e6392a9 100644 --- a/docs/guides/mysql/rotate-auth/overview.md +++ b/docs/guides/mysql/rotate-auth/overview.md @@ -4,7 +4,7 @@ menu: docs_{{ .version }}: identifier: guides-mysql-rotate-auth-overview name: Overview - parent: ms-rotate-auth + parent: guides-mysql-rotate-auth weight: 5 menu_name: docs_{{ .version }} section_menu_id: guides From 8b19eefdaf599d1492a80926fea4671633db20ea Mon Sep 17 00:00:00 2001 From: Bonusree Date: Thu, 12 Mar 2026 10:24:45 +0600 Subject: [PATCH 11/14] csi add Signed-off-by: Bonusree --- docs/guides/mssqlserver/gitops/gitops.md | 5 +++-- docs/guides/mssqlserver/reconfigure-tls/ag_cluster.md | 3 +++ docs/guides/mssqlserver/reconfigure-tls/standalone.md | 2 ++ 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/guides/mssqlserver/gitops/gitops.md b/docs/guides/mssqlserver/gitops/gitops.md index 0e940cbe7..87cb73f1d 100644 --- a/docs/guides/mssqlserver/gitops/gitops.md +++ b/docs/guides/mssqlserver/gitops/gitops.md @@ -187,7 +187,7 @@ Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Mssqlserver` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,Mssqlserver,kfops -n demo +$ kubectl get kf,Mssqlserver,kfops -n demo NAME TYPE VERSION STATUS AGE Mssqlserver.kubedb.com/Mssqlserver-prod kubedb.com/v1 3.9.0 Ready 22h @@ -577,7 +577,8 @@ Elasticsearchopsrequest.ops.kubedb.com/Mssqlserver-prod-volumeexpansion-41xthr We can add, rotate or remove TLS configuration using `gitops`. -To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Mssqlserver. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Mssqlserver. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. As well as we need to install [csi-driver-cacerts](https://github.com/kubeops/csi-driver-cacerts) which will be used to add self-signed ca certificates to the OS trusted certificate store (eg, /etc/ssl/certs/ca-certificates.crt) + - Start off by generating a ca certificates using openssl. diff --git a/docs/guides/mssqlserver/reconfigure-tls/ag_cluster.md b/docs/guides/mssqlserver/reconfigure-tls/ag_cluster.md index 7a17b63d4..c5cf3af71 100644 --- a/docs/guides/mssqlserver/reconfigure-tls/ag_cluster.md +++ b/docs/guides/mssqlserver/reconfigure-tls/ag_cluster.md @@ -24,6 +24,9 @@ KubeDB supports reconfigure i.e. add, remove, update and rotation of TLS/SSL cer - To configure TLS/SSL in `MSSQLServer`, `KubeDB` uses `cert-manager` to issue certificates. So first you have to make sure that the cluster has `cert-manager` installed. To install `cert-manager` in your cluster following steps [here](https://cert-manager.io/docs/installation/kubernetes/). +- Install [csi-driver-cacerts](https://github.com/kubeops/csi-driver-cacerts) which will be used to add self-signed ca certificates to the OS trusted certificate store (eg, /etc/ssl/certs/ca-certificates.crt) + + - To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. ```bash diff --git a/docs/guides/mssqlserver/reconfigure-tls/standalone.md b/docs/guides/mssqlserver/reconfigure-tls/standalone.md index 19f46f300..04903508c 100644 --- a/docs/guides/mssqlserver/reconfigure-tls/standalone.md +++ b/docs/guides/mssqlserver/reconfigure-tls/standalone.md @@ -24,6 +24,8 @@ KubeDB supports reconfigure i.e. add, remove, update and rotation of TLS/SSL cer - To configure TLS/SSL in `MSSQLServer`, `KubeDB` uses `cert-manager` to issue certificates. So first you have to make sure that the cluster has `cert-manager` installed. To install `cert-manager` in your cluster following steps [here](https://cert-manager.io/docs/installation/kubernetes/). +- Install [csi-driver-cacerts](https://github.com/kubeops/csi-driver-cacerts) which will be used to add self-signed ca certificates to the OS trusted certificate store (eg, /etc/ssl/certs/ca-certificates.crt) + - To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial. ```bash From 52fa9411ca03d5153a752b020b5a338744ba3573 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Tue, 17 Mar 2026 13:01:28 +0600 Subject: [PATCH 12/14] mg Signed-off-by: Bonusree --- docs/guides/mongodb/gitops/_index.md | 10 +- docs/guides/mongodb/gitops/gitops.md | 378 +++++++++++++------------ docs/guides/mongodb/gitops/overview.md | 26 +- 3 files changed, 208 insertions(+), 206 deletions(-) diff --git a/docs/guides/mongodb/gitops/_index.md b/docs/guides/mongodb/gitops/_index.md index cc82b5f6b..967af159f 100644 --- a/docs/guides/mongodb/gitops/_index.md +++ b/docs/guides/mongodb/gitops/_index.md @@ -1,10 +1,10 @@ --- -title: Elasticsearch Gitops +title: MongoDB Gitops menu: docs_{{ .version }}: - identifier: es-gitops - name: Elasticsearch Gitops - parent: es-elasticsearch-guides - weight: 70 + identifier: mg-gitops-mongodb + name: Gitops + parent: mg-mongodb-guides + weight: 47 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/mongodb/gitops/gitops.md b/docs/guides/mongodb/gitops/gitops.md index 332c4f29d..7e12027a6 100644 --- a/docs/guides/mongodb/gitops/gitops.md +++ b/docs/guides/mongodb/gitops/gitops.md @@ -1,11 +1,11 @@ --- -title: Elasticsearch Gitops Overview -description: Elasticsearch Gitops Overview +title: MongoDB Gitops +description: MongoDB Gitops menu: docs_{{ .version }}: - identifier: es-gitops-overview - name: Overview - parent: es-gitops + identifier: es-gitops + name: MongoDB Gitops + parent: mg-gitops-mongodb weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides @@ -15,9 +15,9 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Elasticsearch using KubeDB GitOps Operator +# GitOps MongoDB using KubeDB GitOps Operator -This guide will show you how to use `KubeDB` GitOps operator to create Elasticsearch database and manage updates using GitOps workflow. +This guide will show you how to use `KubeDB` GitOps operator to create MongoDB database and manage updates using GitOps workflow. ## Before You Begin @@ -31,10 +31,11 @@ This guide will show you how to use `KubeDB` GitOps operator to create Elasticse $ kubectl create ns monitoring namespace/monitoring created + $ kubectl create ns demo namespace/demo created ``` -> Note: YAML files used in this tutorial are stored in [docs/examples/Elasticsearch](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/Elasticsearch) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). +> Note: YAML files used in this tutorial are stored in [docs/examples/MongoDB](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/examples/MongoDB) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs). We are going to use `ArgoCD` in this tutorial. You can install `ArgoCD` in your cluster by following the steps [here](https://argo-cd.readthedocs.io/en/stable/getting_started/). Also, you need to install `argocd` CLI in your local machine. You can install `argocd` CLI by following the steps [here](https://argo-cd.readthedocs.io/en/stable/cli_installation/). @@ -56,86 +57,87 @@ argocd app create kubedb --repo --path kubedb --dest-server https://k argocd app create kubedb --repo --path kubedb --dest-server https://kubernetes.default.svc --dest-namespace --ssh-private-key-path ~/.ssh/id_rsa ``` -## Create Elasticsearch Database using GitOps +## Create MongoDB Database using GitOps -### Create a Elasticsearch GitOps CR +### Create a MongoDB GitOps CR ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: es-gitops + name: mg-gitops namespace: demo spec: - version: xpack-8.2.3 - enableSSL: true + version: "8.0.10" + replicaSet: + name: "replicaset" replicas: 2 storageType: Durable storage: storageClassName: longhorn accessModes: - - ReadWriteOnce + - ReadWriteOnce resources: requests: storage: 1Gi - deletionPolicy: WipeOut ``` Create a directory like below, ```bash $ tree . ├── kubedb - └── Elasticsearch.yaml + └── MongoDB.yaml 1 directories, 1 files ``` -Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is created in your cluster. +Now commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is created in your cluster. -Our `gitops` operator will create an actual `Elasticsearch` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. +Our `gitops` operator will create an actual `MongoDB` database CR in the cluster. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get Elasticsearch.gitops.kubedb.com,Elasticsearch.kubedb.com -n demo -NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 2m56s +$ kubectl get MongoDB.gitops.kubedb.com,MongoDB.kubedb.com -n demo +NAME AGE +mongodb.gitops.kubedb.com/mg-gitops 33m + +NAME VERSION STATUS AGE +mongodb.kubedb.com/mg-gitops 8.0.10 Ready 33m -NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 2m56s ``` -List the resources created by `kubedb` operator created for `kubedb.com/v1` Elasticsearch. +List the resources created by `kubedb` operator created for `kubedb.com/v1` MongoDB. ```bash -$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' -NAME AGE -petset.apps.k8s.appscode.com/Elasticsearch-prod-broker 4m49s -petset.apps.k8s.appscode.com/Elasticsearch-prod-controller 4m47s +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=mg-gitops' +NAME AGE +petset.apps.k8s.appscode.com/mg-gitops 34m + +NAME READY STATUS RESTARTS AGE +pod/mg-gitops-0 2/2 Running 0 34m +pod/mg-gitops-1 2/2 Running 0 33m -NAME READY STATUS RESTARTS AGE -pod/Elasticsearch-prod-broker-0 1/1 Running 0 4m48s -pod/Elasticsearch-prod-broker-1 1/1 Running 0 4m42s -pod/Elasticsearch-prod-controller-0 1/1 Running 0 4m47s -pod/Elasticsearch-prod-controller-1 1/1 Running 0 4m40s +NAME TYPE DATA AGE +secret/mg-gitops-auth kubernetes.io/basic-auth 2 34m +secret/mg-gitops-key Opaque 1 34m -NAME TYPE DATA AGE -secret/Elasticsearch-prod-auth kubernetes.io/basic-auth 2 4m51s +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +service/mg-gitops ClusterIP 10.43.206.183 27017/TCP 34m +service/mg-gitops-pods ClusterIP None 27017/TCP 34m -NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/Elasticsearch-prod-pods ClusterIP None 9092/TCP,9093/TCP,29092/TCP 4m51s +NAME TYPE VERSION AGE +appbinding.appcatalog.appscode.com/mg-gitops kubedb.com/mongodb 8.0.10 34m -NAME TYPE VERSION AGE -appbinding.appcatalog.appscode.com/Elasticsearch-prod kubedb.com/Elasticsearch 3.9.0 4m47s ``` -## Update Elasticsearch Database using GitOps +## Update MongoDB Database using GitOps -### Scale Elasticsearch Database Resources +### Scale MongoDB Database Resources -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 3.9.0 @@ -144,7 +146,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1540Mi @@ -163,7 +165,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1540Mi @@ -182,25 +184,25 @@ spec: deletionPolicy: WipeOut ``` -Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Resource Requests and Limits are updated to `700m` CPU and `2Gi` Memory. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `Elasticsearch` database. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the resource changes and create a `ElasticsearchOpsRequest` to update the `MongoDB` database. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ $ kubectl get kf,Elasticsearch,kfops -n demo +$ $ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h +MongoDB.gitops.kubedb.com/MongoDB-prod 22h NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-verticalscaling-i0kr1l VerticalScaling Successful 2s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the one of the pod, ```bash -$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.containers[0].resources' +$ kubectl get pod -n demo mg-gitops-0 -o json | jq '.spec.containers[0].resources' { "limits": { "memory": "1536Mi" @@ -212,13 +214,13 @@ $ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o json | jq '.spec.contai } ``` -### Scale Elasticsearch Replicas -Update the `Elasticsearch.yaml` with the following, +### Scale MongoDB Replicas +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 3.9.0 @@ -227,7 +229,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1540Mi @@ -246,7 +248,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1540Mi @@ -265,44 +267,44 @@ spec: deletionPolicy: WipeOut ``` -Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. -Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `Elasticsearch` database replicas. List the resources created by `gitops` operator in the `demo` namespace. +Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. +Now, `gitops` operator will detect the replica changes and create a `HorizontalScaling` ElasticsearchOpsRequest to update the `MongoDB` database replicas. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 22h +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 22h NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 22h +MongoDB.gitops.kubedb.com/MongoDB-prod 22h NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, ```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=MongoDB-prod' NAME READY STATUS RESTARTS AGE -Elasticsearch-prod-broker-0 1/1 Running 0 34m -Elasticsearch-prod-broker-1 1/1 Running 0 33m -Elasticsearch-prod-broker-2 1/1 Running 0 33m -Elasticsearch-prod-controller-0 1/1 Running 0 32m -Elasticsearch-prod-controller-1 1/1 Running 0 31m -Elasticsearch-prod-controller-2 1/1 Running 0 31m +mg-gitops-0 1/1 Running 0 34m +mg-gitops-1 1/1 Running 0 33m +mg-gitops-2 1/1 Running 0 33m +MongoDB-prod-controller-0 1/1 Running 0 32m +MongoDB-prod-controller-1 1/1 Running 0 31m +MongoDB-prod-controller-2 1/1 Running 0 31m ``` We can also scale down the replicas by updating the `replicas` fields. -### Expand Elasticsearch Volume +### Expand MongoDB Volume -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 3.9.0 @@ -311,7 +313,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -330,7 +332,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -349,39 +351,39 @@ spec: deletionPolicy: WipeOut ``` -Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Update the `storage.resources.requests.storage` to `2Gi`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `Elasticsearch` database volume. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the volume changes and create a `VolumeExpansion` ElasticsearchOpsRequest to update the `MongoDB` database volume. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 23m +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 23m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 23m +MongoDB.gitops.kubedb.com/MongoDB-prod 23m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-horizontalscaling-j0wni6 HorizontalScaling Successful 13m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 19m ``` After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, ```bash -$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=Elasticsearch-prod' +$ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=MongoDB-prod' NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE -Elasticsearch-prod-data-Elasticsearch-prod-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m -Elasticsearch-prod-data-Elasticsearch-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m +MongoDB-prod-data-mg-gitops-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m +MongoDB-prod-data-mg-gitops-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m +MongoDB-prod-data-MongoDB-prod-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m +MongoDB-prod-data-MongoDB-prod-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m ``` -## Reconfigure Elasticsearch +## Reconfigure MongoDB At first, we will create a secret containing `user.conf` file with required configuration settings. -To know more about this configuration file, check [here](/docs/guides/Elasticsearch/configuration/Elasticsearch-combined.md) +To know more about this configuration file, check [here](/docs/guides/MongoDB/configuration/MongoDB-combined.md) ```yaml apiVersion: v1 kind: Secret @@ -399,16 +401,16 @@ Now, we will add this file to `kubedb /kf-configuration.yaml`. $ tree . ├── kubedb │ ├── kf-configuration.yaml -│ └── Elasticsearch.yaml +│ └── MongoDB.yaml 1 directories, 2 files ``` -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: configSecret: @@ -419,7 +421,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -438,7 +440,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -457,21 +459,21 @@ spec: deletionPolicy: WipeOut ``` -Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `Elasticsearch` database configuration. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` ElasticsearchOpsRequest to update the `MongoDB` database configuration. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 74m +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 74m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 74m +MongoDB.gitops.kubedb.com/MongoDB-prod 74m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 24m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfigure-ukj41o Reconfigure Successful 24m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 70m ``` @@ -479,7 +481,7 @@ Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr > We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. -### Rotate Elasticsearch Auth +### Rotate MongoDB Auth To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. @@ -488,20 +490,20 @@ We will do that using gitops, create the file `kubedb /kf-auth.yaml` with the fo ```bash kubectl create secret generic kf-rotate-auth -n demo \ --type=kubernetes.io/basic-auth \ ---from-literal=username=Elasticsearch \ ---from-literal=password=Elasticsearch-secret +--from-literal=username=MongoDB \ +--from-literal=password=MongoDB-secret secret/kf-rotate-auth created ``` -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: authSecret: @@ -515,7 +517,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -534,7 +536,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -553,22 +555,22 @@ spec: deletionPolicy: WipeOut ``` -Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Change the `authSecret` field to `kf-rotate-auth`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `Elasticsearch` database auth. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the auth changes and create a `RotateAuth` ElasticsearchOpsRequest to update the `MongoDB` database auth. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 7m11s +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 7m11s NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 7m11s +MongoDB.gitops.kubedb.com/MongoDB-prod 7m11s NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 17h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 28m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfigure-ukj41o Reconfigure Successful 17h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-rotate-auth-43ris8 RotateAuth Successful 28m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 17h ``` @@ -577,7 +579,7 @@ Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr We can add, rotate or remove TLS configuration using `gitops`. -To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in Elasticsearch. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. +To add tls, we are going to create an example `Issuer` that will be used to enable SSL/TLS in MongoDB. Alternatively, you can follow this [cert-manager tutorial](https://cert-manager.io/docs/configuration/ca/) to create your own `Issuer`. - Start off by generating a ca certificates using openssl. @@ -593,14 +595,14 @@ writing new private key to './ca.key' - Now we are going to create a ca-secret using the certificate files that we have just generated. ```bash -$ kubectl create secret tls Elasticsearch-ca \ +$ kubectl create secret tls MongoDB-ca \ --cert=ca.crt \ --key=ca.key \ --namespace=demo -secret/Elasticsearch-ca created +secret/MongoDB-ca created ``` -Now, Let's create an `Issuer` using the `Elasticsearch-ca` secret that we have just created. The `YAML` file looks like this: +Now, Let's create an `Issuer` using the `MongoDB-ca` secret that we have just created. The `YAML` file looks like this: ```yaml apiVersion: cert-manager.io/v1 @@ -610,7 +612,7 @@ metadata: namespace: demo spec: ca: - secretName: Elasticsearch-ca + secretName: MongoDB-ca ``` Let's add that to our `kubedb /kf-issuer.yaml` file. File structure will look like this, @@ -619,16 +621,16 @@ $ tree . ├── kubedb │ ├── kf-configuration.yaml │ ├── kf-issuer.yaml -│ └── Elasticsearch.yaml +│ └── MongoDB.yaml 1 directories, 4 files ``` -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 3.9.0 @@ -637,7 +639,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -656,7 +658,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -675,41 +677,41 @@ spec: deletionPolicy: WipeOut ``` -Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Add `sslMode` and `tls` fields in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `Elasticsearch` database tls. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the tls changes and create a `ReconfigureTLS` ElasticsearchOpsRequest to update the `MongoDB` database tls. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops,pods -n demo +$ kubectl get kf,MongoDB,kfops,pods -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 3.9.0 Ready 41m +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 3.9.0 Ready 41m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 75m +MongoDB.gitops.kubedb.com/MongoDB-prod 75m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d18h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfigure-ukj41o Reconfigure Successful 5d18h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 9m18s +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d1h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d19h ``` -> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for Elasticsearch. +> We can also rotate the certificates updating `.spec.tls.certificates` field. Also you can remove the `.spec.tls` field to remove tls for MongoDB. ### Update Version -List Elasticsearch versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/Elasticsearch/update-version/update-version.md). +List MongoDB versions using `kubectl get Elasticsearchversion` and choose desired version that is compatible for upgrade from current version. Check the version constraints and ops request [here](/docs/guides/MongoDB/update-version/update-version.md). Let's choose `4.0.0` in this example. -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 4.0.0 @@ -718,7 +720,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -737,7 +739,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -756,50 +758,50 @@ spec: deletionPolicy: WipeOut ``` -Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Update the `version` field to `17.4`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `Elasticsearch` database version. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the version changes and create a `VersionUpdate` ElasticsearchOpsRequest to update the `MongoDB` database version. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get kf,Elasticsearch,kfops -n demo +$ kubectl get kf,MongoDB,kfops -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 3h47m +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 4.0.0 Ready 3h47m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 3h47m +MongoDB.gitops.kubedb.com/MongoDB-prod 3h47m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 5d22h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfigure-ukj41o Reconfigure Successful 5d22h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d6h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-versionupdate-wyn2dp UpdateVersion Successful 3h51m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 5d23h ``` -Now, we are going to verify whether the `Elasticsearch`, `PetSet` and it's `Pod` have updated with new image. Let's check, +Now, we are going to verify whether the `MongoDB`, `PetSet` and it's `Pod` have updated with new image. Let's check, ```bash -$ kubectl get Elasticsearch -n demo Elasticsearch-prod -o=jsonpath='{.spec.version}{"\n"}' +$ kubectl get MongoDB -n demo MongoDB-prod -o=jsonpath='{.spec.version}{"\n"}' 4.0.0 -$ kubectl get petset -n demo Elasticsearch-prod-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get petset -n demo mg-gitops -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MongoDB:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 -$ kubectl get pod -n demo Elasticsearch-prod-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Elasticsearch:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 +$ kubectl get pod -n demo mg-gitops-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' +ghcr.io/appscode-images/MongoDB:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 ``` ### Enable Monitoring If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server). -Update the `Elasticsearch.yaml` with the following, +Update the `MongoDB.yaml` with the following, ```yaml apiVersion: gitops.kubedb.com/v1alpha1 -kind: Elasticsearch +kind: MongoDB metadata: - name: Elasticsearch-prod + name: MongoDB-prod namespace: demo spec: version: 4.0.0 @@ -808,7 +810,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -827,7 +829,7 @@ spec: podTemplate: spec: containers: - - name: Elasticsearch + - name: MongoDB resources: limits: memory: 1536Mi @@ -855,24 +857,24 @@ spec: deletionPolicy: WipeOut ``` -Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Elasticsearch` CR is updated in your cluster. +Add `monitor` field in the spec. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `MongoDB` CR is updated in your cluster. -Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `Elasticsearch` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. +Now, `gitops` operator will detect the monitoring changes and create a `Restart` ElasticsearchOpsRequest to add the `MongoDB` database monitoring. List the resources created by `gitops` operator in the `demo` namespace. ```bash $ kubectl get Elasticsearches.gitops.kubedb.com,Elasticsearches.kubedb.com,Elasticsearchopsrequest -n demo NAME TYPE VERSION STATUS AGE -Elasticsearch.kubedb.com/Elasticsearch-prod kubedb.com/v1 4.0.0 Ready 5h12m +MongoDB.kubedb.com/MongoDB-prod kubedb.com/v1 4.0.0 Ready 5h12m NAME AGE -Elasticsearch.gitops.kubedb.com/Elasticsearch-prod 5h12m +MongoDB.gitops.kubedb.com/MongoDB-prod 5h12m NAME TYPE STATUS AGE -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfigure-ukj41o Reconfigure Successful 6d -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-restart-ljpqih Restart Successful 3m51s -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m -Elasticsearchopsrequest.ops.kubedb.com/Elasticsearch-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfigure-ukj41o Reconfigure Successful 6d +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-reconfiguretls-r4mx7v ReconfigureTLS Successful 5h42m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-restart-ljpqih Restart Successful 3m51s +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-rotate-auth-43ris8 RotateAuth Successful 5d7h +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-versionupdate-wyn2dp UpdateVersion Successful 5h16m +Elasticsearchopsrequest.ops.kubedb.com/MongoDB-prod-volumeexpansion-41xthr VolumeExpansion Successful 6d ``` @@ -891,10 +893,10 @@ There are some other fields that will trigger `Restart` ops request. ## Next Steps -[//]: # (- Learn Elasticsearch [GitOps](/docs/guides/Elasticsearch/concepts/Elasticsearch-gitops.md)) -- Learn Elasticsearch Scaling - - [Horizontal Scaling](/docs/guides/Elasticsearch/scaling/horizontal-scaling/combined.md) - - [Vertical Scaling](/docs/guides/Elasticsearch/scaling/vertical-scaling/combined.md) -- Learn Version Update Ops Request and Constraints [here](/docs/guides/Elasticsearch/update-version/overview.md) -- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/Elasticsearch/monitoring/using-builtin-prometheus.md). +[//]: # (- Learn MongoDB [GitOps](/docs/guides/MongoDB/concepts/MongoDB-gitops.md)) +- Learn MongoDB Scaling + - [Horizontal Scaling](/docs/guides/MongoDB/scaling/horizontal-scaling/combined.md) + - [Vertical Scaling](/docs/guides/MongoDB/scaling/vertical-scaling/combined.md) +- Learn Version Update Ops Request and Constraints [here](/docs/guides/MongoDB/update-version/overview.md) +- Monitor your ElasticsearchQL database with KubeDB using [built-in Prometheus](/docs/guides/MongoDB/monitoring/using-builtin-prometheus.md). - Want to hack on KubeDB? Check our [contribution guidelines](/docs/CONTRIBUTING.md). diff --git a/docs/guides/mongodb/gitops/overview.md b/docs/guides/mongodb/gitops/overview.md index bfa01c2ff..10a0e72b6 100644 --- a/docs/guides/mongodb/gitops/overview.md +++ b/docs/guides/mongodb/gitops/overview.md @@ -1,6 +1,6 @@ --- -title: Elasticsearch Gitops Overview -description: Elasticsearch Gitops Overview +title: MongoDB Gitops Overview +description: MongoDB Gitops Overview menu: docs_{{ .version }}: identifier: es-gitops-overview @@ -13,20 +13,20 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Overview for Elasticsearch +# GitOps Overview for MongoDB -This guide will give you an overview of how KubeDB `gitops` operator works with Elasticsearch databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for -managing Elasticsearch databases in Kubernetes. +This guide will give you an overview of how KubeDB `gitops` operator works with MongoDB databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing MongoDB databases in Kubernetes. ## Before You Begin - You should be familiar with the following `KubeDB` concepts: - - [Elasticsearch](/docs/guides/elasticsearch/concepts/elasticsearch/index.md) - - [ElasticsearchOpsRequest](/docs/guides/elasticsearch/concepts/elasticsearch-ops-request/index.md) + - [MongoDB](/docs/guides/MongoDB/concepts/MongoDB/index.md) + - [ElasticsearchOpsRequest](/docs/guides/MongoDB/concepts/MongoDB-ops-request/index.md) -## Workflow GitOps with Elasticsearch +## Workflow GitOps with MongoDB The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. @@ -35,16 +35,16 @@ The following diagram shows how the `KubeDB` GitOps Operator used to sync with y GitOps Flow -
Fig: GitOps process of Elasticsearch
+
Fig: GitOps process of MongoDB
-1. **Define GitOps Elasticsearch**: Create Custom Resource (CR) of kind `Elasticsearch` using the `gitops.kubedb.com/v1alpha1` API. +1. **Define GitOps MongoDB**: Create Custom Resource (CR) of kind `MongoDB` using the `gitops.kubedb.com/v1alpha1` API. 2. **Store in Git**: Push the CR to a Git repository. 3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. -4. **Create Database**: The GitOps operator creates a corresponding KubeDB Elasticsearch CR in the Kubernetes cluster to deploy the database. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB MongoDB CR in the Kubernetes cluster to deploy the database. 5. **Handle Updates**: When you update the ElasticsearchGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. -This flow makes managing Elasticsearch databases efficient, reliable, and fully integrated with GitOps practices. +This flow makes managing MongoDB databases efficient, reliable, and fully integrated with GitOps practices. -In the next doc, we are going to show a step by step guide on running Elasticsearch using GitOps. +In the next doc, we are going to show a step by step guide on running MongoDB using GitOps. From 732b7330da065fda4b7c03d85ad34065fc673185 Mon Sep 17 00:00:00 2001 From: Bonusree Date: Wed, 1 Apr 2026 11:24:48 +0600 Subject: [PATCH 13/14] rd Signed-off-by: Bonusree --- docs/guides/redis/gitops/_index.md | 8 +- docs/guides/redis/gitops/gitops.md | 236 ++++++++++++++------------- docs/guides/redis/gitops/overview.md | 30 ++-- 3 files changed, 139 insertions(+), 135 deletions(-) diff --git a/docs/guides/redis/gitops/_index.md b/docs/guides/redis/gitops/_index.md index 098093afa..118975369 100644 --- a/docs/guides/redis/gitops/_index.md +++ b/docs/guides/redis/gitops/_index.md @@ -1,10 +1,10 @@ --- -title: Gitops Kafka +title: Redis Gitops menu: docs_{{ .version }}: - identifier: kf-gitops + identifier: rd-gitops name: Gitops - parent: kf-kafka-guides - weight: 30 + parent: rd-redis-guides + weight: 170 menu_name: docs_{{ .version }} --- diff --git a/docs/guides/redis/gitops/gitops.md b/docs/guides/redis/gitops/gitops.md index 1eafdcc0e..d0d99726d 100644 --- a/docs/guides/redis/gitops/gitops.md +++ b/docs/guides/redis/gitops/gitops.md @@ -4,7 +4,7 @@ menu: docs_{{ .version }}: identifier: rd-gitops-guides name: Gitops Redis - parent: rd-gitops-Redis + parent: rd-gitops weight: 20 menu_name: docs_{{ .version }} section_menu_id: guides @@ -80,11 +80,14 @@ spec: podTemplate: spec: containers: - - name: redis - resources: - requests: - cpu: "100m" - memory: "100Mi" + - name: redis + resources: + requests: + memory: "300Mi" + cpu: "200m" + limits: + memory: "300Mi" + cpu: "200m" deletionPolicy: WipeOut ``` @@ -104,44 +107,43 @@ Our `gitops` operator will create an actual `Redis` database CR in the cluster. ```bash $ kubectl get redis.gitops.kubedb.com,redis.kubedb.com -n demo NAME AGE -redis.gitops.kubedb.com/rd-gitops 8m46s +redis.gitops.kubedb.com/rd-gitops 2m19s NAME VERSION STATUS AGE -redis.kubedb.com/rd-gitops 8.0.4 Ready 8m46s - +redis.kubedb.com/rd-gitops 8.0.4 Ready 2m18s ``` List the resources created by `kubedb` operator created for `kubedb.com/v1` Redis. ```bash -$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=rd-gitops' +$ kubectl get petset,pod,secret,service,appbinding -n demo -l 'app.kubernetes.io/instance=rd-gitops' NAME AGE -petset.apps.k8s.appscode.com/rd-gitops-shard0 9m38s -petset.apps.k8s.appscode.com/rd-gitops-shard1 9m36s -petset.apps.k8s.appscode.com/rd-gitops-shard2 9m34s +petset.apps.k8s.appscode.com/rd-gitops-shard0 5m53s +petset.apps.k8s.appscode.com/rd-gitops-shard1 5m51s +petset.apps.k8s.appscode.com/rd-gitops-shard2 5m49s NAME READY STATUS RESTARTS AGE -pod/rd-gitops-shard0-0 1/1 Running 0 9m37s -pod/rd-gitops-shard0-1 1/1 Running 0 9m14s -pod/rd-gitops-shard1-0 1/1 Running 0 9m35s -pod/rd-gitops-shard1-1 1/1 Running 0 9m14s -pod/rd-gitops-shard2-0 1/1 Running 0 9m34s -pod/rd-gitops-shard2-1 1/1 Running 0 9m14s +pod/rd-gitops-shard0-0 1/1 Running 0 5m52s +pod/rd-gitops-shard0-1 1/1 Running 0 5m35s +pod/rd-gitops-shard1-0 1/1 Running 0 5m50s +pod/rd-gitops-shard1-1 1/1 Running 0 5m31s +pod/rd-gitops-shard2-0 1/1 Running 0 5m49s +pod/rd-gitops-shard2-1 1/1 Running 0 5m31s NAME TYPE DATA AGE -secret/rd-gitops-9a6ce0 Opaque 1 9m40s -secret/rd-gitops-auth kubernetes.io/basic-auth 2 65m +secret/rd-gitops-auth kubernetes.io/basic-auth 2 5m55s +secret/rd-gitops-b3b686 Opaque 1 5m55s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -service/rd-gitops ClusterIP 10.43.193.228 6379/TCP 9m40s -service/rd-gitops-pods ClusterIP None 6379/TCP,16379/TCP 9m40s +service/rd-gitops ClusterIP 10.43.156.126 6379/TCP 5m55s +service/rd-gitops-pods ClusterIP None 6379/TCP,16379/TCP 5m55s NAME TYPE VERSION AGE -appbinding.appcatalog.appscode.com/rd-gitops kubedb.com/redis 8.0.4 9m34s - +appbinding.appcatalog.appscode.com/rd-gitops kubedb.com/redis 8.0.4 5m49s ``` ## Update Redis Database using GitOps + ### Scale Redis Replicas @@ -156,7 +158,7 @@ spec: version: 8.0.4 mode: Cluster cluster: - shards: 4 + shards: 3 replicas: 3 storageType: Durable storage: @@ -169,11 +171,14 @@ spec: podTemplate: spec: containers: - - name: redis - resources: - requests: - cpu: "100m" - memory: "100Mi" + - name: redis + resources: + requests: + memory: "300Mi" + cpu: "200m" + limits: + memory: "300Mi" + cpu: "200m" deletionPolicy: WipeOut ``` Update the `replicas` to `3`. Commit the changes and push to your Git repository. Your repository is synced with `ArgoCD` and the `Redis` CR is updated in your cluster. @@ -182,33 +187,28 @@ Now, `gitops` operator will detect the replica changes and create a `HorizontalS ```bash $ kubectl get rd,redis,redisopsrequest -n demo NAME VERSION STATUS AGE -redis.kubedb.com/rd-gitops 8.0.4 Ready 12m +redis.kubedb.com/rd-gitops 8.0.4 Ready 55m NAME AGE -redis.gitops.kubedb.com/rd-gitops 12m +redis.gitops.kubedb.com/rd-gitops 55m NAME TYPE STATUS AGE -redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-1nblpl HorizontalScaling Successful 10m - +redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-ule15j HorizontalScaling Successful 16m ``` After Ops Request becomes `Successful`, We can validate the changes by checking the number of pods, ```bash -$ kubectl get pod -n demo -l 'app.kubernetes.io/instance=rd-gitops' +$ kkubectl get pod -n demo -l 'app.kubernetes.io/instance=rd-gitops' NAME READY STATUS RESTARTS AGE -rd-gitops-shard0-0 1/1 Running 0 16m -rd-gitops-shard0-1 1/1 Running 0 16m -rd-gitops-shard0-2 1/1 Running 0 13m -rd-gitops-shard1-0 1/1 Running 0 16m -rd-gitops-shard1-1 1/1 Running 0 16m -rd-gitops-shard1-2 1/1 Running 0 12m -rd-gitops-shard2-0 1/1 Running 0 16m -rd-gitops-shard2-1 1/1 Running 0 16m -rd-gitops-shard2-2 1/1 Running 0 12m -rd-gitops-shard3-0 1/1 Running 0 14m -rd-gitops-shard3-1 1/1 Running 0 14m -rd-gitops-shard3-2 1/1 Running 0 11m - +rd-gitops-shard0-0 1/1 Running 0 55m +rd-gitops-shard0-1 1/1 Running 0 55m +rd-gitops-shard0-2 1/1 Running 0 16m +rd-gitops-shard1-0 1/1 Running 0 55m +rd-gitops-shard1-1 1/1 Running 0 55m +rd-gitops-shard1-2 1/1 Running 0 16m +rd-gitops-shard2-0 1/1 Running 0 55m +rd-gitops-shard2-1 1/1 Running 0 55m +rd-gitops-shard2-2 1/1 Running 0 15m ``` We can also scale down the replicas by updating the `replicas` fields. @@ -219,12 +219,15 @@ before Ops Request becomes `Successful`, We can validate the changes by checking ```bash $ kubectl get pod -n demo rd-gitops-shard0-0 -o json | jq '.spec.containers[0].resources' { + "limits": { + "cpu": "200m", + "memory": "300Mi" + }, "requests": { - "cpu": "100m", - "memory": "100Mi" + "cpu": "200m", + "memory": "300Mi" } } - ``` @@ -239,7 +242,7 @@ spec: version: 8.0.4 mode: Cluster cluster: - shards: 4 + shards: 3 replicas: 3 storageType: Durable storage: @@ -252,12 +255,16 @@ spec: podTemplate: spec: containers: - - name: redis - resources: - requests: - cpu: "200m" - memory: "200Mi" + - name: redis + resources: + requests: + memory: "500Mi" + cpu: "500m" + limits: + memory: "500Mi" + cpu: "500m" deletionPolicy: WipeOut + ``` @@ -268,17 +275,27 @@ Now, `gitops` operator will detect the resource changes and create a `RedisOpsRe ```bash $ kubectl get rd,redis,redisopsrequest -n demo NAME VERSION STATUS AGE -redis.kubedb.com/rd-gitops 8.0.4 Ready 28m +redis.kubedb.com/rd-gitops 8.0.4 Ready 17h NAME AGE -redis.gitops.kubedb.com/rd-gitops 28m +redis.gitops.kubedb.com/rd-gitops 17h NAME TYPE STATUS AGE -redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-1nblpl HorizontalScaling Successful 26m -redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-or3shk VerticalScaling Successful 10m - +redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-ule15j HorizontalScaling Successful 16h +redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-lliwo8 VerticalScaling Successful 16h ``` +kubectl get pod -n demo rd-gitops-shard0-0 -o json | jq '.spec.containers[0].resources' +{ + "limits": { + "cpu": "500m", + "memory": "500Mi" + }, + "requests": { + "cpu": "500m", + "memory": "500Mi" + } +} ### Expand Redis Volume @@ -291,47 +308,30 @@ metadata: name: rd-gitops namespace: demo spec: - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce + version: 8.0.4 + mode: Cluster + cluster: + shards: 3 + replicas: 3 + storageType: Durable + storage: + resources: + requests: + storage: "2Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis resources: requests: - storage: 2Gi - storageClassName: Standard - storageType: Durable + memory: "500Mi" + cpu: "500m" + limits: + memory: "500Mi" + cpu: "500m" deletionPolicy: WipeOut ``` @@ -341,27 +341,31 @@ Now, `gitops` operator will detect the volume changes and create a `VolumeExpans ```bash $ kubectl get rd,redis,redisopsrequest -n demo -NAME TYPE VERSION STATUS AGE -Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 23m +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 17h -NAME AGE -Redis.gitops.kubedb.com/rd-gitops 23m +NAME AGE +redis.gitops.kubedb.com/rd-gitops 17h -NAME TYPE STATUS AGE -Redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-j0wni6 HorizontalScaling Successful 13m -Redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-tfkvi8 VerticalScaling Successful 8m29s -Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 19m +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-horizontalscaling-ule15j HorizontalScaling Successful 16h +redisopsrequest.ops.kubedb.com/rd-gitops-verticalscaling-lliwo8 VerticalScaling Successful 16h +redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-3twy3x VolumeExpansion Successful 2m23s ``` After Ops Request becomes `Successful`, We can validate the changes by checking the pvc size, ```bash $ kubectl get pvc -n demo -l 'app.kubernetes.io/instance=rd-gitops' -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE -rd-gitops-data-rd-gitops-broker-0 Bound pvc-2afd4835-5686-492b-be93-c6e040e0a6c6 2Gi RWO Standard 3h39m -rd-gitops-data-rd-gitops-broker-1 Bound pvc-aaf994cc-6b04-4c37-80d5-5e966dad8487 2Gi RWO Standard 3h39m -rd-gitops-data-rd-gitops-controller-0 Bound pvc-82d2b233-203d-4df2-a0fd-ecedbc0825b7 2Gi RWO Standard 3h39m -rd-gitops-data-rd-gitops-controller-1 Bound pvc-91852c29-ab1a-48ad-9255-a0b15d5a7515 2Gi RWO Standard 3h39m - +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE +data-rd-gitops-shard0-0 Bound pvc-77634b30-7ccc-4391-a768-54d4e3c1907c 2Gi RWO longhorn 17h +data-rd-gitops-shard0-1 Bound pvc-e4b23a61-5de7-4e5c-b4b2-0722bffc7464 2Gi RWO longhorn 17h +data-rd-gitops-shard0-2 Bound pvc-c3ffc095-248b-4091-bfaa-a1de401e3e36 2Gi RWO longhorn 16h +data-rd-gitops-shard1-0 Bound pvc-62e7a6c0-ddd5-4281-8878-9ec42e68585d 2Gi RWO longhorn 17h +data-rd-gitops-shard1-1 Bound pvc-0e9ca41e-8b25-4ea1-865a-17aba48f619c 2Gi RWO longhorn 17h +data-rd-gitops-shard1-2 Bound pvc-c1e6806d-bb6c-4521-94c6-8e47ec6ad36e 2Gi RWO longhorn 16h +data-rd-gitops-shard2-0 Bound pvc-0954e736-e699-46f4-9dfb-33b23f65e351 2Gi RWO longhorn 17h +data-rd-gitops-shard2-1 Bound pvc-1fd1321a-ff04-4c81-b903-80473598ffac 2Gi RWO longhorn 17h +data-rd-gitops-shard2-2 Bound pvc-8458eb68-930b-4817-8577-24616079e9f7 2Gi RWO longhorn 16h ``` ## Reconfigure Redis diff --git a/docs/guides/redis/gitops/overview.md b/docs/guides/redis/gitops/overview.md index cf959a479..00705eec8 100644 --- a/docs/guides/redis/gitops/overview.md +++ b/docs/guides/redis/gitops/overview.md @@ -1,11 +1,11 @@ --- -title: Kafka Gitops Overview -description: Kafka Gitops Overview +title: Redis Gitops Overview +description: Redis Gitops Overview menu: docs_{{ .version }}: - identifier: kf-gitops-overview + identifier: rd-gitops-overview name: Overview - parent: kf-gitops + parent: rd-gitops weight: 10 menu_name: docs_{{ .version }} section_menu_id: guides @@ -13,20 +13,20 @@ section_menu_id: guides > New to KubeDB? Please start [here](/docs/README.md). -# GitOps Overview for Kafka +# GitOps Overview for Redis -This guide will give you an overview of how KubeDB `gitops` operator works with Kafka databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for -managing Kafka databases in Kubernetes. +This guide will give you an overview of how KubeDB `gitops` operator works with Redis databases using the `gitops.kubedb.com/v1alpha1` API. It will help you understand the GitOps workflow for +managing Redis databases in Kubernetes. ## Before You Begin - You should be familiar with the following `KubeDB` concepts: - - [Kafka](/docs/guides/kafka/concepts/kafka.md) - - [KafkaOpsRequest](/docs/guides/kafka/concepts/kafkaopsrequest.md) + - [Redis](/docs/guides/redis/concepts/kafka.md) + - [KafkaOpsRequest](/docs/guides/redis/concepts/kafkaopsrequest.md) -## Workflow GitOps with Kafka +## Workflow GitOps with Redis The following diagram shows how the `KubeDB` GitOps Operator used to sync with your database. Open the image in a new tab to see the enlarged version. @@ -35,16 +35,16 @@ The following diagram shows how the `KubeDB` GitOps Operator used to sync with y GitOps Flow -
Fig: GitOps process of Kafka
+
Fig: GitOps process of Redis
-1. **Define GitOps Kafka**: Create Custom Resource (CR) of kind `Kafka` using the `gitops.kubedb.com/v1alpha1` API. +1. **Define GitOps Redis**: Create Custom Resource (CR) of kind `Redis` using the `gitops.kubedb.com/v1alpha1` API. 2. **Store in Git**: Push the CR to a Git repository. 3. **Automated Deployment**: Use a GitOps tool (like `ArgoCD` or `FluxCD`) to monitor the Git repository and synchronize the state of the Kubernetes cluster with the desired state defined in Git. -4. **Create Database**: The GitOps operator creates a corresponding KubeDB Kafka CR in the Kubernetes cluster to deploy the database. +4. **Create Database**: The GitOps operator creates a corresponding KubeDB Redis CR in the Kubernetes cluster to deploy the database. 5. **Handle Updates**: When you update the KafkaGitOps CR, the operator generates an Ops Request to safely apply the update(e.g. `VerticalScaling`, `HorizontalScaling`, `VolumeExapnsion`, `Reconfigure`, `RotateAuth`, `ReconfigureTLS`, `VersionUpdate`, ans `Restart`. -This flow makes managing Kafka databases efficient, reliable, and fully integrated with GitOps practices. +This flow makes managing Redis databases efficient, reliable, and fully integrated with GitOps practices. -In the next doc, we are going to show a step by step guide on running Kafka using GitOps. +In the next doc, we are going to show a step by step guide on running Redis using GitOps. From d821d7060ed8083400461c5bd47d969e70dbbaeb Mon Sep 17 00:00:00 2001 From: Bonusree Date: Wed, 1 Apr 2026 17:08:30 +0600 Subject: [PATCH 14/14] rd Signed-off-by: Bonusree --- docs/guides/redis/gitops/gitops.md | 289 +++++++++++------------------ 1 file changed, 110 insertions(+), 179 deletions(-) diff --git a/docs/guides/redis/gitops/gitops.md b/docs/guides/redis/gitops/gitops.md index d0d99726d..fbe9eda4e 100644 --- a/docs/guides/redis/gitops/gitops.md +++ b/docs/guides/redis/gitops/gitops.md @@ -374,13 +374,14 @@ At first, we will create a secret containing `user.conf` file with required conf To know more about this configuration file, check [here](/docs/guides/Redis/configuration/Redis-combined.md) ```yaml apiVersion: v1 +stringData: + redis.conf: | + maxclients 500 kind: Secret metadata: - name: new-rd-combined-custom-config + name: rd-new-configuration namespace: demo -stringData: - server.properties: |- - log.retention.hours=125 +type: Opaque ``` Now, we will add this file to `kubedb /rd-configuration.yaml`. @@ -401,49 +402,32 @@ metadata: name: rd-gitops namespace: demo spec: - configSecret: - name: new-rd-combined-custom-config - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce + version: 8.0.4 + mode: Cluster + cluster: + shards: 3 + replicas: 3 + storageType: Durable + configuration: + secretName: rd-new-configuration + storage: + resources: + requests: + storage: "2Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis resources: requests: - storage: 2Gi - storageClassName: Standard - storageType: Durable + memory: "500Mi" + cpu: "500m" + limits: + memory: "500Mi" + cpu: "500m" deletionPolicy: WipeOut ``` @@ -452,23 +436,19 @@ Commit the changes and push to your Git repository. Your repository is synced wi Now, `gitops` operator will detect the configuration changes and create a `Reconfigure` RedisOpsRequest to update the `Redis` database configuration. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get rd,redis,redisopsrequest -n demo -NAME TYPE VERSION STATUS AGE -Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 74m - -NAME AGE -Redis.gitops.kubedb.com/rd-gitops 74m +$ kubectl get rd,redis,redisopsrequest -n demo +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 22m -NAME TYPE STATUS AGE -Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 24m -Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 70m +NAME AGE +redis.gitops.kubedb.com/rd-gitops 22m +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-5v9956 Reconfigure Successful 15m ``` -> We can also reconfigure the parameters creating another secret and reference the secret in the `configSecret` field. Also you can remove the `configSecret` field to use the default parameters. - ### Rotate Redis Auth To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new username and password. @@ -476,12 +456,11 @@ To do that, create a `kubernetes.io/basic-auth` type k8s secret with the new use We will create a secret named `rd-rotate-auth ` with the following content, ```bash -kubectl create secret generic rd-rotate-auth -n demo \ ---type=kubernetes.io/basic-auth \ ---from-literal=username=Redis \ ---from-literal=password=Redis-secret +$ kubectl create secret generic rd-rotate-auth -n demo \ + --type=kubernetes.io/basic-auth \ + --from-literal=username=redis \ + --from-literal=password=redis-secret secret/rd-rotate-auth created - ``` @@ -494,52 +473,35 @@ metadata: name: rd-gitops namespace: demo spec: + version: 8.0.4 + mode: Cluster + cluster: + shards: 3 + replicas: 3 + storageType: Durable authSecret: kind: Secret name: rd-rotate-auth configuration: - secretName: new-rd-combined-custom-config - version: 3.9.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce + secretName: rd-new-configuration + storage: + resources: + requests: + storage: "2Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis resources: requests: - storage: 2Gi - storageClassName: Standard - storageType: Durable + memory: "500Mi" + cpu: "500m" + limits: + memory: "500Mi" + cpu: "500m" deletionPolicy: WipeOut ``` @@ -548,18 +510,16 @@ Change the `authSecret` field to `rd-rotate-auth`. Commit the changes and push t Now, `gitops` operator will detect the auth changes and create a `RotateAuth` RedisOpsRequest to update the `Redis` database auth. List the resources created by `gitops` operator in the `demo` namespace. ```bash -$ kubectl get rd,redis,redisopsrequest -n demo -NAME TYPE VERSION STATUS AGE -Redis.kubedb.com/rd-gitops kubedb.com/v1 3.9.0 Ready 7m11s - -NAME AGE -Redis.gitops.kubedb.com/rd-gitops 7m11s +$ kubectl get rd,redis,redisopsrequest -n demo +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 33m -NAME TYPE STATUS AGE -Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 17h -Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 28m -Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 17h +NAME AGE +redis.gitops.kubedb.com/rd-gitops 33m +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-5v9956 Reconfigure Successful 26m +redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-oz83g9 RotateAuth Successful 7m39s ``` @@ -583,11 +543,11 @@ writing new private key to './ca.key' - Now we are going to create a ca-secret using the certificate files that we have just generated. ```bash -$ kubectl create secret tls Redis-ca \ - --cert=ca.crt \ - --key=ca.key \ - --namespace=demo -secret/Redis-ca created +$ kubectl create secret tls redis-ca \ + --cert=ca.crt \ + --key=ca.key \ + --namespace=demo +secret/redis-ca created ``` Now, Let's create an `Issuer` using the `Redis-ca` secret that we have just created. The `YAML` file looks like this: @@ -600,7 +560,7 @@ metadata: namespace: demo spec: ca: - secretName: Redis-ca + secretName: redis-ca ``` Let's add that to our `kubedb /rd-issuer.yaml` file. File structure will look like this, @@ -702,47 +662,35 @@ metadata: name: rd-gitops namespace: demo spec: - version: 4.0.0 - topology: - broker: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 2Gi - storageClassName: Standard - controller: - podTemplate: - spec: - containers: - - name: Redis - resources: - limits: - memory: 1536Mi - requests: - cpu: 500m - memory: 1536Mi - replicas: 2 - storage: - accessModes: - - ReadWriteOnce + version: 8.0.4 + mode: Cluster + cluster: + shards: 3 + replicas: 3 + storageType: Durable + authSecret: + kind: Secret + name: rd-rotate-auth + configuration: + secretName: rd-new-configuration + storage: + resources: + requests: + storage: "2Gi" + storageClassName: "longhorn" + accessModes: + - ReadWriteOnce + podTemplate: + spec: + containers: + - name: redis resources: requests: - storage: 2Gi - storageClassName: Standard - storageType: Durable + memory: "500Mi" + cpu: "500m" + limits: + memory: "500Mi" + cpu: "500m" deletionPolicy: WipeOut ``` @@ -752,34 +700,17 @@ Now, `gitops` operator will detect the version changes and create a `VersionUpda ```bash $ kubectl get rd,redis,redisopsrequest -n demo -NAME TYPE VERSION STATUS AGE -Redis.kubedb.com/rd-gitops kubedb.com/v1 4.0.0 Ready 3h47m +NAME VERSION STATUS AGE +redis.kubedb.com/rd-gitops 8.0.4 Ready 14m -NAME AGE -Redis.gitops.kubedb.com/rd-gitops 3h47m +NAME AGE +redis.gitops.kubedb.com/rd-gitops 14m -NAME TYPE STATUS AGE -Redisopsrequest.ops.kubedb.com/rd-gitops-reconfigure-ukj41o Reconfigure Successful 5d22h -Redisopsrequest.ops.kubedb.com/rd-gitops-reconfiguretls-r4mx7v ReconfigureTLS Successful 4h16m -Redisopsrequest.ops.kubedb.com/rd-gitops-rotate-auth-43ris8 RotateAuth Successful 5d6h -Redisopsrequest.ops.kubedb.com/rd-gitops-versionupdate-wyn2dp UpdateVersion Successful 3h51m -Redisopsrequest.ops.kubedb.com/rd-gitops-volumeexpansion-41xthr VolumeExpansion Successful 5d23h +NAME TYPE STATUS AGE +redisopsrequest.ops.kubedb.com/rd-gitops-versionupdate-xvnekk UpdateVersion Successful 11m ``` -Now, we are going to verify whether the `Redis`, `PetSet` and it's `Pod` have updated with new image. Let's check, - -```bash -$ kubectl get Redis -n demo rd-gitops -o=jsonpath='{.spec.version}{"\n"}' -4.0.0 - -$ kubectl get petset -n demo rd-gitops-broker -o=jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Redis:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 - -$ kubectl get pod -n demo rd-gitops-broker-0 -o=jsonpath='{.spec.containers[0].image}{"\n"}' -ghcr.io/appscode-images/Redis:4.0.0@sha256:42a79fe8f14b00b1c76d135bbbaf7605b8c66f45cf3eb749c59138f6df288b31 -``` - ### Enable Monitoring If you already don't have a Prometheus server running, deploy one following tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/monitoring/prometheus/operator/README.md#deploy-prometheus-server).