You've already forked helm-gitea
[Breaking] Add HA-support; switch to Deployment (#437)
# Changes A big shoutout to @luhahn for all his work in #205 which served as the base for this PR. ## Documentation - [x] After thinking for some time about it, I still prefer the distinct option (as started in #350), i.e. having a standalone "HA" doc under `docs/ha-setup.md` to not have a very long README (which is already quite long). Most of the information below should go into it with more details and explanations behind all of the individual components. ## Chart deps ~~- Adds `meilisearch` as a chart dependency for a HA-ready issue indexer. Only works with >= Gitea 1.20~~ ~~- Adds `redis` as a chart dependency for a HA-ready session and queue store.~~ - Adds `redis-cluster` as a chart dependency for a HA-ready session and queue store (alternative to `redis`). Only works with >= Gitea 1.19.2. - Removes `memcached` instead of `redis-cluster` - Add `postgresql-ha` as default DB dep in favor of `postgres` ## Adds smart HA chart logic The goal is to set smart config values that result in a HA-ready Gitea deployment if `replicaCount` > 1. - If `replicaCount` > 1, - `gitea.config.session.PROVIDER` is automatically set to `redis-cluster` - `gitea.config.indexer.REPO_INDEXER_ENABLED` is automatically set to `false` unless the value is `elasticsearch` or `meilisearch` - `redis-cluster` is used for `[queue]` and `[cache]` and `[session]`mode or not Configuration of external instances of `meilisearch` and `minio` are documented in a new markdown doc. ## Deployment vs Statefulset Given all the discussions about this lately (#428), I think we could use both. In the end, we do not have the requirement for a sequential pod scale up/scale down as it would happen in statefulsets. On the other side, we do not have actual stateless pods as we are attaching a RWX to the deployment. Yet I think because we do not have a leader-election requirement, spawning the pods as a deployment makes "Rolling Updates" easier and also signals users that there is no "leader election" logic and each pod can just be "destroyed" at anytime without causing interruption. Hence I think we should be able to switch from a statefulset to a deployment, even in the single-replica case. This change also brought up a templating/linting issue: the definition of `.Values.gitea.config.server.SSH_LISTEN_PORT` in `ssh-svc.yaml` just "luckily" worked so far due to naming-related lint processing. Due to the change from "statefulset" to "deployment", the processing queue changed and caused a failure complaining about `config.server.SSH_LISTEN_PORT` not being defined yet. The only way I could see to fix this was to "properly" define the value in `values.yaml` instead of conditionally definining it in `helpers.tpl`. Maybe there's a better way? ## Chart PVC Creation I've adapted the automated PVC creation from another chart to be able to provide the `storageClassName` as I couldn't get dynamic provisioning for EFS going with the current implementation. In addition the naming and approach within the Gitea chart for PV creation is a bit unusual and aligning it might be beneficial. A semi-unrelated change which will result in a breaking change for existing users but this PR includes a lot of breaking changes already, so including another one might not make it much worse... - New `persistence.mount`: whether to mount an existing PVC (via `persistence.existingClaim` - New `persistence.create`: whether to create a new PVC ## Testing As this PR does a lot of things, we need proper testing. The helm chart can be installed from the Git branch via `helm-git` as follows: ``` helm repo add gitea-charts git+https://gitea.com/gitea/helm-chart@/?ref=deployment helm install gitea --version 0.0.0 ``` It is **highly recommended** to test the chart in a dedicated namespace. I've tested this myself with both `redis` and `redis-cluster` and it seemed to work fine. I just did some basic operations though and we should do more niche testing before merging. Examplary `values.yml` for testing (only needs a valid RWX storage class): <details> <summary>values.yaml</summary> ```yml image: tag: "dev" PullPolicy: "Always" rootless: true replicaCount: 2 persistence: enabled: true accessModes: - ReadWriteMany storageClass: FIXME redis-cluster: enabled: false global: redis: password: gitea gitea: config: indexer: ISSUE_INDEXER_ENABLED: true REPO_INDEXER_ENABLED: false ``` </details> ## Preferred setup The preferred HA setup with respect to performance and stability might currently be as follows: - Repos: RWX (e.g. EFS or Azurefiles NFS) - Issue indexer: Meilisearch (HA) - Session and cache: Redis Cluster (HA) - Attachments/Avatars: Minio (HA) This will result in a ~ 10-pod HA setup overall. All pods have very low resource requests. fix #98 Co-authored-by: pat-s <pat-s@noreply.gitea.io> Reviewed-on: https://gitea.com/gitea/helm-chart/pulls/437 Co-authored-by: pat-s <patrick.schratz@gmail.com> Co-committed-by: pat-s <patrick.schratz@gmail.com>
This commit is contained in:
pat-s
@ -2,6 +2,27 @@
|
||||
{{/*
|
||||
Expand the name of the chart.
|
||||
*/}}
|
||||
|
||||
{{- /* multiple replicas assertions */ -}}
|
||||
{{- if gt .Values.replicaCount 1.0 -}}
|
||||
{{- fail "When using multiple replicas, a RWX file system is required" -}}
|
||||
{{- if eq (get (.Values.persistence.accessModes 0) "ReadWriteOnce") -}}
|
||||
{{- fail "When using multiple replicas, a RWX file system is required" -}}
|
||||
{{- end }}
|
||||
|
||||
{{- if eq (get .Values.gitea.config.indexer "ISSUE_INDEXER_TYPE") "bleve" -}}
|
||||
{{- fail "When using multiple replicas, the repo indexer must be set to 'meilisearch' or 'elasticsearch'" -}}
|
||||
{{- end }}
|
||||
|
||||
{{- if and (eq .Values.gitea.config.indexer.REPO_INDEXER_TYPE "bleve") (eq .Values.gitea.config.indexer.REPO_INDEXER_ENABLED "true") -}}
|
||||
{{- fail "When using multiple replicas, the repo indexer must be set to 'meilisearch' or 'elasticsearch'" -}}
|
||||
{{- end }}
|
||||
|
||||
{{- if eq .Values.gitea.config.indexer.ISSUE_INDEXER_TYPE "bleve" -}}
|
||||
{{- (printf "DEBUG: When using multiple replicas, the repo indexer must be set to 'meilisearch' or 'elasticsearch'") | fail -}}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{- define "gitea.name" -}}
|
||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
|
||||
{{- end -}}
|
||||
@ -95,8 +116,22 @@ app.kubernetes.io/instance: {{ .Release.Name }}
|
||||
{{- printf "%s-postgresql.%s.svc.%s:%g" .Release.Name .Release.Namespace .Values.clusterDomain .Values.postgresql.global.postgresql.service.ports.postgresql -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "memcached.dns" -}}
|
||||
{{- printf "%s-memcached.%s.svc.%s:%g" .Release.Name .Release.Namespace .Values.clusterDomain .Values.memcached.service.ports.memcached | trunc 63 | trimSuffix "-" -}}
|
||||
{{- define "redis.dns" -}}
|
||||
{{- if (index .Values "redis-cluster").enabled -}}
|
||||
{{- printf "redis+cluster://:%s@%s-redis-cluster-headless.%s.svc.%s:%g/0?pool_size=100&idle_timeout=180s&" (index .Values "redis-cluster").global.redis.password .Release.Name .Release.Namespace .Values.clusterDomain (index .Values "redis-cluster").service.ports.redis -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "redis.port" -}}
|
||||
{{- if (index .Values "redis-cluster").enabled -}}
|
||||
{{ (index .Values "redis-cluster").service.ports.redis }}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "redis.servicename" -}}
|
||||
{{- if (index .Values "redis-cluster").enabled -}}
|
||||
{{- printf "%s-redis-cluster-headless.%s.svc.%s" .Release.Name .Release.Namespace .Values.clusterDomain -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "gitea.default_domain" -}}
|
||||
@ -182,6 +217,7 @@ https
|
||||
{{- else -}}
|
||||
{{- (printf "Key %s cannot be on top level of configuration" $key) | fail -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
@ -211,6 +247,18 @@ https
|
||||
{{- if not (hasKey .Values.gitea.config "oauth2") -}}
|
||||
{{- $_ := set .Values.gitea.config "oauth2" dict -}}
|
||||
{{- end -}}
|
||||
{{- if not (hasKey .Values.gitea.config "session") -}}
|
||||
{{- $_ := set .Values.gitea.config "session" dict -}}
|
||||
{{- end -}}
|
||||
{{- if not (hasKey .Values.gitea.config "queue") -}}
|
||||
{{- $_ := set .Values.gitea.config "queue" dict -}}
|
||||
{{- end -}}
|
||||
{{- if not (hasKey .Values.gitea.config "queue.issue_indexer") -}}
|
||||
{{- $_ := set .Values.gitea.config "queue.issue_indexer" dict -}}
|
||||
{{- end -}}
|
||||
{{- if not (hasKey .Values.gitea.config "indexer") -}}
|
||||
{{- $_ := set .Values.gitea.config "indexer" dict -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "gitea.inline_configuration.defaults" -}}
|
||||
@ -226,13 +274,30 @@ https
|
||||
{{- if not (hasKey .Values.gitea.config.metrics "ENABLED") -}}
|
||||
{{- $_ := set .Values.gitea.config.metrics "ENABLED" .Values.gitea.metrics.enabled -}}
|
||||
{{- end -}}
|
||||
{{- if .Values.memcached.enabled -}}
|
||||
{{- if (index .Values "redis-cluster").enabled -}}
|
||||
{{- $_ := set .Values.gitea.config.cache "ENABLED" "true" -}}
|
||||
{{- $_ := set .Values.gitea.config.cache "ADAPTER" "memcache" -}}
|
||||
{{- $_ := set .Values.gitea.config.cache "ADAPTER" "redis" -}}
|
||||
{{- if not (.Values.gitea.config.cache.HOST) -}}
|
||||
{{- $_ := set .Values.gitea.config.cache "HOST" (include "memcached.dns" .) -}}
|
||||
{{- $_ := set .Values.gitea.config.cache "HOST" (include "redis.dns" .) -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
{{- /* redis queue */ -}}
|
||||
{{- if (index .Values "redis-cluster").enabled -}}
|
||||
{{- $_ := set .Values.gitea.config.queue "TYPE" "redis" -}}
|
||||
{{- $_ := set .Values.gitea.config.queue "CONN_STR" (include "redis.dns" .) -}}
|
||||
{{- end -}}
|
||||
{{- /* multiple replicas */ -}}
|
||||
{{- if gt .Values.replicaCount 1.0 -}}
|
||||
{{- if not (get .Values.gitea.config.session "PROVIDER") -}}
|
||||
{{- $_ := set .Values.gitea.config.session "PROVIDER" "redis" -}}
|
||||
{{- end -}}
|
||||
{{- if not (get .Values.gitea.config.session "PROVIDER_CONFIG") -}}
|
||||
{{- $_ := set .Values.gitea.config.session "PROVIDER_CONFIG" (include "redis.dns" .) -}}
|
||||
{{- end -}}
|
||||
{{- if not .Values.gitea.config.indexer.ISSUE_INDEXER_TYPE -}}
|
||||
{{- $_ := set .Values.gitea.config.indexer "ISSUE_INDEXER_TYPE" "db" -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
{{- define "gitea.inline_configuration.defaults.server" -}}
|
||||
|
||||
Reference in New Issue
Block a user