Consolidate your GitLab stack with Gitaly on Kubernetes
Source: GitLab Blog
With GitLab 18.11, Gitaly on Kubernetes is now generally available. Teams that previously ran most GitLab components in Kubernetes while keeping Gitaly on virtual machines can finally consolidate their stack and eliminate the hybrid architecture that added operational complexity.
The road to Kubernetes
Gitaly has hard requirements that don’t map naturally to Kubernetes:
- Memory‑intensive Git operations: To protect the main Gitaly process from out‑of‑memory (OOM) events, each Git process can be run inside a dedicated cgroup. If a Git process exceeds its cgroup’s memory limit and is terminated, the main Gitaly process remains unaffected.
- cgroup handling in containers: Most clusters use
containerd, which historically allowed containers to write tocgroupfsonly in privileged mode. The workaround was to mount/sys/fs/cgroupvia an init container and make the path writable. - Pod restarts: On VMs, Omnibus can upgrade the Gitaly binary in‑place and reload gracefully by keeping the socket open. In Kubernetes, a StatefulSet pod replacement (Helm upgrade, node drain, config change) results in a hard stop—no graceful reload. For sharded Gitaly, which lacks high‑availability, this could cause downtime.
Our solution
We made client retries configurable. By configuring Gitaly clients (e.g., Rails) to retry requests long enough for Gitaly to restart, users may see a brief latency increase, but requests ultimately succeed and downtime is avoided.
Benchmark results
To verify that client retries eliminate downtime, we benchmarked common Git operations against two GitLab instances—one with Gitaly on VMs and another on Kubernetes—while triggering an upgrade mid‑test.
| Operation | VM Success Rate | Kubernetes Success Rate |
|---|---|---|
git clone | 100% | 100% |
git pull | 100% | 99.16% |
git push | 99.66% | 100% |
The success rates are nearly identical. Although a 100% success across every operation would require the high‑availability solution Gitaly Cluster (Praefect)—which currently doesn’t support Kubernetes—support is actively being developed with GA on the horizon.
What Gitaly on Kubernetes means for you
- Hybrid deployments: If you’re running GitLab components in Kubernetes but Gitaly on VMs, you can now move Gitaly into the cluster, removing the need to maintain a separate VM fleet.
- New adopters: If you’re deploying GitLab for the first time on Kubernetes, you can use a fully Kubernetes‑native deployment via the official Helm chart.
Installing Gitaly on Kubernetes
The recommended deployment method is the GitLab Helm chart. Before proceeding, review the Gitaly on Kubernetes documentation for key configuration guidance and common pitfalls.
Gitaly can be deployed:
- As part of a full GitLab installation, or
- As an external component.
Both scenarios are covered in the documentation.