Starting with kubernetes can be funny, especially when you finally understand a bit more about the internals and how to use all those manifests and objects.
After my first few steps and mistakes, which basically lead to a complete re-setup of my kind cluster, I decided I want to find a proper way to manage the configs.
After some research, I found support for kubernetes manifests in ansible, which a good friend of mine is fond of, so I gave it a spin.
- name: Create a k8s namespace community.kubernetes.k8s: name: testing api_version: v1 kind: Namespace state: present
This basically let’s us rewrite or rather copy the whole manifests and put them into playbooks, there is bit of template magic available, but I don’t see any really advantage here. Next!
Using good ol’ make should be sufficient here, right? We just apply a bunch of yaml files and can define dependencies to targets, this should make it pretty worthwhile.
After some hours fighting multiline commands and a complete lack of any heredoc, I assembled some
simple targets, which basically consisted of
kubectl apply -f deployment/file.yaml or the
reversal of it with delete.
During my time with make, I had a few learnings along the way:
- Setting variables inside of targets is difficult and can only be done via:
$(eval VAR := $(shell ls))
- Default targets can be set like this:
.DEFAULT_GOAL := target
- Use .PHONY, when the name of the target is also a directory:
I used this for a while, but.. NEXT!
Helm is another way of managing your resources. Behind the nautical-themed name is a full-featured repository manager and a powerful templating engine, which I mentioned earlier looks pretty similar to jinja2
So you can either use one of the many ready-to-use charts from a chart museum - that is the name of the repositories (still nautical - you see?) or roll your own. There is lots of ground to cover and I don’t want to even try to give a glimpse into it, there are dozen of better resources available.
Still one noteworthy thing is the new library type in v3 of helm: They allow us to create non-installable base charts and let new charts inherit from. This way you don’t have to copy/paste manifests and can just include them from a base.
This is pretty straight forward, here is a small example:
The layout can be like this:
$ ls -R1 base-chart Chart.yaml templates/ base-chart/templates: _deployment.tpl _helpers.tpl _service.tpl
$ ls -R1 inherited-chart Chart.yaml templates/ values.yaml inherited-chart/templates: _helpers.tpl deployment.yaml service.yaml
And the content of files can be like this:
So back to the task at hand, I wanted to manage a complete set up and with Helm I can create a bunch of charts to install them and manage the versions.
And to makle this easier: We can create a so-called umbrella chart, which just consists of dependencies to other charts and install them accordingly.
Helmfile is another layer on-top of Helm and makes the whole handling a bit easier. Once you’ve created a helm chart, you can take the values file from it and use the remaining chart as a template for the rest.
For new deployments, you just create a new values file and let Helmfile handle the rest.
A simple layout can be like this:
$ ls -R1 helmfile helmfile.yaml charts: base-chart/ charts/base-chart/ # snip environments: default.yaml values: inherited.yaml
The usage of environments is a bit tricky, but I will explain it down the road. So let us focus on the helmfile, which contains all the fun:
repositories: - name: stable url: https://charts.helm.sh/stable releases: - name: inherited chart: ./charts/base-chart #needs: #- other_chart values: - ./values/inherited.yaml
Here we describe a single release, with no other dependencies (needs), which uses the our
base-chart as a base and its values from a file named inherited.yaml. With this ready,
a single run of
helmfile sync should do the trick.
Environments offer a way to set stuff like ports, credentials and stuff like that for a complete env - so you don’t have to use different versions here.
Here we define the username and password for a postgress database_
postgres: username: test password: test
In order to use this config, we have to rename inherited.yaml to inherited.tpl:
# snip config: - key: POSTGRES_USER value: - key: POSTGRES_PASSWORD value: # snip