Skip to content
Adam Learns notes

Other Kubernetes tools ecosystem

Created: 2020-10-27 08:56:12 -0700 Modified: 2020-11-20 18:32:48 -0800

Background

Section titled Background

In my opinion, one of the hardest questions to answer with most development-related tools is “why?”; this note tries to at least answer that question for each of the tools listed.

Kubebuilder (reference)

Section titled Kubebuilder (reference)
  • Overview (reference): it’s a framework intended to overcome the boilerplate associated with making tools and APIs for Kubernetes. It does so by creating CRDs, custom controllers and admission webhooks for you (or at least the scaffolding of all of these things; you still have to configure/code some things afterward).

kpt (pronounced “kept”) (reference, reference2)

Section titled kpt (pronounced “kept”) (reference, reference2)
  • Overview (reference): from what I can tell, the main use-case seems to be to take declarative**,** Kubernetes configuration (i.e. YAML files) from a generic Git repo (e.g. your company’s “best-practices” repo), transform them (e.g. change “image: example” → “image: nginx”), then commit them back to a specific Git repo (e.g. your team’s repo). Once pushed, the CI/CD pipeline will provide the configuration files to Kubernetes so that it knows the desired state of your application (and thus can reconcile it with the current state).
    • Making sense of their own description: After reading what’s above, their own description on the site starts to make more sense—they bill themselves as “a tool for building declarative workflows on top of resource configurations”: “fetch, display, customize, update, validate, and apply Kubernetes configuration”.
  • kpt vs. kustomize: they’re both very similar and can be used together. Kustomize has been around since 2017 (at least based on this commit), whereas kpt is from 2020 (reference). Kustomize tries to match Kubernetes’ configuration format exactly, so overwriting “replicas” from 3 to 5 for a Deployment doesn’t really involve learning a new format. kpt has a concept of commands (like “kpt cfg set helloworld replicas 5”) which turn into YAML (in the form of a kptfile) that looks like what you’d write yourself (reference).
    • Docs: at least as of 10/27/20, kpt’s docs seem to be in pretty bad shape in terms of answering my questions (e.g. the kptfile page just says “Reference for the Kptfile schema” with literally nothing else on it).
  • kpt functions (reference): they’re similar to Kubernetes controllers and validating webhooks in that they can CRUD/validate resources. As such, they take in a desired state of a resource and reconcile that with the current state.

Kustomize (reference)

Section titled Kustomize (reference)
  • Overview: like kpt, the goal is to customize existing, general declarative configurations to form specific configurations.
    • Example (reference): suppose you have “base.yaml” which defines a Deployment with 1 replica, but in your application, you want it to have 3 replicas. With kustomize, you would make replicacount.yaml with identifying information for the Deployment (i.e. you specify apiVersion, kind, metadata → name) and the desired spec → replicas value for the Deployment, and you _also make a kustomization.yaml which identifies your base file (base.yaml) and your patches (replica_count.yaml). Kustomize will then know how to patch your generic base file to form the specific version that you need.
    • Overlay vs. templating (reference): with templating, you would have something like “replicas: $replicaCount” in your YAML file, then something would be responsible for replacing it. Kustomize uses an overlay system instead, which allows you to overwrite arbitrary parts of the YAML file. The “arbitrary” part is the main benefit, and the referenced slides talk about why: imagine you’re using someone’s generic base.yaml that didn’t templatize whatever you wanted to override. You’d have to fork the base configuration and then manage when you pull updates from the original upstream.
    • Built into kubectl: “kubectl apply -k” takes a kustomization.yaml file to create or update objects (more information about “kubectl apply” itself here).
    • Benefits of Kustomize: it’s the same YAML format that Kubernetes uses, so you don’t really have to learn new tools, a domain-specific language, or new concepts.
    • Other features (reference)
      • Matching (reference): you can match resources based on name prefixes, common labels or common annotations, and probably other criteria.
      • ConfigMap generation (reference): you can generate a Kubernetes ConfigMap (i.e. just a set of key-value pairs) from an easy-to-define file. This is likely just to simplify the formatting and keep your ConfigMaps separate in source control.

Helm (reference)

Section titled Helm (reference)
  • Overview: it’s a package manager for Kubernetes applications. Packages are called “charts”. Charts can be simple (e.g. an nginx Pod) or complex (e.g. a full web-app stack with HTTP servers, databases, caches, etc.). They can depend on other charts, introduce CRDs and/or custom controllers, and specify templatized Kubernetes YAML files. Just like pretty much any other package manager, I believe its primary goal is to be super easy/fast to get something up and running—install a chart, configure its variables, and you should be good to go.