Deploying my static site with Kubernetes (k3s)

The notes below reflect my mental model and personal difficulties setting up Kubernetes and may contain inaccuracies.

Kubernetes basics for developers who know docker

K8s abstracts literally everything you know and love about docker. Instead of directly creating containers we create Pods that contain 1 or more containers. Instead of exposing ports on a container we create Services that represent how you connect to pods. Services by default don't expose your pods to the outside world, so you need to create an Ingress resource to describe how connections from a host reach a service.

Simply put: network request -> Ingress -> Service -> Pod

Generally you wouldn't provision pods directly, rather you describe Deployments that create replicated pods which spin up automatically so the lifecycle of a request looks more like:

network request -> Ingress -> Service -> Deployment [Round-robin selection] -> Pod

First steps

Kubernetes Authorization

In Kubernetes, roles and role bindings are used to control access to resources and actions within a namespace, while cluster roles and cluster role bindings are used to control access to resources and actions across the entire cluster.

A brief overview of the differences between these resources:

In general, you would use roles and role bindings to grant permissions to users within a specific namespace, and you would use cluster roles and cluster role bindings to grant permissions to users across the entire cluster.

Cert Manager

Really cool tool to provision and renew TLS certs for your hosts but there's a lot of idiosyncrasies that I messed up and spent hours resolving.

If the cluster issuer needs secrets (API tokens for DNS challenges) then the secrets must reside in the same namespace as cert-manager.

Steps to provision TLS with DNS challenge for Cloudflare

  1. Create a ClusterIssuer (works across all namespaces)
  2. Create a Secret with Cloudflare API token and deploy in cert-manager namespace
  3. Annotate ingress with clusterIssuerName

If you wish to use this annotation method to serve a specific host (e.g. with a certificate at which contains the SAN * then see the following note from the documentation.

Note: dnsNames take an exact match and do not resolve wildcards, meaning the following Issuer will not solve for DNS names such as Use the dnsZones selector type to match all subdomains within a zone.

Don't trust yourself and assume that you will get this right on the first try, you're probably going to fuck it up and bust the rate limiting for your certificate authority. Use a staging environment if available to provision your certificates. Even if it succeeds make sure that you're not regenerating certs when you redeploy ingress, restart the cluster, etc.

Traefik Ingress Controller

Exposing the traefik dashboard k -n kube-system port-forward traefik-66c46d954f-fkwdf 8080:9000