120 lines
4.3 KiB
Markdown
120 lines
4.3 KiB
Markdown
# Homelab Kubernetes Pipeline
|
|
|
|
This repo bootstraps a hybrid kubeadm cluster and then hands app delivery to
|
|
Argo CD.
|
|
|
|
## Flow
|
|
|
|
1. `bootstrap/cluster`
|
|
- creates the kubeadm control plane on the Debian amd64 node
|
|
- joins worker nodes such as Raspberry Pi arm64 nodes
|
|
- configures Calico-compatible pod CIDR
|
|
- configures containerd to pull from the in-cluster NodePort registry
|
|
- creates retained host directories under `/var/openebs/local`
|
|
|
|
2. `bootstrap/platform`
|
|
- installs a minimal Calico deployment through the Tigera operator
|
|
- installs OpenEBS
|
|
- creates `openebs-hostpath-retain`
|
|
- installs Argo CD
|
|
- registers the private GitOps repo without storing the SSH private key in
|
|
Terraform state
|
|
|
|
3. `bootstrap/apps`
|
|
- registers Argo CD Applications from the `applications` map
|
|
- default apps are `container-registry`, `gitea`, and
|
|
`website-production`
|
|
|
|
4. `bootstrap/edge`
|
|
- connects to the OCI jump box
|
|
- uploads nginx, HAProxy, Varnish, and Squid configs
|
|
- obtains and renews Let's Encrypt certificates for the configured hostname
|
|
- runs the edge cache/proxy chain with Docker Compose
|
|
|
|
## Adding Nodes
|
|
|
|
Add entries to `bootstrap/cluster/variables.tf` or a `.tfvars` file:
|
|
|
|
```hcl
|
|
worker_nodes = {
|
|
raspberrypi = {
|
|
host = "192.168.100.89"
|
|
user = "jv"
|
|
node_name = "raspberry"
|
|
ssh_key_path = "/home/jv/.ssh/id_ed25519"
|
|
}
|
|
}
|
|
```
|
|
|
|
Stateful apps currently pin retained local PVs to the `debian` node. Move or
|
|
duplicate those PV manifests when you want storage on another node.
|
|
|
|
The website NodePort is reachable from the OCI jump box through the Raspberry Pi
|
|
Tailscale interface. `bootstrap/cluster` installs a persistent
|
|
`homelab-tailscale-nodeport.service` on the configured worker to restore the
|
|
route, rp_filter settings, and iptables rules after reboot. Override the
|
|
defaults through `tailscale_nodeport_access` when the jump-box IP, Pi Tailscale
|
|
IP, pod CIDR, or NodePort changes:
|
|
|
|
```hcl
|
|
tailscale_nodeport_access = {
|
|
enabled = true
|
|
worker_key = "raspberrypi"
|
|
peer_ip = "100.118.255.19"
|
|
node_tailscale_ip = "100.77.80.72"
|
|
pod_cidr = "10.244.0.0/16"
|
|
node_port = 30080
|
|
target_port = 80
|
|
}
|
|
```
|
|
|
|
For `./lab.sh nuke`, set `WORKER_SSH_TARGETS` to a space-separated list of
|
|
remote SSH targets when more worker nodes exist. Set it to an empty string for a
|
|
single-node rebuild.
|
|
|
|
## Adding Platform Tools
|
|
|
|
Add Helm releases through `bootstrap/platform`'s `extra_helm_releases` map.
|
|
|
|
## Edge Services
|
|
|
|
The OCI jump box runs the public edge path:
|
|
|
|
```text
|
|
nginx -> HAProxy -> Varnish/Squid -> Raspberry Pi Tailscale NodePort
|
|
```
|
|
|
|
The `bootstrap/edge` stack renders configs from `bootstrap/edge/templates` and
|
|
deploys them to `/opt/homelab-edge` on the OCI host. Defaults are in
|
|
`bootstrap/edge/variables.tf`; override them through `TF_VAR_*` or a `.tfvars`
|
|
file when the public host, SSH key, server name, backend Tailscale IP, or
|
|
NodePort changes.
|
|
|
|
Use the configured `server_name` in the browser, for example
|
|
`https://lab2025.duckdns.org`. A raw OCI IP address will still show a browser
|
|
certificate warning because the trusted certificate is issued for the hostname.
|
|
|
|
The edge stack uses HTTP-01 validation, so public DNS for `server_name` must
|
|
point to the OCI public IP and inbound TCP 80 and 443 must be open before
|
|
`./lab.sh deploy` runs. Set `TF_VAR_letsencrypt_email` to receive expiry notices,
|
|
or leave it empty to register without an email. Set
|
|
`TF_VAR_enable_letsencrypt=false` to keep using the temporary local certificate.
|
|
|
|
## Adding Apps
|
|
|
|
Add Kubernetes manifests under `apps/<name>` and register them in
|
|
`bootstrap/apps`'s `applications` map. Argo CD will own sync, pruning, and
|
|
self-healing for the app.
|
|
|
|
## Storage
|
|
|
|
OpenEBS provides the platform storage provisioner. Stateful homelab apps use
|
|
retained local PV paths such as `/var/openebs/local/gitea` and
|
|
`/var/openebs/local/registry`; these paths are intentionally outside kubeadm
|
|
reset paths so data can survive cluster destroy/create cycles. Those critical
|
|
volumes are declared explicitly as retained local PVs so a rebuilt cluster binds
|
|
back to the same host paths instead of creating fresh directories.
|
|
|
|
Keep the `.terraform.lock.hcl` files committed. They pin provider selections and
|
|
make bootstrap behavior reproducible across nodes and rebuilds.
|