# Homelab Provisioning Layer This layer prepares a Debian server to PXE boot a Debian 13 arm64 VM and install a reusable worker-node golden image for Pimox on Orange Pi 5 Plus. `./jeannie up` drives this layer in auto mode when the Pimox host is reachable at `192.168.100.80`, `qm` is installed, and the existing bridge `vmbr0` is present. The automation creates VM definitions only; it validates the bridge and refuses to edit Orange Pi host networking. ## What It Installs - `dnsmasq` for proxyDHCP and TFTP - `nginx` for preseed, installer, and guest-prep scripts - Debian 13 arm64 netboot assets under `/opt/homelab-provisioning` - a preseed file for unattended Debian install - a GRUB PXE rescue menu and HTTP rescue page at `/rescue/` - guest prep scripts that install Kubernetes tools, containerd, qemu guest agent, cloud-init, OpenEBS dependencies, cgroup prerequisites, swap disablement, and the local registry trust path - Kubernetes tools from the pinned minor in `TF_VAR_kubernetes_minor_version`, defaulting to `v1.36` - `containerd.io` from Docker's Debian repository so worker runtimes use the same package source as the Debian control-plane host - kernel boot options for cgroup support through `TF_VAR_kernel_cgroup_boot_options` - a template sealing script at `/usr/local/sbin/homelab-prepare-template.sh` inside the installed VM that verifies cgroup boot state before sealing ## Apply From Debian The normal path is: ```bash cd ~/my-homelab-configs ./jeannie up ``` For manual provisioning-only testing, find the LAN interface on the Debian server: ```bash ip -br addr ``` Apply the provisioning layer: ```bash cd ~/my-homelab-configs tofu -chdir=bootstrap/provisioning init TF_VAR_provisioning_interface=enp1s0 tofu -chdir=bootstrap/provisioning apply ``` Override `TF_VAR_provisioning_interface` with the interface that serves the Pimox VM network. The default VM user is `jv`. The account uses `/home/jv/.ssh/id_ed25519.pub` from the Debian server when that key exists, and the password is locked by default. Set `TF_VAR_template_user_ssh_authorized_keys` or `TF_VAR_template_user_password_hash` before applying if you want different access. Clones should get their intended hostname through cloud-init or the later clone automation. If they boot with the template hostname, the first-boot service generates a unique fallback name using `TF_VAR_clone_hostname_prefix`. ## Pimox VM Template Flow Create an arm64 VM in Pimox with UEFI firmware, a virtio disk, and a NIC on the same LAN as the Debian provisioning host. Put network boot first. PXE should load `grubaa64.efi` and display a GRUB menu. The default entry keeps the automated template flow intact: it boots the Debian installer, fetches the preseed from `http://192.168.100.73:8088/preseed/debian13-arm64-worker.cfg`, and installs the golden image. The same menu also includes rescue-oriented entries: - Debian 13 arm64 rescue shell - Debian 13 arm64 expert/manual install - Debian 13 arm64 automated install, verbose - UEFI firmware settings - Exit PXE and continue local boot - Reboot The Debian provisioning host also publishes a rescue landing page: ```bash curl http://192.168.100.73:8088/rescue/ ``` If your Pimox firmware needs a different Debian arm64 EFI loader, override `TF_VAR_pxe_boot_file`. After the first successful boot, run this inside the VM before converting it to a template: ```bash sudo /usr/local/sbin/homelab-prepare-template.sh sudo poweroff ``` Convert the powered-off VM to a Pimox template. Clone it for each new worker, set a unique hostname and IP address through cloud-init or DHCP reservation, then add it to `bootstrap/cluster/variables.tf` or a `.tfvars` file: ```hcl worker_nodes = { pimox01 = { host = "192.168.100.90" user = "jv" node_name = "pimox01" ssh_key_path = "/home/jv/.ssh/id_ed25519" } } ``` Run the cluster layer from the Debian homelab server after the cloned VM is reachable over SSH. ## Pimox Automation `./jeannie up` runs Pimox provisioning by default and sets `TF_VAR_pimox_template_builder_enabled=true` when Pimox is reachable. The layer SSHes into the Pimox host, create the template-build VM with `qm`, boot it from PXE, wait for the installed VM over SSH, run `/usr/local/sbin/homelab-prepare-template.sh`, power it off, switch boot order back to disk first, and run `qm template`. The template sealing step discovers the installed VM IP through qemu-guest-agent, so a DHCP reservation is no longer required for the temporary template-build VM. If you still want to force a known address, set `TF_VAR_pimox_template_build_host`. ```bash ./jeannie up ``` Defaults match the observed Pimox template VM shape: OVMF firmware, virtio networking, virtio-scsi disk, `vmbr0`, `local` template storage, 1 socket with 2 cores, and 4 GiB memory. Override `TF_VAR_pimox_template_scsi0`, `TF_VAR_pimox_template_efidisk0`, `TF_VAR_pimox_template_cores`, or `TF_VAR_pimox_template_memory` if the Orange Pi template layout changes. `./jeannie up` also creates or reuses worker clones after the template exists. It defaults to one worker, VMID `9010`, name `pimox-worker-01`, deterministic locally administered MAC address, 1 socket with 2 cores, 4 GiB RAM, `opi5_ssd` clone storage, and qemu-guest-agent IP discovery when that Pimox storage is available. New workers are full clones created with `qm clone --storage`, so the template can remain on `local` while worker disks land on the Orange Pi SSD-backed non-local Pimox datastore. Keep durable data on the HP Debian laptop's `data-vg` or external backups and treat Orange Pi VM disks as rebuildable worker capacity. Set `LAB_PIMOX_WORKER_REPLACE_EXISTING=true` to destroy and recreate existing worker VMs from the current template. The pipeline refuses `LAB_PIMOX_WORKER_STORAGE=local` so only the template VM lives on local storage. Worker networking defaults to static because the ISP modem DHCP server does not answer Pimox worker VM DHCP requests behind `vmbr0`: `LAB_PIMOX_WORKER_NETWORK_MODE=static`. DHCP mode remains available when LAN DHCP is known to answer VM traffic: set `LAB_PIMOX_WORKER_NETWORK_MODE=dhcp`. Static mode uses `LAB_PIMOX_WORKER_STATIC_IPS="192.168.100.66 192.168.100.67 192.168.100.76"`, `LAB_PIMOX_WORKER_GATEWAY=192.168.100.1`, `LAB_PIMOX_WORKER_INTERFACE=enp0s18`, and `LAB_PIMOX_WORKER_DNS_SERVERS="192.168.100.89 1.1.1.1"`. For static workers, Jeannie also adds targeted Pimox-host forwarding exceptions for each worker IP in `DOCKER-USER` when that chain exists, or `FORWARD` otherwise. Disable this with `LAB_PIMOX_WORKER_CONFIGURE_FORWARDING=false` only when the Pimox host firewall already permits bridged worker IP traffic. Before adding rules, Jeannie saves `iptables-save` output on the Pimox host under `${LAB_PIMOX_WORKER_IPTABLES_BACKUP_DIR:-~/iptables-backups}`. Worker NIC MAC addresses default to the VMID-derived deterministic format used by the original working Pimox worker path. Set `LAB_PIMOX_WORKER_MAC_MODE=auto` only when Pimox-generated MAC addresses are preferred. Worker clone failures are intentionally faster than template build failures: `LAB_PIMOX_GUEST_AGENT_CONFIG_TIMEOUT_SECONDS=300`, `LAB_PIMOX_WORKER_WAIT_TIMEOUT_SECONDS=600`, and `LAB_PIMOX_GUEST_SSH_TIMEOUT_SECONDS=600` by default. The broader `LAB_PIMOX_GUEST_TIMEOUT_SECONDS` still controls template preparation. Static guest-network validation skips the extra LAN ICMP probe by default because the ISP modem may not answer worker VM pings. Set `LAB_PIMOX_WORKER_REQUIRE_LAN_PROBE=true` only when the gateway/probe host should be required during qemu-guest-agent network setup. After qemu-guest-agent reports an IP, Debian skips the extra host-side ICMP probe by default and lets SSH be the reachability check. Enable the host-side ICMP gate with `LAB_PIMOX_WORKER_REQUIRE_HOST_PROBE=true`. Worker indexes are stable: index `1` maps to VMID `9010`, `pimox-worker-01`, and worker key `pimox01`; index `2` maps to VMID `9011`, `pimox-worker-02`, and worker key `pimox02`. `LAB_PIMOX_SKIP_WORKER_INDEXES` defaults to empty, so the pipeline owns index `1` unless you set `LAB_PIMOX_SKIP_WORKER_INDEXES=1`. Useful overrides: ```bash ./jeannie rebuild-cluster LAB_PIMOX_PIPELINE=false ./jeannie up LAB_PIMOX_TEMPLATE_REPLACE_EXISTING=true ./jeannie up LAB_PIMOX_WORKER_COUNT=0 ./jeannie up LAB_PIMOX_WORKER_COUNT=2 ./jeannie up LAB_PIMOX_SKIP_WORKER_INDEXES=1 LAB_PIMOX_WORKER_COUNT=2 ./jeannie up LAB_PIMOX_WORKER_BASE_VMID=9020 ./jeannie up LAB_PIMOX_WORKER_STORAGE= ./jeannie up LAB_PIMOX_WORKER_REPLACE_EXISTING=true ./jeannie up LAB_PIMOX_HOST=192.168.100.80 LAB_PIMOX_BRIDGE=vmbr0 ./jeannie up ``` ## OpenWrt firewall VM OpenWrt is not built from the Debian golden-node template. The Kubernetes template remains Debian-only; OpenWrt uses the upstream ARM SystemReady `armsr/armv8` combined EFI image instead. The OpenWrt path is separate from the Kubernetes worker template. Run it only after `vmbr1` exists on the Pimox host and the second NIC/LAN side is safe to use: ```bash ./jeannie openwrt ``` Defaults: - VMID `9100` - VM name `openwrt-firewall` - disk storage `opi5_ssd` when that Pimox storage is available - WAN bridge `vmbr0` - LAN bridge `vmbr1` - LAN address `192.168.50.1/24` - LAN DHCP disabled by default - VM start disabled by default - VM onboot disabled by default - OpenWrt version `24.10.6` Useful overrides: ```bash LAB_OPENWRT_VMID=9100 LAB_OPENWRT_STORAGE= LAB_OPENWRT_WAN_BRIDGE=vmbr0 LAB_OPENWRT_LAN_BRIDGE=vmbr1 LAB_OPENWRT_LAN_IP=192.168.50.1 LAB_OPENWRT_LAN_NETMASK=255.255.255.0 LAB_OPENWRT_LAN_DHCP_ENABLED=false LAB_OPENWRT_START=false LAB_OPENWRT_ONBOOT=false LAB_OPENWRT_VERSION=24.10.6 LAB_OPENWRT_IMAGE_URL=https://downloads.openwrt.org/releases/24.10.6/targets/armsr/armv8/openwrt-24.10.6-armsr-armv8-generic-ext4-combined-efi.img.gz ``` The pipeline validates `vmbr0`, `vmbr1`, and the selected storage on the Pimox host. It refuses `local` as OpenWrt storage and refuses to create or modify host network bridges.