# Homelab Provisioning Layer This layer prepares a Debian server to PXE boot a Debian 13 arm64 VM and install a reusable worker-node golden image for Pimox on Orange Pi 5 Plus. `./lab.sh up` drives this layer in auto mode when the Pimox host is reachable at `192.168.100.80`, `qm` is installed, and the existing bridge `vmbr0` is present. The automation creates VM definitions only; it validates the bridge and refuses to edit Orange Pi host networking. ## What It Installs - `dnsmasq` for proxyDHCP and TFTP - `nginx` for preseed, installer, and guest-prep scripts - Debian 13 arm64 netboot assets under `/opt/homelab-provisioning` - a preseed file for unattended Debian install - guest prep scripts that install Kubernetes tools, containerd, qemu guest agent, cloud-init, OpenEBS dependencies, cgroup prerequisites, swap disablement, and the local registry trust path - kernel boot options for cgroup support through `TF_VAR_kernel_cgroup_boot_options` - a template sealing script at `/usr/local/sbin/homelab-prepare-template.sh` inside the installed VM that verifies cgroup boot state before sealing ## Apply From Debian The normal path is: ```bash cd ~/my-homelab-configs ./lab.sh up ``` For manual provisioning-only testing, find the LAN interface on the Debian server: ```bash ip -br addr ``` Apply the provisioning layer: ```bash cd ~/my-homelab-configs tofu -chdir=bootstrap/provisioning init TF_VAR_provisioning_interface=enp1s0 tofu -chdir=bootstrap/provisioning apply ``` Override `TF_VAR_provisioning_interface` with the interface that serves the Pimox VM network. The default VM user is `jv`. The account uses `/home/jv/.ssh/id_ed25519.pub` from the Debian server when that key exists, and the password is locked by default. Set `TF_VAR_template_user_ssh_authorized_keys` or `TF_VAR_template_user_password_hash` before applying if you want different access. Clones should get their intended hostname through cloud-init or the later clone automation. If they boot with the template hostname, the first-boot service generates a unique fallback name using `TF_VAR_clone_hostname_prefix`. ## Pimox VM Template Flow Create an arm64 VM in Pimox with UEFI firmware, a virtio disk, and a NIC on the same LAN as the Debian provisioning host. Put network boot first. PXE should load `grubaa64.efi`, boot the Debian installer, fetch the preseed from `http://192.168.100.68:8088/preseed/debian13-arm64-worker.cfg`, and install the golden image. If your Pimox firmware needs a different Debian arm64 EFI loader, override `TF_VAR_pxe_boot_file`. After the first successful boot, run this inside the VM before converting it to a template: ```bash sudo /usr/local/sbin/homelab-prepare-template.sh sudo poweroff ``` Convert the powered-off VM to a Pimox template. Clone it for each new worker, set a unique hostname and IP address through cloud-init or DHCP reservation, then add it to `bootstrap/cluster/variables.tf` or a `.tfvars` file: ```hcl worker_nodes = { pimox01 = { host = "192.168.100.90" user = "jv" node_name = "pimox01" ssh_key_path = "/home/jv/.ssh/id_ed25519" } } ``` Run the cluster layer from the Debian homelab server after the cloned VM is reachable over SSH. ## Pimox Automation `./lab.sh up` sets `TF_VAR_pimox_template_builder_enabled=true` by default when Pimox is reachable. The layer SSHes into the Pimox host, create the template-build VM with `qm`, boot it from PXE, wait for the installed VM over SSH, run `/usr/local/sbin/homelab-prepare-template.sh`, power it off, switch boot order back to disk first, and run `qm template`. The template sealing step discovers the installed VM IP through qemu-guest-agent, so a DHCP reservation is no longer required for the temporary template-build VM. If you still want to force a known address, set `TF_VAR_pimox_template_build_host`. ```bash LAB_PIMOX_PIPELINE=true ./lab.sh up ``` Defaults match the observed Pimox template VM shape: OVMF firmware, virtio networking, virtio-scsi disk, `vmbr0`, `local` template storage, 1 socket with 2 cores, 4 GiB memory, and high-speed CPU affinity `4-5`. Override `TF_VAR_pimox_template_scsi0`, `TF_VAR_pimox_template_efidisk0`, `TF_VAR_pimox_template_cores`, `TF_VAR_pimox_template_memory`, or `TF_VAR_pimox_template_cpu_affinity` if the Orange Pi template layout changes. `./lab.sh up` also creates or reuses worker clones after the template exists. It defaults to two workers, VMIDs `9010` and `9011`, names like `pimox-worker-01`, deterministic locally administered MAC addresses, 1 socket with 2 cores, 4 GiB RAM, Orange Pi 5 high-speed CPU affinity pairs `4-5` and `6-7`, `nvme_thin_pool` clone storage, and qemu-guest-agent IP discovery. New workers are full clones created with `qm clone --storage`, so the template can remain on `local` while worker disks land on the NVMe thin pool. Set `LAB_PIMOX_WORKER_REPLACE_EXISTING=true` to destroy and recreate existing worker VMs from the current template. The pipeline refuses `LAB_PIMOX_WORKER_STORAGE=local` so only the template VM lives on local storage. Useful overrides: ```bash ./lab.sh rebuild-cluster LAB_PIMOX_PIPELINE=false ./lab.sh up LAB_PIMOX_TEMPLATE_REPLACE_EXISTING=true ./lab.sh up LAB_PIMOX_WORKER_COUNT=0 ./lab.sh up LAB_PIMOX_WORKER_COUNT=2 ./lab.sh up LAB_PIMOX_WORKER_BASE_VMID=9020 ./lab.sh up LAB_PIMOX_WORKER_STORAGE=nvme_thin_pool ./lab.sh up LAB_PIMOX_WORKER_REPLACE_EXISTING=true ./lab.sh up LAB_PIMOX_WORKER_CPU_AFFINITIES="4-5 6-7" ./lab.sh up LAB_PIMOX_HOST=192.168.100.80 LAB_PIMOX_BRIDGE=vmbr0 ./lab.sh up ``` ## OpenWrt firewall VM OpenWrt is not built from the Debian golden-node template. The Kubernetes template remains Debian-only; OpenWrt uses the upstream ARM SystemReady `armsr/armv8` combined EFI image instead. The OpenWrt path is disabled by default. Enable it only after `vmbr1` exists on the Pimox host and the second NIC/LAN side is safe to use: ```bash LAB_OPENWRT_VM=true ./lab.sh up ``` Defaults: - VMID `9050` - VM name `openwrt-firewall` - disk storage `nvme_thin_pool` - WAN bridge `vmbr0` - LAN bridge `vmbr1` - LAN address `192.168.50.1/24` - LAN DHCP disabled by default - OpenWrt version `24.10.6` Useful overrides: ```bash LAB_OPENWRT_VMID=9050 LAB_OPENWRT_STORAGE=nvme_thin_pool LAB_OPENWRT_WAN_BRIDGE=vmbr0 LAB_OPENWRT_LAN_BRIDGE=vmbr1 LAB_OPENWRT_LAN_IP=192.168.50.1 LAB_OPENWRT_LAN_NETMASK=255.255.255.0 LAB_OPENWRT_LAN_DHCP_ENABLED=true LAB_OPENWRT_START=true LAB_OPENWRT_VERSION=24.10.6 LAB_OPENWRT_IMAGE_URL=https://downloads.openwrt.org/releases/24.10.6/targets/armsr/armv8/openwrt-24.10.6-armsr-armv8-generic-ext4-combined-efi.img.gz ``` The pipeline validates `vmbr0`, `vmbr1`, and `nvme_thin_pool` on the Pimox host. It refuses `local` as OpenWrt storage and refuses to create or modify host network bridges.