my-homelab-configs/bootstrap/provisioning

Homelab Provisioning Layer

This layer prepares a Debian server to PXE boot a Debian 13 arm64 VM and install a reusable worker-node golden image for Pimox on Orange Pi 5 Plus.

./lab.sh up drives this layer in auto mode when the Pimox host is reachable at 192.168.100.80, qm is installed, and the existing bridge vmbr0 is present. The automation creates VM definitions only; it validates the bridge and refuses to edit Orange Pi host networking.

What It Installs

• dnsmasq for proxyDHCP and TFTP
• nginx for preseed, installer, and guest-prep scripts
• Debian 13 arm64 netboot assets under /opt/homelab-provisioning
• a preseed file for unattended Debian install
• guest prep scripts that install Kubernetes tools, containerd, qemu guest agent, cloud-init, OpenEBS dependencies, cgroup prerequisites, swap disablement, and the local registry trust path
• kernel boot options for cgroup support through TF_VAR_kernel_cgroup_boot_options
• a template sealing script at /usr/local/sbin/homelab-prepare-template.sh inside the installed VM that verifies cgroup boot state before sealing

Apply From Debian

The normal path is:

cd ~/my-homelab-configs
./lab.sh up

For manual provisioning-only testing, find the LAN interface on the Debian server:

ip -br addr

Apply the provisioning layer:

cd ~/my-homelab-configs
tofu -chdir=bootstrap/provisioning init
TF_VAR_provisioning_interface=enp1s0 tofu -chdir=bootstrap/provisioning apply

Override TF_VAR_provisioning_interface with the interface that serves the Pimox VM network.

The default VM user is jv. The account uses /home/jv/.ssh/id_ed25519.pub from the Debian server when that key exists, and the password is locked by default. Set TF_VAR_template_user_ssh_authorized_keys or TF_VAR_template_user_password_hash before applying if you want different access.

Clones should get their intended hostname through cloud-init or the later clone automation. If they boot with the template hostname, the first-boot service generates a unique fallback name using TF_VAR_clone_hostname_prefix.

Pimox VM Template Flow

Create an arm64 VM in Pimox with UEFI firmware, a virtio disk, and a NIC on the same LAN as the Debian provisioning host. Put network boot first.

PXE should load grubaa64.efi, boot the Debian installer, fetch the preseed from http://192.168.100.68:8088/preseed/debian13-arm64-worker.cfg, and install the golden image.

If your Pimox firmware needs a different Debian arm64 EFI loader, override TF_VAR_pxe_boot_file.

After the first successful boot, run this inside the VM before converting it to a template:

sudo /usr/local/sbin/homelab-prepare-template.sh
sudo poweroff

Convert the powered-off VM to a Pimox template. Clone it for each new worker, set a unique hostname and IP address through cloud-init or DHCP reservation, then add it to bootstrap/cluster/variables.tf or a .tfvars file:

worker_nodes = {
  pimox01 = {
    host         = "192.168.100.90"
    user         = "jv"
    node_name    = "pimox01"
    ssh_key_path = "/home/jv/.ssh/id_ed25519"
  }
}

Run the cluster layer from the Debian homelab server after the cloned VM is reachable over SSH.

Pimox Automation

./lab.sh up runs Pimox provisioning by default and sets TF_VAR_pimox_template_builder_enabled=true when Pimox is reachable. The layer SSHes into the Pimox host, create the template-build VM with qm, boot it from PXE, wait for the installed VM over SSH, run /usr/local/sbin/homelab-prepare-template.sh, power it off, switch boot order back to disk first, and run qm template.

The template sealing step discovers the installed VM IP through qemu-guest-agent, so a DHCP reservation is no longer required for the temporary template-build VM. If you still want to force a known address, set TF_VAR_pimox_template_build_host.

./lab.sh up

Defaults match the observed Pimox template VM shape: OVMF firmware, virtio networking, virtio-scsi disk, vmbr0, local template storage, 1 socket with 2 cores, and 4 GiB memory. Override TF_VAR_pimox_template_scsi0, TF_VAR_pimox_template_efidisk0, TF_VAR_pimox_template_cores, or TF_VAR_pimox_template_memory if the Orange Pi template layout changes.

./lab.sh up also creates or reuses worker clones after the template exists. It defaults to one worker, VMID 9010, name pimox-worker-01, deterministic locally administered MAC address, 1 socket with 2 cores, 4 GiB RAM, nvme_thin_pool clone storage, and qemu-guest-agent IP discovery. New workers are full clones created with qm clone --storage, so the template can remain on local while worker disks land on the NVMe thin pool. Set LAB_PIMOX_WORKER_REPLACE_EXISTING=true to destroy and recreate existing worker VMs from the current template. The pipeline refuses LAB_PIMOX_WORKER_STORAGE=local so only the template VM lives on local storage. Useful overrides:

./lab.sh rebuild-cluster
LAB_PIMOX_PIPELINE=false ./lab.sh up
LAB_PIMOX_TEMPLATE_REPLACE_EXISTING=true ./lab.sh up
LAB_PIMOX_WORKER_COUNT=0 ./lab.sh up
LAB_PIMOX_WORKER_COUNT=2 ./lab.sh up
LAB_PIMOX_WORKER_BASE_VMID=9020 ./lab.sh up
LAB_PIMOX_WORKER_STORAGE=nvme_thin_pool ./lab.sh up
LAB_PIMOX_WORKER_REPLACE_EXISTING=true ./lab.sh up
LAB_PIMOX_HOST=192.168.100.80 LAB_PIMOX_BRIDGE=vmbr0 ./lab.sh up

OpenWrt firewall VM

OpenWrt is not built from the Debian golden-node template. The Kubernetes template remains Debian-only; OpenWrt uses the upstream ARM SystemReady armsr/armv8 combined EFI image instead.

The OpenWrt path is disabled by default. Enable it only after vmbr1 exists on the Pimox host and the second NIC/LAN side is safe to use:

LAB_OPENWRT_VM=true ./lab.sh up

Defaults:

• VMID 9050
• VM name openwrt-firewall
• disk storage nvme_thin_pool
• WAN bridge vmbr0
• LAN bridge vmbr1
• LAN address 192.168.50.1/24
• LAN DHCP disabled by default
• OpenWrt version 24.10.6

Useful overrides:

LAB_OPENWRT_VMID=9050
LAB_OPENWRT_STORAGE=nvme_thin_pool
LAB_OPENWRT_WAN_BRIDGE=vmbr0
LAB_OPENWRT_LAN_BRIDGE=vmbr1
LAB_OPENWRT_LAN_IP=192.168.50.1
LAB_OPENWRT_LAN_NETMASK=255.255.255.0
LAB_OPENWRT_LAN_DHCP_ENABLED=true
LAB_OPENWRT_START=true
LAB_OPENWRT_VERSION=24.10.6
LAB_OPENWRT_IMAGE_URL=https://downloads.openwrt.org/releases/24.10.6/targets/armsr/armv8/openwrt-24.10.6-armsr-armv8-generic-ext4-combined-efi.img.gz

The pipeline validates vmbr0, vmbr1, and nvme_thin_pool on the Pimox host. It refuses local as OpenWrt storage and refuses to create or modify host network bridges.