124 lines
4.9 KiB
Markdown
124 lines
4.9 KiB
Markdown
# Homelab Provisioning Layer
|
|
|
|
This layer prepares a Debian server to PXE boot a Debian 13 arm64 VM and install a reusable worker-node golden image for Pimox on Orange Pi 5 Plus.
|
|
|
|
`./lab.sh up` drives this layer in auto mode when the Pimox host is reachable at
|
|
`192.168.100.80`, `qm` is installed, and the existing bridge `vmbr0` is present.
|
|
The automation creates VM definitions only; it validates the bridge and refuses
|
|
to edit Orange Pi host networking.
|
|
|
|
## What It Installs
|
|
|
|
- `dnsmasq` for proxyDHCP and TFTP
|
|
- `nginx` for preseed, installer, and guest-prep scripts
|
|
- Debian 13 arm64 netboot assets under `/opt/homelab-provisioning`
|
|
- a preseed file for unattended Debian install
|
|
- guest prep scripts that install Kubernetes tools, containerd, qemu guest agent, cloud-init, OpenEBS dependencies, cgroup prerequisites, swap disablement, and the local registry trust path
|
|
- kernel boot options for cgroup support through `TF_VAR_kernel_cgroup_boot_options`
|
|
- a template sealing script at `/usr/local/sbin/homelab-prepare-template.sh` inside the installed VM that verifies cgroup boot state before sealing
|
|
|
|
## Apply From Debian
|
|
|
|
The normal path is:
|
|
|
|
```bash
|
|
cd ~/my-homelab-configs
|
|
./lab.sh up
|
|
```
|
|
|
|
For manual provisioning-only testing, find the LAN interface on the Debian
|
|
server:
|
|
|
|
```bash
|
|
ip -br addr
|
|
```
|
|
|
|
Apply the provisioning layer:
|
|
|
|
```bash
|
|
cd ~/my-homelab-configs
|
|
tofu -chdir=bootstrap/provisioning init
|
|
TF_VAR_provisioning_interface=enp1s0 tofu -chdir=bootstrap/provisioning apply
|
|
```
|
|
|
|
Override `TF_VAR_provisioning_interface` with the interface that serves the Pimox VM network.
|
|
|
|
The default VM user is `jv`. The account uses `/home/jv/.ssh/id_ed25519.pub`
|
|
from the Debian server when that key exists, and the password is locked by
|
|
default. Set `TF_VAR_template_user_ssh_authorized_keys` or
|
|
`TF_VAR_template_user_password_hash` before applying if you want different
|
|
access.
|
|
|
|
Clones should get their intended hostname through cloud-init or the later clone
|
|
automation. If they boot with the template hostname, the first-boot service
|
|
generates a unique fallback name using `TF_VAR_clone_hostname_prefix`.
|
|
|
|
## Pimox VM Template Flow
|
|
|
|
Create an arm64 VM in Pimox with UEFI firmware, a virtio disk, and a NIC on the same LAN as the Debian provisioning host. Put network boot first.
|
|
|
|
PXE should load `grubaa64.efi`, boot the Debian installer, fetch the preseed from `http://192.168.100.68:8088/preseed/debian13-arm64-worker.cfg`, and install the golden image.
|
|
|
|
If your Pimox firmware needs a different Debian arm64 EFI loader, override `TF_VAR_pxe_boot_file`.
|
|
|
|
After the first successful boot, run this inside the VM before converting it to a template:
|
|
|
|
```bash
|
|
sudo /usr/local/sbin/homelab-prepare-template.sh
|
|
sudo poweroff
|
|
```
|
|
|
|
Convert the powered-off VM to a Pimox template. Clone it for each new worker, set a unique hostname and IP address through cloud-init or DHCP reservation, then add it to `bootstrap/cluster/variables.tf` or a `.tfvars` file:
|
|
|
|
```hcl
|
|
worker_nodes = {
|
|
pimox01 = {
|
|
host = "192.168.100.90"
|
|
user = "jv"
|
|
node_name = "pimox01"
|
|
ssh_key_path = "/home/jv/.ssh/id_ed25519"
|
|
}
|
|
}
|
|
```
|
|
|
|
Run the cluster layer from the Debian homelab server after the cloned VM is reachable over SSH.
|
|
|
|
## Pimox Automation
|
|
|
|
`./lab.sh up` sets `TF_VAR_pimox_template_builder_enabled=true` by default when
|
|
Pimox is reachable. The layer SSHes into the
|
|
Pimox host, create the template-build VM with `qm`, boot it from PXE, wait for
|
|
the installed VM over SSH, run `/usr/local/sbin/homelab-prepare-template.sh`,
|
|
power it off, switch boot order back to disk first, and run `qm template`.
|
|
|
|
The template sealing step discovers the installed VM IP through
|
|
qemu-guest-agent, so a DHCP reservation is no longer required for the temporary
|
|
template-build VM. If you still want to force a known address, set
|
|
`TF_VAR_pimox_template_build_host`.
|
|
|
|
```bash
|
|
LAB_PIMOX_PIPELINE=true ./lab.sh up
|
|
```
|
|
|
|
Defaults match the observed Pimox template VM shape: OVMF firmware, virtio
|
|
networking, virtio-scsi disk, `vmbr0`, `local` template storage, 2 vCPU, and
|
|
2 GiB memory. Override `TF_VAR_pimox_template_scsi0`,
|
|
`TF_VAR_pimox_template_efidisk0`, `TF_VAR_pimox_template_cores`, or
|
|
`TF_VAR_pimox_template_memory` if the Orange Pi template layout changes.
|
|
|
|
`./lab.sh up` also creates or reuses worker clones after the template exists. It
|
|
defaults to one worker, VMID `9010`, names like `pimox-worker-01`, deterministic
|
|
locally administered MAC addresses, `nvme_thin_pool` clone storage, and
|
|
qemu-guest-agent IP discovery. New workers are full clones created with
|
|
`qm clone --storage`, so the template can remain on `local` while worker disks
|
|
land on the NVMe thin pool. Useful overrides:
|
|
|
|
```bash
|
|
LAB_PIMOX_PIPELINE=false ./lab.sh up
|
|
LAB_PIMOX_WORKER_COUNT=0 ./lab.sh up
|
|
LAB_PIMOX_WORKER_COUNT=2 ./lab.sh up
|
|
LAB_PIMOX_WORKER_BASE_VMID=9020 ./lab.sh up
|
|
LAB_PIMOX_WORKER_STORAGE=nvme_thin_pool ./lab.sh up
|
|
LAB_PIMOX_HOST=192.168.100.80 LAB_PIMOX_BRIDGE=vmbr0 ./lab.sh up
|
|
```
|