diff --git a/README.md b/README.md index e6fc4d0..ff54333 100644 --- a/README.md +++ b/README.md @@ -29,11 +29,12 @@ accidentally modify the cluster. - serves Debian 13 arm64 netboot assets through TFTP and HTTP - creates a golden image install path with Kubernetes, containerd, qemu-guest-agent, cloud-init, and storage client packages ready - - stays out of `./lab.sh up` so VM template creation remains manual + - is driven by `./lab.sh up` when Pimox is reachable, without changing + Orange Pi host networking 2. `bootstrap/cluster` - creates the kubeadm control plane on the Debian amd64 node - - joins worker nodes such as Raspberry Pi arm64 nodes + - joins worker nodes such as Raspberry Pi and Pimox Debian arm64 nodes - configures Calico-compatible pod CIDR - configures containerd to pull from the in-cluster NodePort registry - creates retained host directories under `/var/openebs/local` @@ -80,10 +81,19 @@ cd ~/my-homelab-configs ./lab.sh up ``` -The script applies the OpenTofu stacks in order, refreshes Argo CD apps, waits -for the local registry, builds the website and demos images when their source -changed, pushes them to the registry, recreates pods only after a new image is -built, and then applies the edge stack. +The script detects the Pimox host at `192.168.100.80` in auto mode. When SSH, +`qm`, and `vmbr0` are available, it applies `bootstrap/provisioning`, creates or +reuses the Debian 13 arm64 template, creates or reuses one worker VM clone, +discovers the guest IP through qemu-guest-agent, and passes that worker into the +cluster layer. It then applies the remaining OpenTofu stacks, refreshes Argo CD +apps, waits for the local registry, builds the website and demos images when +their source changed, pushes them to the registry, recreates pods only after a +new image is built, and applies the edge stack. + +Set `LAB_PIMOX_PIPELINE=false` to skip Pimox automation. Set +`LAB_PIMOX_WORKER_COUNT=0` to create or refresh only the template. The pipeline +checks that the Pimox bridge already exists and refuses to edit Orange Pi host +networking. The website and demos images default to `linux/arm64` because both deployments are pinned to the Raspberry Pi worker. Override with `WEBSITE_IMAGE_PLATFORMS` @@ -117,10 +127,11 @@ hostname. ## Adding Nodes -For Pimox on Orange Pi 5 Plus, use `bootstrap/provisioning` to create a Debian -13 arm64 golden image first. The layer serves PXE, preseed, and guest-prep -assets from the Debian homelab server, then the installed VM can be sealed and -converted to a Pimox template. Details are in `bootstrap/provisioning/README.md`. +For Pimox on Orange Pi 5 Plus, `./lab.sh up` can create the Debian 13 arm64 +template and worker VM clones automatically. Defaults are intentionally tied to +the observed host: Pimox SSH host `192.168.100.80`, bridge `vmbr0`, template VMID +`9000`, and worker VMIDs starting at `9010`. Details and override variables are +in `bootstrap/provisioning/README.md`. Add entries to `bootstrap/cluster/variables.tf` or a `.tfvars` file: diff --git a/bootstrap/provisioning/README.md b/bootstrap/provisioning/README.md index e938ed6..027e564 100644 --- a/bootstrap/provisioning/README.md +++ b/bootstrap/provisioning/README.md @@ -2,7 +2,10 @@ This layer prepares a Debian server to PXE boot a Debian 13 arm64 VM and install a reusable worker-node golden image for Pimox on Orange Pi 5 Plus. -It is intentionally separate from `./lab.sh up`. Run it manually from the Debian homelab server only when you want to create or refresh the provisioning service. +`./lab.sh up` drives this layer in auto mode when the Pimox host is reachable at +`192.168.100.80`, `qm` is installed, and the existing bridge `vmbr0` is present. +The automation creates VM definitions only; it validates the bridge and refuses +to edit Orange Pi host networking. ## What It Installs @@ -16,7 +19,15 @@ It is intentionally separate from `./lab.sh up`. Run it manually from the Debian ## Apply From Debian -Find the LAN interface on the Debian server: +The normal path is: + +```bash +cd ~/my-homelab-configs +./lab.sh up +``` + +For manual provisioning-only testing, find the LAN interface on the Debian +server: ```bash ip -br addr @@ -72,20 +83,21 @@ worker_nodes = { Run the cluster layer from the Debian homelab server after the cloned VM is reachable over SSH. -## Optional Pimox Automation +## Pimox Automation -Set `TF_VAR_pimox_template_builder_enabled=true` to have this layer SSH into the +`./lab.sh up` sets `TF_VAR_pimox_template_builder_enabled=true` by default when +Pimox is reachable. The layer SSHes into the Pimox host, create the template-build VM with `qm`, boot it from PXE, wait for the installed VM over SSH, run `/usr/local/sbin/homelab-prepare-template.sh`, power it off, switch boot order back to disk first, and run `qm template`. -The automation needs a predictable temporary IP for the installed VM. Use a DHCP -reservation for the generated or configured MAC address, then set: +The template sealing step discovers the installed VM IP through +qemu-guest-agent, so a DHCP reservation is no longer required for the temporary +template-build VM. If you still want to force a known address, set +`TF_VAR_pimox_template_build_host`. ```bash -export TF_VAR_pimox_template_builder_enabled=true -export TF_VAR_pimox_host=192.168.100.91 -export TF_VAR_pimox_template_build_host=192.168.100.90 +LAB_PIMOX_PIPELINE=true ./lab.sh up ``` Defaults match the observed Pimox VM shape: OVMF firmware, virtio networking, @@ -93,3 +105,16 @@ virtio-scsi disk, `vmbr0`, `local` storage, 2 vCPU, and 2 GiB memory. Override `TF_VAR_pimox_template_scsi0`, `TF_VAR_pimox_template_efidisk0`, `TF_VAR_pimox_template_cores`, or `TF_VAR_pimox_template_memory` if the Orange Pi storage layout changes. + +`./lab.sh up` also creates or reuses worker clones after the template exists. It +defaults to one worker, VMID `9010`, names like `pimox-worker-01`, deterministic +locally administered MAC addresses, and qemu-guest-agent IP discovery. Useful +overrides: + +```bash +LAB_PIMOX_PIPELINE=false ./lab.sh up +LAB_PIMOX_WORKER_COUNT=0 ./lab.sh up +LAB_PIMOX_WORKER_COUNT=2 ./lab.sh up +LAB_PIMOX_WORKER_BASE_VMID=9020 ./lab.sh up +LAB_PIMOX_HOST=192.168.100.80 LAB_PIMOX_BRIDGE=vmbr0 ./lab.sh up +``` diff --git a/bootstrap/provisioning/main.tf b/bootstrap/provisioning/main.tf index e51ecd2..cd783ba 100644 --- a/bootstrap/provisioning/main.tf +++ b/bootstrap/provisioning/main.tf @@ -123,10 +123,12 @@ resource "null_resource" "pimox_template_vm_create" { pimox_host = var.pimox_host pimox_user = var.pimox_user ssh_key_path = var.pimox_ssh_key_path + builder_version = "2" vmid = tostring(var.pimox_template_vmid) name = var.pimox_template_name cores = tostring(var.pimox_template_cores) memory = tostring(var.pimox_template_memory) + bridge = var.pimox_template_bridge net0 = local.pimox_template_net0 scsi0 = var.pimox_template_scsi0 efidisk0 = var.pimox_template_efidisk0 @@ -153,8 +155,14 @@ if ! command -v qm >/dev/null 2>&1; then exit 1 fi +if ! ip link show "${self.triggers.bridge}" >/dev/null 2>&1; then + echo "Pimox bridge ${self.triggers.bridge} does not exist. Refusing to change Orange Pi networking." >&2 + exit 1 +fi + if sudo qm status "$vmid" >/dev/null 2>&1; then if sudo qm config "$vmid" | grep -q '^template: 1$'; then + sudo qm set "$vmid" --agent enabled=1 exit 0 fi if [ "$replace_existing" != "true" ]; then @@ -184,10 +192,12 @@ sudo qm create "$vmid" \ --ostype l26 \ --scsihw virtio-scsi-pci \ --sockets 1 \ - --vga virtio + --vga virtio \ + --agent enabled=1 sudo qm set "$vmid" --efidisk0 "${self.triggers.efidisk0}" sudo qm set "$vmid" --scsi0 "${self.triggers.scsi0}" +sudo qm set "$vmid" --agent enabled=1 sudo qm start "$vmid" EOT ] @@ -200,35 +210,107 @@ resource "null_resource" "pimox_template_vm_seal" { depends_on = [null_resource.pimox_template_vm_create] triggers = { - host = var.pimox_template_build_host - user = var.pimox_template_build_user - ssh_key_path = var.pimox_template_build_ssh_key_path - timeout = var.pimox_template_build_timeout - vmid = tostring(var.pimox_template_vmid) + pimox_host = var.pimox_host + pimox_user = var.pimox_user + pimox_key_path = var.pimox_ssh_key_path + guest_host = var.pimox_template_build_host + guest_user = var.pimox_template_build_user + guest_key_path = var.pimox_template_build_ssh_key_path + seal_version = "2" + timeout = var.pimox_template_build_timeout + timeout_seconds = tostring(var.pimox_template_build_timeout_seconds) + guest_ip_prefix = var.pimox_template_guest_ip_prefix + vmid = tostring(var.pimox_template_vmid) } - connection { - type = "ssh" - user = self.triggers.user - private_key = file(self.triggers.ssh_key_path) - host = self.triggers.host - timeout = self.triggers.timeout - } + provisioner "local-exec" { + interpreter = ["/bin/bash", "-lc"] + command = <&2 +if ! command -v python3 >/dev/null 2>&1; then + echo "python3 is required to discover the Pimox guest IP from qemu-guest-agent" >&2 exit 1 fi -sudo /usr/local/sbin/homelab-prepare-template.sh -sudo nohup sh -c 'sleep 2; poweroff' >/dev/null 2>&1 & +ssh_pimox() { + ssh -i "$pimox_key" -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "$pimox_user@$pimox_host" "$@" +} + +ssh_guest() { + ssh -i "$guest_key" -o BatchMode=yes -o ConnectTimeout=8 -o StrictHostKeyChecking=accept-new "$guest_user@$guest_host" "$@" +} + +guest_ip_from_agent() { + guest_json="$(ssh_pimox "sudo qm guest cmd '$vmid' network-get-interfaces" 2>/dev/null || true)" + if [ -z "$guest_json" ]; then + return 1 + fi + GUEST_JSON="$guest_json" python3 - "$guest_ip_prefix" <<'PY' +import json +import os +import sys + +prefix = sys.argv[1] +try: + interfaces = json.loads(os.environ.get("GUEST_JSON", "")) +except Exception: + sys.exit(1) + +for iface in interfaces or []: + for address in iface.get("ip-addresses") or []: + if address.get("ip-address-type") != "ipv4": + continue + ip = address.get("ip-address", "") + if not ip or ip.startswith(("127.", "169.254.")): + continue + if prefix and not ip.startswith(prefix): + continue + print(ip) + sys.exit(0) +sys.exit(1) +PY +} + +if ssh_pimox "sudo qm config '$vmid' | grep -q '^template: 1$'"; then + ssh_pimox "sudo qm set '$vmid' --agent enabled=1" + exit 0 +fi + +deadline=$((SECONDS + timeout_seconds)) +while (( SECONDS < deadline )); do + if [ -z "$guest_host" ]; then + guest_host="$(guest_ip_from_agent || true)" + fi + if [ -n "$guest_host" ] && ssh_guest "test -x /usr/local/sbin/homelab-prepare-template.sh"; then + break + fi + sleep 15 +done + +if [ -z "$guest_host" ]; then + echo "Timed out waiting for VM $vmid to report a guest IP through qemu-guest-agent" >&2 + exit 1 +fi + +if ! ssh_guest "test -x /usr/local/sbin/homelab-prepare-template.sh"; then + echo "Timed out waiting for SSH on template-build VM $vmid at $guest_host" >&2 + exit 1 +fi + +ssh_guest "sudo /usr/local/sbin/homelab-prepare-template.sh" +ssh_guest "sudo nohup sh -c 'sleep 2; poweroff' >/dev/null 2>&1 &" || true EOT - ] } } @@ -238,10 +320,11 @@ resource "null_resource" "pimox_template_vm_finalize" { depends_on = [null_resource.pimox_template_vm_seal] triggers = { - pimox_host = var.pimox_host - pimox_user = var.pimox_user - ssh_key_path = var.pimox_ssh_key_path - vmid = tostring(var.pimox_template_vmid) + pimox_host = var.pimox_host + pimox_user = var.pimox_user + ssh_key_path = var.pimox_ssh_key_path + finalizer_version = "2" + vmid = tostring(var.pimox_template_vmid) } connection { @@ -257,6 +340,11 @@ resource "null_resource" "pimox_template_vm_finalize" { set -eu vmid="${self.triggers.vmid}" +if sudo qm config "$vmid" | grep -q '^template: 1$'; then + sudo qm set "$vmid" --agent enabled=1 + exit 0 +fi + elapsed=0 while [ "$elapsed" -lt 600 ]; do if sudo qm status "$vmid" | grep -q 'status: stopped'; then @@ -280,17 +368,18 @@ EOT resource "null_resource" "provisioning_host" { triggers = { - host = var.provisioning_host - user = var.provisioning_user - ssh_key_path = var.provisioning_ssh_key_path - install_dir = var.provisioning_install_dir - tftp_root = local.tftp_root - http_root = local.http_root - netboot_base_url = local.debian_netboot_base_url - pxe_boot_file = var.pxe_boot_file - http_port = tostring(var.http_port) - config_hash = local.config_hash - provisioning_layer = "1" + host = var.provisioning_host + user = var.provisioning_user + ssh_key_path = var.provisioning_ssh_key_path + install_dir = var.provisioning_install_dir + tftp_root = local.tftp_root + http_root = local.http_root + netboot_base_url = local.debian_netboot_base_url + pxe_boot_file = var.pxe_boot_file + http_port = tostring(var.http_port) + config_hash = local.config_hash + provisioning_layer = "1" + provisioner_version = "2" } connection { @@ -362,7 +451,7 @@ install_missing_packages() { fi } -install_missing_packages ca-certificates curl dnsmasq nginx +install_missing_packages ca-certificates curl dnsmasq nginx python3 sudo install -d -m 0755 \ "$install_dir" \ diff --git a/bootstrap/provisioning/variables.tf b/bootstrap/provisioning/variables.tf index 3d0d67d..2322559 100644 --- a/bootstrap/provisioning/variables.tf +++ b/bootstrap/provisioning/variables.tf @@ -161,7 +161,7 @@ variable "pimox_template_builder_enabled" { variable "pimox_host" { type = string - default = "192.168.100.91" + default = "192.168.100.80" } variable "pimox_user" { @@ -238,3 +238,13 @@ variable "pimox_template_build_timeout" { type = string default = "60m" } + +variable "pimox_template_build_timeout_seconds" { + type = number + default = 3600 +} + +variable "pimox_template_guest_ip_prefix" { + type = string + default = "192.168.100." +} diff --git a/lab.sh b/lab.sh index 551a089..0c0750e 100755 --- a/lab.sh +++ b/lab.sh @@ -28,9 +28,352 @@ require_debian_server() { run_tofu_stack() { local stack="$1" + local -a apply_args=(-auto-approve) + + if [[ "${stack}" == "bootstrap/cluster" && -n "${LAB_CLUSTER_VAR_FILE:-}" ]]; then + apply_args+=("-var-file=${LAB_CLUSTER_VAR_FILE}") + fi tofu -chdir="${REPO_ROOT}/${stack}" init - tofu -chdir="${REPO_ROOT}/${stack}" apply -auto-approve + tofu -chdir="${REPO_ROOT}/${stack}" apply "${apply_args[@]}" +} + +truthy() { + case "${1,,}" in + 1 | true | yes | on) + return 0 + ;; + *) + return 1 + ;; + esac +} + +disabled_value() { + case "${1,,}" in + 0 | false | no | off | disabled) + return 0 + ;; + *) + return 1 + ;; + esac +} + +ensure_python3() { + if command -v python3 >/dev/null 2>&1; then + return 0 + fi + + sudo apt-get update + sudo apt-get install -y --no-install-recommends python3 +} + +detect_route_interface() { + local target="$1" + + ip route get "${target}" 2>/dev/null | awk ' + { + for (i = 1; i <= NF; i++) { + if ($i == "dev") { + print $(i + 1) + exit + } + } + } + ' +} + +pimox_ssh() { + local host="$1" + local user="$2" + local key_path="$3" + + shift 3 + ssh -i "${key_path}" -o BatchMode=yes -o ConnectTimeout=10 -o StrictHostKeyChecking=accept-new "${user}@${host}" "$@" +} + +pimox_guest_ipv4() { + local guest_json + local host="$1" + local user="$2" + local key_path="$3" + local vmid="$4" + local ip_prefix="$5" + + guest_json="$(pimox_ssh "${host}" "${user}" "${key_path}" "sudo qm guest cmd '${vmid}' network-get-interfaces" 2>/dev/null || true)" + if [[ -z "${guest_json}" ]]; then + return 1 + fi + + GUEST_JSON="${guest_json}" python3 - "${ip_prefix}" <<'PY' +import json +import os +import sys + +prefix = sys.argv[1] +try: + interfaces = json.loads(os.environ.get("GUEST_JSON", "")) +except Exception: + sys.exit(1) + +for iface in interfaces or []: + for address in iface.get("ip-addresses") or []: + if address.get("ip-address-type") != "ipv4": + continue + ip = address.get("ip-address", "") + if not ip or ip.startswith(("127.", "169.254.")): + continue + if prefix and not ip.startswith(prefix): + continue + print(ip) + sys.exit(0) +sys.exit(1) +PY +} + +wait_for_pimox_guest_ssh() { + local host="$1" + local user="$2" + local key_path="$3" + local vmid="$4" + local guest_user="$5" + local guest_key_path="$6" + local ip_prefix="$7" + local timeout_seconds="$8" + local deadline + local guest_ip + + deadline=$((SECONDS + timeout_seconds)) + while ((SECONDS < deadline)); do + guest_ip="$(pimox_guest_ipv4 "${host}" "${user}" "${key_path}" "${vmid}" "${ip_prefix}" || true)" + if [[ -n "${guest_ip}" ]] && + ssh -i "${guest_key_path}" -o BatchMode=yes -o ConnectTimeout=8 -o StrictHostKeyChecking=accept-new "${guest_user}@${guest_ip}" true >/dev/null 2>&1; then + printf '%s\n' "${guest_ip}" + return 0 + fi + sleep 10 + done + + return 1 +} + +pimox_generated_mac() { + local vmid="$1" + + printf '02:68:10:%02x:%02x:%02x\n' \ + $(((vmid >> 16) & 255)) \ + $(((vmid >> 8) & 255)) \ + $((vmid & 255)) +} + +ensure_pimox_worker_node() { + local index="$1" + local spec_file="$2" + local pimox_host="$3" + local pimox_user="$4" + local pimox_key="$5" + local template_vmid="$6" + local bridge="$7" + local worker_base_vmid="$8" + local worker_name_prefix="$9" + local worker_node_prefix="${10}" + local worker_key_prefix="${11}" + local worker_cores="${12}" + local worker_memory="${13}" + local worker_user="${14}" + local worker_key_path="${15}" + local ip_prefix="${16}" + local timeout_seconds="${17}" + local padded + local vmid + local worker_key + local worker_name + local node_name + local mac + local guest_ip + + printf -v padded '%02d' "${index}" + vmid=$((worker_base_vmid + index - 1)) + worker_key="${worker_key_prefix}${padded}" + worker_name="${worker_name_prefix}-${padded}" + node_name="${worker_node_prefix}-${padded}" + mac="$(pimox_generated_mac "${vmid}")" + + if pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "sudo qm status '${vmid}' >/dev/null 2>&1"; then + if pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "sudo qm config '${vmid}' | grep -q '^template: 1$'"; then + echo "VM ${vmid} exists as a template; refusing to reuse it as worker ${worker_name}." >&2 + exit 1 + fi + pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "sudo qm set '${vmid}' --agent enabled=1 +if sudo qm status '${vmid}' | grep -q 'status: stopped'; then sudo qm start '${vmid}'; fi" + else + pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "set -eu +if ! ip link show '${bridge}' >/dev/null 2>&1; then + echo 'Pimox bridge ${bridge} does not exist. Refusing to change Orange Pi networking.' >&2 + exit 1 +fi +sudo qm clone '${template_vmid}' '${vmid}' --name '${worker_name}' --full 1 +sudo qm set '${vmid}' --agent enabled=1 +sudo qm set '${vmid}' --cores '${worker_cores}' --memory '${worker_memory}' +sudo qm set '${vmid}' --net0 'virtio=${mac},bridge=${bridge}' +sudo qm set '${vmid}' --boot 'order=scsi0;net0' +sudo qm set '${vmid}' --onboot 1 +sudo qm start '${vmid}'" + fi + + if ! guest_ip="$(wait_for_pimox_guest_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "${vmid}" "${worker_user}" "${worker_key_path}" "${ip_prefix}" "${timeout_seconds}")"; then + echo "Timed out waiting for worker VM ${vmid} (${worker_name}) to report a reachable guest IP." >&2 + exit 1 + fi + + printf '%s\t%s\t%s\t%s\t%s\n' "${worker_key}" "${guest_ip}" "${worker_user}" "${node_name}" "${worker_key_path}" >>"${spec_file}" +} + +write_cluster_worker_var_file() { + local spec_file="$1" + local var_file="$2" + + LAB_INCLUDE_RASPBERRY_WORKER="${LAB_INCLUDE_RASPBERRY_WORKER:-true}" \ + LAB_RASPBERRY_HOST="${LAB_RASPBERRY_HOST:-192.168.100.89}" \ + LAB_RASPBERRY_USER="${LAB_RASPBERRY_USER:-jv}" \ + LAB_RASPBERRY_NODE_NAME="${LAB_RASPBERRY_NODE_NAME:-raspberry}" \ + LAB_RASPBERRY_SSH_KEY_PATH="${LAB_RASPBERRY_SSH_KEY_PATH:-/home/jv/.ssh/id_ed25519}" \ + python3 - "${spec_file}" "${var_file}" <<'PY' +import json +import os +import sys + +spec_file, var_file = sys.argv[1:3] +nodes = {} + +if os.environ["LAB_INCLUDE_RASPBERRY_WORKER"].lower() not in {"0", "false", "no", "off", "disabled"}: + nodes["raspberrypi"] = { + "host": os.environ["LAB_RASPBERRY_HOST"], + "user": os.environ["LAB_RASPBERRY_USER"], + "node_name": os.environ["LAB_RASPBERRY_NODE_NAME"], + "ssh_key_path": os.environ["LAB_RASPBERRY_SSH_KEY_PATH"], + } + +with open(spec_file, encoding="utf-8") as handle: + for line in handle: + line = line.rstrip("\n") + if not line: + continue + key, host, user, node_name, ssh_key_path = line.split("\t") + nodes[key] = { + "host": host, + "user": user, + "node_name": node_name, + "ssh_key_path": ssh_key_path, + } + +with open(var_file, "w", encoding="utf-8") as handle: + json.dump({"worker_nodes": nodes}, handle, indent=2) + handle.write("\n") +PY +} + +run_pimox_pipeline() { + local mode="${LAB_PIMOX_PIPELINE:-auto}" + local pimox_host="${LAB_PIMOX_HOST:-${TF_VAR_pimox_host:-192.168.100.80}}" + local pimox_user="${LAB_PIMOX_USER:-${TF_VAR_pimox_user:-jv}}" + local pimox_key="${LAB_PIMOX_SSH_KEY_PATH:-${TF_VAR_pimox_ssh_key_path:-/home/jv/.ssh/id_ed25519}}" + local bridge="${LAB_PIMOX_BRIDGE:-${TF_VAR_pimox_template_bridge:-vmbr0}}" + local template_vmid="${LAB_PIMOX_TEMPLATE_VMID:-${TF_VAR_pimox_template_vmid:-9000}}" + local template_name="${LAB_PIMOX_TEMPLATE_NAME:-${TF_VAR_pimox_template_name:-debian13-arm64-k8s-template}}" + local provisioning_interface + local worker_count="${LAB_PIMOX_WORKER_COUNT:-1}" + local worker_base_vmid="${LAB_PIMOX_WORKER_BASE_VMID:-9010}" + local worker_name_prefix="${LAB_PIMOX_WORKER_NAME_PREFIX:-pimox-worker}" + local worker_node_prefix="${LAB_PIMOX_WORKER_NODE_PREFIX:-pimox-worker}" + local worker_key_prefix="${LAB_PIMOX_WORKER_KEY_PREFIX:-pimox}" + local worker_cores="${LAB_PIMOX_WORKER_CORES:-2}" + local worker_memory="${LAB_PIMOX_WORKER_MEMORY:-2048}" + local worker_user="${LAB_PIMOX_WORKER_USER:-jv}" + local worker_key_path="${LAB_PIMOX_WORKER_SSH_KEY_PATH:-/home/jv/.ssh/id_ed25519}" + local ip_prefix="${LAB_PIMOX_GUEST_IP_PREFIX:-192.168.100.}" + local timeout_seconds="${LAB_PIMOX_GUEST_TIMEOUT_SECONDS:-3600}" + local spec_file="${REPO_ROOT}/.lab/pimox-workers.tsv" + local var_file="${REPO_ROOT}/.lab/cluster-workers.auto.tfvars.json" + local index + + if disabled_value "${mode}"; then + return 0 + fi + + if ! [[ "${worker_count}" =~ ^[0-9]+$ ]]; then + echo "LAB_PIMOX_WORKER_COUNT must be a non-negative integer." >&2 + exit 1 + fi + + if ! pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "command -v qm >/dev/null 2>&1 && ip link show '${bridge}' >/dev/null 2>&1"; then + if [[ "${mode}" == "auto" ]]; then + echo "Skipping Pimox automation because ${pimox_user}@${pimox_host} with bridge ${bridge} is not ready." + return 0 + fi + echo "Pimox automation requested, but ${pimox_user}@${pimox_host} is not reachable or bridge ${bridge} is missing." >&2 + exit 1 + fi + + ensure_python3 + provisioning_interface="${TF_VAR_provisioning_interface:-${LAB_PROVISIONING_INTERFACE:-$(detect_route_interface "${pimox_host}")}}" + if [[ -z "${provisioning_interface}" ]]; then + echo "Could not detect the Debian interface used to reach ${pimox_host}; set LAB_PROVISIONING_INTERFACE." >&2 + exit 1 + fi + + export TF_VAR_provisioning_interface="${provisioning_interface}" + export TF_VAR_pimox_host="${pimox_host}" + export TF_VAR_pimox_user="${pimox_user}" + export TF_VAR_pimox_ssh_key_path="${pimox_key}" + export TF_VAR_pimox_template_bridge="${bridge}" + export TF_VAR_pimox_template_vmid="${template_vmid}" + export TF_VAR_pimox_template_name="${template_name}" + export TF_VAR_pimox_template_builder_enabled="${TF_VAR_pimox_template_builder_enabled:-true}" + export TF_VAR_pimox_template_build_ssh_key_path="${TF_VAR_pimox_template_build_ssh_key_path:-${worker_key_path}}" + export TF_VAR_pimox_template_build_user="${TF_VAR_pimox_template_build_user:-${worker_user}}" + export TF_VAR_pimox_template_guest_ip_prefix="${TF_VAR_pimox_template_guest_ip_prefix:-${ip_prefix}}" + export TF_VAR_pimox_template_build_timeout_seconds="${TF_VAR_pimox_template_build_timeout_seconds:-${timeout_seconds}}" + + echo "Preparing Pimox provisioning and Debian worker template on ${pimox_host} without changing Orange Pi host networking..." + run_tofu_stack "bootstrap/provisioning" + + if ((worker_count == 0)); then + return 0 + fi + + if ! pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "sudo qm config '${template_vmid}' | grep -q '^template: 1$'"; then + echo "Template VM ${template_vmid} is not available as a Pimox template after provisioning." >&2 + exit 1 + fi + pimox_ssh "${pimox_host}" "${pimox_user}" "${pimox_key}" "sudo qm set '${template_vmid}' --agent enabled=1" + + mkdir -p "${REPO_ROOT}/.lab" + : >"${spec_file}" + for ((index = 1; index <= worker_count; index++)); do + ensure_pimox_worker_node \ + "${index}" \ + "${spec_file}" \ + "${pimox_host}" \ + "${pimox_user}" \ + "${pimox_key}" \ + "${template_vmid}" \ + "${bridge}" \ + "${worker_base_vmid}" \ + "${worker_name_prefix}" \ + "${worker_node_prefix}" \ + "${worker_key_prefix}" \ + "${worker_cores}" \ + "${worker_memory}" \ + "${worker_user}" \ + "${worker_key_path}" \ + "${ip_prefix}" \ + "${timeout_seconds}" + done + + write_cluster_worker_var_file "${spec_file}" "${var_file}" + export LAB_CLUSTER_VAR_FILE="${var_file}" } cleanup_calico_links() { @@ -668,6 +1011,7 @@ up() { echo "Deploying the homelab infrastructure..." + run_pimox_pipeline run_tofu_stack "bootstrap/cluster" run_tofu_stack "bootstrap/platform" apply_gitea_bootstrap_manifests