my-homelab-configs/docs/runbooks/gitea-failures.md

124 lines
2.6 KiB
Markdown

# Gitea Failures Runbook
Use this when Gitea is down, `/git/` returns `502`, Git push/pull fails, login
does not work, or the pipeline cannot access the repository.
## Topology
Gitea is intentionally outside Kubernetes:
- Host: Debian `192.168.100.73`
- Tailscale IP: `100.85.138.30`
- Install directory: `/data/homelab-gitea`
- HTTP: `3000`
- SSH: `32222`
- Public path: `https://lab2025.duckdns.org/git/`
## Quick Triage
Run on the Debian host:
```bash
cd ~/my-homelab-configs
./jeannie status
cd /data/homelab-gitea
sudo docker compose ps
sudo docker compose logs --tail=120 gitea
curl -I http://127.0.0.1:3000/
tailscale ip -4
```
Check the edge path:
```bash
ssh ubuntu@132.145.170.74 'curl -I --max-time 5 http://100.85.138.30:3000/'
curl -I https://lab2025.duckdns.org/git/
```
Check Git SSH from Debian:
```bash
ssh -T -p 32222 git@192.168.100.73 || true
git remote -v
git ls-remote gitea main
```
## Common Cases
Container is stopped:
```bash
cd /data/homelab-gitea
sudo docker compose up -d
sudo docker compose ps
```
Docker or containerd is wedged:
```bash
sudo systemctl restart containerd
sudo systemctl restart docker
cd /data/homelab-gitea
sudo docker compose up -d
```
Gitea works locally but not from OCI edge:
```bash
tailscale ip -4
ssh ubuntu@132.145.170.74 'tailscale status; curl -I --max-time 5 http://100.85.138.30:3000/'
```
If the Debian Tailscale IP changed, update the edge variable and reapply:
```bash
tofu -chdir=bootstrap/edge apply
```
Git HTTPS prompts or fails authentication:
- Prefer SSH for writes from the homelab.
- Keep the Debian working copy remote pointed at the LAN SSH endpoint:
```bash
git remote set-url gitea ssh://git@192.168.100.73:32222/jv/my-homelab-configs.git
git branch --set-upstream-to=gitea/main main
```
Login password is lost:
```bash
cd /data/homelab-gitea
sudo docker exec -it homelab-gitea gitea admin user list --config /data/gitea/conf/app.ini
sudo docker exec -it homelab-gitea gitea admin user change-password \
--config /data/gitea/conf/app.ini \
--username jv \
--password '<new-password>'
```
## Safe Redeploy
Redeploying Gitea should not delete data because the Compose stack uses the
persistent data directory under `/data/homelab-gitea/data`.
```bash
cd ~/my-homelab-configs
./jeannie deploy-gitea
./jeannie bootstrap-gitea-repo
```
## Validation
```bash
curl -I http://127.0.0.1:3000/
curl -I https://lab2025.duckdns.org/git/
git ls-remote gitea main
sudo docker compose -f /data/homelab-gitea/docker-compose.yml ps
```
Expected:
- Local HTTP returns a web response.
- Public `/git/` loads through edge.
- `git ls-remote` returns the `main` ref.