# Homelab AI Evals

This directory contains deterministic eval cases for Jeannie and the local
homelab RAG workflow. The goal is to test whether explanations and retrieved
context point to useful commands, avoid dangerous actions, and stay grounded in
repo-owned runbooks.

Run the evals locally:

```bash
./jeannie ai-evals run
```

Useful commands:

```bash
./jeannie ai-evals list
./jeannie ai-evals show explain_capacity_missing_resources
./jeannie ai-evals run --case explain_capacity_missing_resources --details
./jeannie ai-evals run --json
```

The harness is intentionally read-only. It does not call an external model by
default and does not mutate the cluster. Cases use fixtures plus allowlisted
repo commands so they are suitable for Gitea Actions.