Document translation workflow architecture
This commit is contained in:
parent
d5abeca4b1
commit
b45cd2d508
12
README.md
12
README.md
|
|
@ -652,16 +652,20 @@ cleaned unless you explicitly include it.
|
||||||
## Website App
|
## Website App
|
||||||
|
|
||||||
The website is a PHP app under `apps/website`. It includes a home page, CV page,
|
The website is a PHP app under `apps/website`. It includes a home page, CV page,
|
||||||
blog page, and demos page, plus a lightweight translation flow backed by Ollama.
|
blog page, and demos page, plus a lightweight translation flow backed by Redis,
|
||||||
|
n8n, and Ollama.
|
||||||
Static language files live in `apps/website/lang`; `en.php` and `nah.php` are
|
Static language files live in `apps/website/lang`; `en.php` and `nah.php` are
|
||||||
curated source files, with the Nahuatl home page intentionally biased toward as
|
curated source files, with the Nahuatl home page intentionally biased toward as
|
||||||
many Nahuatl words as possible while keeping technical terms understandable.
|
many Nahuatl words as possible while keeping technical terms understandable.
|
||||||
Unsupported browser languages use the same-origin `/translate.php` endpoint,
|
Unsupported browser languages use the same-origin `/translate.php` endpoint,
|
||||||
which calls Ollama server-side through `OLLAMA_HOST` and `OLLAMA_MODEL`; the
|
which calls Ollama server-side through `OLLAMA_HOST` and `OLLAMA_MODEL`; the
|
||||||
browser never calls the private Ollama IP directly. The default model is the
|
browser never calls the private Ollama IP directly. Redis caches per-string
|
||||||
custom `website-translator` Ollama model defined in
|
translation results in the `website-production` namespace, while n8n is
|
||||||
|
available for translation prewarming and batch workflow jobs. The default model
|
||||||
|
is the custom `website-translator` Ollama model defined in
|
||||||
`apps/website/ollama/Modelfile`. Generated runtime language JSON is saved
|
`apps/website/ollama/Modelfile`. Generated runtime language JSON is saved
|
||||||
through `save_lang.php` on the website PVC.
|
through `save_lang.php` on the website PVC, and `translate.php` emits structured
|
||||||
|
logs for cache hits, misses, Ollama latency, JSON parse failures, and timeouts.
|
||||||
|
|
||||||
Create or refresh the Ollama model on the Debian server before deploying a
|
Create or refresh the Ollama model on the Debian server before deploying a
|
||||||
website image that points at it:
|
website image that points at it:
|
||||||
|
|
|
||||||
|
|
@ -55,6 +55,18 @@ The OCI edge nginx layer handles gzip, cache HITs, HTTP/2, TLS session reuse,
|
||||||
client keepalive, and upstream keepalive to the Traefik backend. CSS and
|
client keepalive, and upstream keepalive to the Traefik backend. CSS and
|
||||||
JavaScript keep the edge's longer immutable asset policy.
|
JavaScript keep the edge's longer immutable asset policy.
|
||||||
|
|
||||||
|
## Translation Flow
|
||||||
|
|
||||||
|
Unsupported browser languages call the same-origin `translate.php` endpoint.
|
||||||
|
The endpoint sanitizes and batches text, checks Redis per string, calls the
|
||||||
|
custom `website-translator` Ollama model only for cache misses, writes fresh
|
||||||
|
translations back to Redis with a TTL, then returns results in the original
|
||||||
|
order. The browser never calls the private Ollama address directly.
|
||||||
|
|
||||||
|
n8n is deployed separately at `https://n8n.lab2025.duckdns.org` and receives
|
||||||
|
`OLLAMA_BASE_URL=http://192.168.100.68:11434` so workflows can prewarm
|
||||||
|
translations or run batch jobs against the Debian-host Ollama API.
|
||||||
|
|
||||||
## Translation Observability
|
## Translation Observability
|
||||||
|
|
||||||
`translate.php` emits one structured JSON log event for each valid translation
|
`translate.php` emits one structured JSON log event for each valid translation
|
||||||
|
|
|
||||||
|
|
@ -57,6 +57,7 @@ $stackKeys = [
|
||||||
'blog_stack_16',
|
'blog_stack_16',
|
||||||
'blog_stack_17',
|
'blog_stack_17',
|
||||||
'blog_stack_18',
|
'blog_stack_18',
|
||||||
|
'blog_stack_19',
|
||||||
];
|
];
|
||||||
|
|
||||||
$caseStudyCards = [
|
$caseStudyCards = [
|
||||||
|
|
@ -174,6 +175,12 @@ $stackSourceLinks = [
|
||||||
['label' => 'Cosign build flow', 'path' => 'lab.sh'],
|
['label' => 'Cosign build flow', 'path' => 'lab.sh'],
|
||||||
['label' => 'apps variables', 'path' => 'bootstrap/apps/variables.tf'],
|
['label' => 'apps variables', 'path' => 'bootstrap/apps/variables.tf'],
|
||||||
],
|
],
|
||||||
|
'blog_stack_19' => [
|
||||||
|
['label' => 'translate.php', 'path' => 'apps/website/translate.php'],
|
||||||
|
['label' => 'website Redis manifest', 'path' => 'apps/website/web-app.yaml'],
|
||||||
|
['label' => 'n8n app manifest', 'path' => 'apps/n8n/web-app.yaml'],
|
||||||
|
['label' => 'Ollama Modelfile', 'path' => 'apps/website/ollama/Modelfile'],
|
||||||
|
],
|
||||||
];
|
];
|
||||||
|
|
||||||
function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $sourceBase): void {
|
function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $sourceBase): void {
|
||||||
|
|
@ -310,7 +317,7 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<div class="diagram-shell" aria-label="Professional homelab architecture diagram">
|
<div class="diagram-shell" aria-label="Professional homelab architecture diagram">
|
||||||
<svg class="homelab-map" viewBox="0 0 1120 960" role="img" aria-labelledby="homelab-map-title homelab-map-desc">
|
<svg class="homelab-map" viewBox="0 0 1120 960" role="img" aria-labelledby="homelab-map-title homelab-map-desc">
|
||||||
<title id="homelab-map-title">Homelab architecture map</title>
|
<title id="homelab-map-title">Homelab architecture map</title>
|
||||||
<desc id="homelab-map-desc">Git push enters Gitea, Gitea Actions validates and builds app images, OpenTofu manages cluster and provisioning layers, Debian keeps the control plane and PXE services, Pimox app workers run Argo CD, Kyverno, and app workloads on NVMe-backed VMs, OpenWrt can run as an opt-in firewall VM, and the OCI edge routes traffic into Kubernetes services.</desc>
|
<desc id="homelab-map-desc">Git push enters Gitea, Gitea Actions validates and builds app images, OpenTofu manages cluster and provisioning layers, Debian keeps the control plane, PXE services, and Ollama translation model, Pimox app workers run Argo CD, Kyverno, Redis-backed website translation, n8n workflow automation, and app workloads on NVMe-backed VMs, OpenWrt can run as an opt-in firewall VM, and the OCI edge routes traffic into Kubernetes services.</desc>
|
||||||
<defs>
|
<defs>
|
||||||
<marker id="map-arrow" markerWidth="12" markerHeight="12" refX="10" refY="6" orient="auto">
|
<marker id="map-arrow" markerWidth="12" markerHeight="12" refX="10" refY="6" orient="auto">
|
||||||
<path d="M2,2 L10,6 L2,10 Z" fill="#2b6cb0"></path>
|
<path d="M2,2 L10,6 L2,10 Z" fill="#2b6cb0"></path>
|
||||||
|
|
@ -405,8 +412,8 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<g class="diagram-node node-accent-orange" transform="translate(412 774)">
|
<g class="diagram-node node-accent-orange" transform="translate(412 774)">
|
||||||
<rect width="280" height="82" rx="8"></rect>
|
<rect width="280" height="82" rx="8"></rect>
|
||||||
<text x="18" y="27">Storage and backups</text>
|
<text x="18" y="27">Storage and backups</text>
|
||||||
<text class="diagram-small" x="18" y="50">OpenEBS retained PVs</text>
|
<text class="diagram-small" x="18" y="50">OpenEBS retained PVs, n8n data</text>
|
||||||
<text class="diagram-small" x="18" y="68">Gitea dumps and monitoring data</text>
|
<text class="diagram-small" x="18" y="68">Gitea dumps, logs, Redis cache</text>
|
||||||
</g>
|
</g>
|
||||||
|
|
||||||
<g class="diagram-node node-accent-blue" transform="translate(800 92)">
|
<g class="diagram-node node-accent-blue" transform="translate(800 92)">
|
||||||
|
|
@ -437,8 +444,8 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<text x="18" y="28">Orange Pi 5 Plus Pimox</text>
|
<text x="18" y="28">Orange Pi 5 Plus Pimox</text>
|
||||||
<text class="diagram-small" x="18" y="52">pimox-worker app nodes</text>
|
<text class="diagram-small" x="18" y="52">pimox-worker app nodes</text>
|
||||||
<text class="diagram-small" x="18" y="70">workers on nvme_thin_pool</text>
|
<text class="diagram-small" x="18" y="70">workers on nvme_thin_pool</text>
|
||||||
<text class="diagram-small" x="18" y="88">Argo CD, Kyverno, apps</text>
|
<text class="diagram-small" x="18" y="88">Argo CD, Kyverno, website, n8n</text>
|
||||||
<text class="diagram-small" x="18" y="106">idempotent qm automation</text>
|
<text class="diagram-small" x="18" y="106">Redis cache and app workloads</text>
|
||||||
</g>
|
</g>
|
||||||
|
|
||||||
<g class="diagram-node node-accent-orange" transform="translate(800 616)">
|
<g class="diagram-node node-accent-orange" transform="translate(800 616)">
|
||||||
|
|
@ -457,6 +464,12 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<text class="diagram-small" x="18" y="68">pulled by app workers</text>
|
<text class="diagram-small" x="18" y="68">pulled by app workers</text>
|
||||||
</g>
|
</g>
|
||||||
|
|
||||||
|
<g class="diagram-node node-accent-purple" transform="translate(800 866)">
|
||||||
|
<rect width="258" height="54" rx="8"></rect>
|
||||||
|
<text x="18" y="24">Ollama translation API</text>
|
||||||
|
<text class="diagram-small" x="18" y="44">website-translator on Debian</text>
|
||||||
|
</g>
|
||||||
|
|
||||||
<path class="diagram-link" d="M174 174 L174 208"></path>
|
<path class="diagram-link" d="M174 174 L174 208"></path>
|
||||||
<path class="diagram-link" d="M174 290 L174 318"></path>
|
<path class="diagram-link" d="M174 290 L174 318"></path>
|
||||||
<path class="diagram-link" d="M174 400 L174 430"></path>
|
<path class="diagram-link" d="M174 400 L174 430"></path>
|
||||||
|
|
@ -476,6 +489,7 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<path class="diagram-link" d="M929 188 L929 218"></path>
|
<path class="diagram-link" d="M929 188 L929 218"></path>
|
||||||
<path class="diagram-link" d="M929 300 L929 330"></path>
|
<path class="diagram-link" d="M929 300 L929 330"></path>
|
||||||
<path class="diagram-link" d="M929 766 L929 426"></path>
|
<path class="diagram-link" d="M929 766 L929 426"></path>
|
||||||
|
<path class="diagram-link" d="M800 893 C724 893 720 367 692 367"></path>
|
||||||
|
|
||||||
<text class="diagram-link-label" x="184" y="198">push</text>
|
<text class="diagram-link-label" x="184" y="198">push</text>
|
||||||
<text class="diagram-link-label" x="196" y="312">workflow</text>
|
<text class="diagram-link-label" x="196" y="312">workflow</text>
|
||||||
|
|
@ -491,6 +505,7 @@ function renderStackSourceLinks(string $stackKey, array $sourceLinks, string $so
|
||||||
<text class="diagram-link-label" x="934" y="212">secure tunnel</text>
|
<text class="diagram-link-label" x="934" y="212">secure tunnel</text>
|
||||||
<text class="diagram-link-label" x="934" y="322">service traffic</text>
|
<text class="diagram-link-label" x="934" y="322">service traffic</text>
|
||||||
<text class="diagram-link-label" x="946" y="754">image pulls</text>
|
<text class="diagram-link-label" x="946" y="754">image pulls</text>
|
||||||
|
<text class="diagram-link-label" x="710" y="860">translation calls</text>
|
||||||
</svg>
|
</svg>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -150,10 +150,11 @@ return [
|
||||||
'blog_stack_16' => 'The monitoring layer now includes Prometheus Stack, Grafana, Loki, Promtail, node-exporter, and kube-state-metrics, with placement guardrails so platform add-ons stop defaulting to the control plane.',
|
'blog_stack_16' => 'The monitoring layer now includes Prometheus Stack, Grafana, Loki, Promtail, node-exporter, and kube-state-metrics, with placement guardrails so platform add-ons stop defaulting to the control plane.',
|
||||||
'blog_stack_17' => 'The portfolio website keeps Apache at the core, runs PHP through PHP-FPM with OPcache, pre-renders read-heavy pages into static HTML during the image build, and leaves PHP for write and translation endpoints.',
|
'blog_stack_17' => 'The portfolio website keeps Apache at the core, runs PHP through PHP-FPM with OPcache, pre-renders read-heavy pages into static HTML during the image build, and leaves PHP for write and translation endpoints.',
|
||||||
'blog_stack_18' => 'The supply-chain path now uses BuildKit SBOM attestations, Cosign signatures, a cluster-published public key, and an audit-first Kyverno ImageValidatingPolicy that can be promoted to deny unsigned local-registry pods.',
|
'blog_stack_18' => 'The supply-chain path now uses BuildKit SBOM attestations, Cosign signatures, a cluster-published public key, and an audit-first Kyverno ImageValidatingPolicy that can be promoted to deny unsigned local-registry pods.',
|
||||||
|
'blog_stack_19' => 'The translation path now uses Redis for per-string cache hits, a custom Ollama website-translator model on the Debian server for cache misses, n8n for prewarming and batch workflow automation, and structured logs for cache, latency, JSON parse, and timeout signals.',
|
||||||
'blog_arch_kicker' => 'Architecture map',
|
'blog_arch_kicker' => 'Architecture map',
|
||||||
'blog_arch_title' => 'The homelab, end to end',
|
'blog_arch_title' => 'The homelab, end to end',
|
||||||
'blog_arch_intro' => 'The current delivery path starts with a push to Gitea, runs local validation, builds arm64 images, syncs the validated commit into the GitOps mirror, and lets Argo CD reconcile from the app workers. The infrastructure path stays manual through lab.sh, including the PXE/Pimox template builder, NVMe-backed worker clones, Kyverno policy placement, and the opt-in OpenWrt firewall VM, while the OCI edge routes public traffic back through the private path.',
|
'blog_arch_intro' => 'The current delivery path starts with a push to Gitea, runs local validation, builds arm64 images, syncs the validated commit into the GitOps mirror, and lets Argo CD reconcile from the app workers. The infrastructure path stays manual through lab.sh, including the PXE/Pimox template builder, NVMe-backed worker clones, Kyverno policy placement, Redis-backed website translation, n8n workflow automation, and the opt-in OpenWrt firewall VM, while the OCI edge routes public traffic back through the private path.',
|
||||||
'blog_arch_caption' => 'The diagram is intentionally operational: it shows the app delivery loop, image flow, provisioning path, worker-placement boundary, monitoring layer, OpenWrt firewall option, and public traffic path without hiding the practical bits that make a small lab behave like a platform.',
|
'blog_arch_caption' => 'The diagram is intentionally operational: it shows the app delivery loop, image flow, provisioning path, worker-placement boundary, monitoring layer, Redis/n8n/Ollama translation path, OpenWrt firewall option, and public traffic path without hiding the practical bits that make a small lab behave like a platform.',
|
||||||
'blog_arch_fun_link' => 'Open the Christmas-tree version',
|
'blog_arch_fun_link' => 'Open the Christmas-tree version',
|
||||||
'blog_activity_kicker' => 'Recent activity log',
|
'blog_activity_kicker' => 'Recent activity log',
|
||||||
'blog_activity_title' => 'What changed since the first build',
|
'blog_activity_title' => 'What changed since the first build',
|
||||||
|
|
|
||||||
|
|
@ -150,10 +150,11 @@ return [
|
||||||
'blog_stack_16' => 'Monitoring layer axcan includes Prometheus Stack, Grafana, Loki, Promtail, node-exporter, ihuan kube-state-metrics, ika placement guardrails para platform add-ons ma amo default to control plane.',
|
'blog_stack_16' => 'Monitoring layer axcan includes Prometheus Stack, Grafana, Loki, Promtail, node-exporter, ihuan kube-state-metrics, ika placement guardrails para platform add-ons ma amo default to control plane.',
|
||||||
'blog_stack_17' => 'Portfolio website keeps Apache at core, runs PHP through PHP-FPM ika OPcache, pre-renders read-heavy pages into static HTML during image build, ihuan leaves PHP para write ihuan translation endpoints.',
|
'blog_stack_17' => 'Portfolio website keeps Apache at core, runs PHP through PHP-FPM ika OPcache, pre-renders read-heavy pages into static HTML during image build, ihuan leaves PHP para write ihuan translation endpoints.',
|
||||||
'blog_stack_18' => 'Supply-chain path axcan uses BuildKit SBOM attestations, Cosign signatures, public key ipan cluster, ihuan audit-first Kyverno ImageValidatingPolicy in huel mopromote para deny unsigned local-registry pods.',
|
'blog_stack_18' => 'Supply-chain path axcan uses BuildKit SBOM attestations, Cosign signatures, public key ipan cluster, ihuan audit-first Kyverno ImageValidatingPolicy in huel mopromote para deny unsigned local-registry pods.',
|
||||||
|
'blog_stack_19' => 'Translation path axcan uses Redis para per-string cache hits, custom Ollama website-translator model ipan Debian server para cache misses, n8n para prewarming ihuan batch workflows, ihuan structured logs para cache, latency, JSON parse, ihuan timeout signals.',
|
||||||
'blog_arch_kicker' => 'Architecture mapa',
|
'blog_arch_kicker' => 'Architecture mapa',
|
||||||
'blog_arch_title' => 'Homelab, end to end',
|
'blog_arch_title' => 'Homelab, end to end',
|
||||||
'blog_arch_intro' => 'Current delivery path starts ika push to Gitea, runs local validation, builds arm64 images, syncs validated commit into GitOps mirror, ihuan lets Argo CD reconcile from app workers. Infrastructure path stays manual through lab.sh, including PXE/Pimox template builder, NVMe-backed worker clones, Kyverno policy placement, ihuan opt-in OpenWrt firewall VM, while OCI edge routes public traffic back through private path.',
|
'blog_arch_intro' => 'Current delivery path starts ika push to Gitea, runs local validation, builds arm64 images, syncs validated commit into GitOps mirror, ihuan lets Argo CD reconcile from app workers. Infrastructure path stays manual through lab.sh, including PXE/Pimox template builder, NVMe-backed worker clones, Kyverno policy placement, Redis-backed website translation, n8n workflow automation, ihuan opt-in OpenWrt firewall VM, while OCI edge routes public traffic back through private path.',
|
||||||
'blog_arch_caption' => 'Diagram intentionally operational: quinextia app delivery loop, image flow, provisioning path, worker-placement boundary, monitoring layer, OpenWrt firewall option, ihuan public traffic path ahmo hiding practical bits in quichihua small lab behave like platform.',
|
'blog_arch_caption' => 'Diagram intentionally operational: quinextia app delivery loop, image flow, provisioning path, worker-placement boundary, monitoring layer, Redis/n8n/Ollama translation path, OpenWrt firewall option, ihuan public traffic path ahmo hiding practical bits in quichihua small lab behave like platform.',
|
||||||
'blog_arch_fun_link' => 'Xiquitta Christmas-tree version',
|
'blog_arch_fun_link' => 'Xiquitta Christmas-tree version',
|
||||||
'blog_activity_kicker' => 'Yancuic activity log',
|
'blog_activity_kicker' => 'Yancuic activity log',
|
||||||
'blog_activity_title' => 'Tlein omopatla since first build',
|
'blog_activity_title' => 'Tlein omopatla since first build',
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue