Church Translate, as it runs at the campuses
Live sermon translation into 100+ languages, running entirely on one Mac per campus: audio in, voice detection, Whisper on the GPU, NLLB-200 in its own process, and a built-in web server the congregation reads from on their phones. No audio leaves the building. This is the source only. What used to sit beside it — 14 GB of speech and translation models, 8 GB of voice bundles, 6 GB of natural voices, the Electron caches, and the app data holding the operator PIN and every service transcript — is machine state and downloaded assets, and stays in the NAS backup where it belongs. The vendored node, cloudflared and Windows ffmpeg binaries are out too (533 MB, re-fetched for packaging); the browser-side runtime beside them is kept, so a checkout still serves the congregation page. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,105 @@
|
||||
# Plan — Bible-faithful translation, "bulletproof for live preaching"
|
||||
|
||||
## The complaint
|
||||
Operator, watching the Spanish listener page during a live test:
|
||||
- "why is the voice asking questions to me? it keeps saying '¿Qué es eso?'"
|
||||
- "if i say 'john' it must be translated properly"
|
||||
- "make this bulletproof so native speakers in any language can completely
|
||||
understand what is being said in english. this will primarily be used for
|
||||
live preaching of the christian bible."
|
||||
|
||||
## Measured, not assumed
|
||||
Ran `Xenova/nllb-200-distilled-600M` fp32 (the shipped model, shipped dtype,
|
||||
shipped cache) against the exact lines from the screenshot plus stress cases,
|
||||
under four generation configs. Results:
|
||||
|
||||
| English in | Spanish out | verdict |
|
||||
|---|---|---|
|
||||
| `Donald Trump.` | `Donald Trump. ¿Qué es eso?` | invented question |
|
||||
| `Amen.` | `- Sí, estoy bien.` | "Yes, I'm fine." — catastrophic |
|
||||
| `Hello, hello.` | `- Hola, está bien.` | invented content + subtitle dash |
|
||||
| `His name was John.` | `Se llamaba John.` | name left in English |
|
||||
| `Turn with me to John chapter 3.` | `…a Juan capítulo 3.` | same name, translated |
|
||||
| `Open up to the book, Matthew.` | `Abre al libro, Matthew.` | name left in English |
|
||||
|
||||
**The decisive finding:** `¿Qué es eso?` appears under ALL FOUR configs —
|
||||
no params, `max_new_tokens` cap, `no_repeat_ngram_size:3`, and `num_beams:2`.
|
||||
It is not a decoding-budget artifact. Generation parameters cannot fix it.
|
||||
Post-processing is the only reliable lever. (My first hypothesis was wrong;
|
||||
the probe is why this plan is not built on it.)
|
||||
|
||||
Secondary finding: cap + `no_repeat_ngram_size:3` cut the 11-case suite from
|
||||
5897ms to 3924ms. Strictly faster AND safer, so it goes in regardless.
|
||||
|
||||
## Existing machinery to build on (do not reinvent)
|
||||
- `applyTable()` / `termMatcher()` in engine.js — a single-pass, longest-first
|
||||
find-and-replace that already survived three real bugs: terms ending in
|
||||
punctuation, CJK word boundaries, and case-sensitivity ("job" → "Job").
|
||||
- `glossary.fixes` — source-side transcript corrections, operator-editable.
|
||||
- `glossary.overrides[lang]` — target-side per-language pins. Currently `{}`.
|
||||
- Both MT paths converge on `applyOverrides(text, lang)`:
|
||||
`translateSentence()` (live, per sentence) and `translateAll()` (batch).
|
||||
- `translateSentence` receives ONE sentence, so a 1-in-1-out clamp is safe there.
|
||||
|
||||
## The five layers
|
||||
|
||||
### L1 — source-side Bible vocabulary (before MT)
|
||||
Built-in `fixes` for common Whisper mishearings of scripture vocabulary
|
||||
("have a cook" → "Habakkuk" already exists as the only entry). Ship a real
|
||||
list. Must NOT clobber the operator's own fixes.
|
||||
|
||||
### L2 — generation parameters (mt-service.js)
|
||||
`max_new_tokens` derived from input token count, `no_repeat_ngram_size: 3`.
|
||||
Measured: faster and safer. Does NOT fix the hallucination — that is L3.
|
||||
|
||||
### L3 — post-MT hallucination guard ← the fix for the actual complaint
|
||||
Applied to every language, no per-language data needed:
|
||||
1. **Invented-question rule.** Source has no `?`/`？`, output contains `?`/`？`
|
||||
→ the question is hallucinated. Drop the offending clause. This is the
|
||||
direct fix for `¿Qué es eso?`. Precision is high: real scripture questions
|
||||
carry a `?` in the source.
|
||||
2. **Sentence-count clamp.** N sentence terminators in → at most N out
|
||||
(min 1). Kills appended-sentence hallucination generally.
|
||||
3. **Subtitle-dash strip.** NLLB emits leading `- ` / `– ` from its subtitle
|
||||
training data.
|
||||
4. **Length-ratio guard.** Output wildly longer than input → keep first
|
||||
sentence only. Needs a per-script ratio (CJK is far shorter per character).
|
||||
|
||||
### L4 — Bible lexicon, target side (per language)
|
||||
Canonical forms for the 66 book names, ~120 proper nouns (people, places),
|
||||
and key doctrinal terms. Applied through `applyTable` — the machinery is
|
||||
already hardened. Built-in table, SEPARATE from operator overrides, with the
|
||||
operator's entries winning so an app update never clobbers their work.
|
||||
|
||||
**Honesty constraint:** ship a language only where the canonical forms are
|
||||
verifiable. A wrong Korean Bible name is worse than NLLB's output. Structure
|
||||
the data so unverified languages simply have no entry and fall through to NLLB.
|
||||
|
||||
### L5 — liturgical phrasebook (skip MT entirely)
|
||||
`Amen.` must never reach NLLB. Short liturgical constants — Amen, Hallelujah,
|
||||
Hosanna, Glory to God, Praise the Lord — have stable known forms and are the
|
||||
highest-frequency utterances in preaching. Direct lookup, no inference. This
|
||||
is also the class that produces the worst failures, because NLLB has almost no
|
||||
context to work from on a 1-3 word input.
|
||||
|
||||
## What must not regress
|
||||
- Latency: EN 2-4s / ES 4-7s worst case. Every layer here is string work except
|
||||
L2, which is faster. No layer may add an inference pass.
|
||||
- `mtWorkers` stays 1 (benchmarked: 2 workers = 0.92x, slower).
|
||||
- Portuguese variants (`PtVariants`) run BEFORE `applyOverrides`; ordering with
|
||||
the new lexicon must be decided explicitly.
|
||||
- Operator's saved glossary must survive app updates.
|
||||
- `por_PT` is a pseudo-code aliased to `por_Latn` at the MT boundary only.
|
||||
|
||||
## Open questions for the audit
|
||||
1. Does the invented-question rule ever fire on legitimate output? Spanish
|
||||
inverts questions (`¿…?`); other languages may add a question particle where
|
||||
English used intonation with no `?`.
|
||||
2. Ordering: lexicon before or after the hallucination guard? Before or after
|
||||
PtVariants?
|
||||
3. Can Whisper be biased toward scripture vocabulary in transformers.js
|
||||
(an initial prompt), or is L1 find-and-replace the only available lever?
|
||||
4. Is a length-ratio guard safe across scripts, or does it need per-script
|
||||
calibration that we cannot verify for 200 languages?
|
||||
5. Which languages can we honestly ship a lexicon for, and how should the
|
||||
unverified ones behave?
|
||||
Reference in New Issue
Block a user