A stumble should not colour the desk red for the rest of the service

Build 121. Three things a single 16 s stall did that it should not have.

captionOnlyFinals is a running total for the whole service, and the verdict
tested it for more than zero — so one stale sentence pinned the light red until
the next restart. MEASURED: a campus Mac stalled once at 14:25, recovered
immediately to a 421 ms lag with an empty queue, and still showed red half an
hour later, with the operator quite reasonably reporting it as behind. The
totals still travel for the record; the verdict now asks when it last happened,
inside a two-minute window. A light that cannot go back to green teaches the
desk to ignore it.

That sentence also had its translations skipped. Dropping them is how the app
catches up — the listener has moved on and the time is better spent on what is
behind it. But that reasoning needs a backlog, and the queue was EMPTY: the room
lost a line of the sermon to a catch-up that had nothing to catch up on. It now
skips only when something is actually waiting, and says plainly when it chooses
to translate a late sentence anyway.

The stall itself was not the app. Two Screen Sharing sessions were taking as
much of the machine as the app was, with WindowServer on top — watching a campus
Mac during a service measurably degrades it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

app/engine.css

@@ -700,3 +700,13 @@ textarea:focus { border-color: var(--accent); }
   .langfly { display: none; }
   .langrow.touched { animation: none; }
 }
+
+/* Release notes are prose from the forge — wrap them, keep them short, and let
+   them scroll rather than push the rest of Settings off the pane. */
+.updnotes {
+  max-height: 16rem; overflow: auto; white-space: pre-wrap; word-break: break-word;
+  font-size: .75rem; line-height: 1.45; color: var(--muted);
+  background: var(--panel, rgba(255,255,255,.03));
+  border-radius: 6px; padding: .625rem .75rem; margin: .375rem 0 0;
+}
+#updFound { margin-top: .5rem; }

app/engine.js

@@ -190,7 +190,9 @@ const state = {
 
   /* health counters, reported to the operator page */
   degradedFinals: 0,         // finals that shipped with fewer languages (stale)
-  captionOnlyFinals: 0       // finals that shipped with no translations at all
+  captionOnlyFinals: 0,      // finals that shipped with no translations at all
+  captionOnlyAt: 0,          // when the last one happened
+  degradedAt: 0
 };
 
 // ────────────────────────────────────────────────────────── logging
@@ -891,12 +893,54 @@ async function respawnWorker(worker) {
         log('the speech model is too heavy for this machine — switching to ' +
             lighter.split('/').pop(), 'err');
       }
+      /*
+       * A LIGHTER MODEL GETS THE PREVIEW'S DTYPE, NOT THE BIG ONE'S.
+       *
+       * asrDtype is chosen for the accurate model and its weights are cached against
+       * it. Reusing it when stepping down asks for a variant that was never
+       * downloaded — and those weights live in the browser cache, with no network at
+       * a venue — so the step-down failed and the ladder could not move at all.
+       *
+       * MEASURED on a campus Mac: the cache held whisper-large-v3-turbo as
+       * encoder_model_q4f16.onnx (its asrDtype) and whisper-small as
+       * encoder_model_fp16.onnx (the preview's). Stepping down asked for
+       * whisper-small at q4f16, a file that is not there; it went to the network,
+       * failed, and logged "Unsupported model type: whisper" three times a minute
+       * for ever while the service sat dead.
+       *
+       * The preview runs whisper-small on exactly these settings every service, so
+       * they are the one combination this machine is known to be able to load.
+       */
+      const stepDtype = lighter
+        ? resolveDtype(cfg.asrFastDtype, { encoder_model: 'fp16', decoder_model_merged: 'q4' })
+        : resolveDtype(cfg.asrDtype, { encoder_model: 'fp16', decoder_model_merged: 'q4' });
       state.asrWorker = await loadWorkerWithFallback('speech', 'automatic-speech-recognition',
         state.asrModelInUse || cfg.asrModel,
-        resolveDtype(cfg.asrDtype, { encoder_model: 'fp16', decoder_model_merged: 'q4' }),
+        stepDtype,
         { encoder_model: 'q4', decoder_model_merged: 'q4' });
       log('speech model restarted' + (lighter ? ' on ' + lighter.split('/').pop() : ''), 'ok');
       state.slowRun = 0;
+      if (lighter) {
+        /*
+         * THROW AWAY THE BACKLOG THAT CAUSED THIS, AND LET THE NEW MODEL SETTLE.
+         *
+         * Swapping the model does nothing about the queue that triggered the swap.
+         * Those chunks are already past the caption-only cutoff, so they can no
+         * longer be translated in time and are worth nothing to the room — but the
+         * first of them gets handed to a model that has just loaded, takes the full
+         * worker timeout, and trips the absolute rule. MEASURED: a campus Mac fell
+         * large-v3-turbo -> small -> base in 34 seconds exactly that way, landing on
+         * a model SLOWER than the one it started from.
+         *
+         * So drop them, and refuse to step down again for a grace window, so the new
+         * model is judged on sentences it was actually given.
+         */
+        const dropped = queue.length;
+        queue.length = 0;
+        state.steppedDownAt = Date.now();
+        if (dropped) log('dropped ' + dropped + ' stale sentence' + (dropped === 1 ? '' : 's') +
+                         ' the old model never reached', 'warn');
+      }
     }
   } catch (err) {
     log('model restart failed: ' + err.message, 'err');
@@ -1794,6 +1838,7 @@ async function pump() {
 
   state.busy = true;
   const started = Date.now();
+  let jobAudioMs = 0;
   try {
     /* job.at is when the audio ENDED. Live translation of stale audio is
        worthless, but the WORDS are not: however old the job, it still gets
@@ -1805,6 +1850,9 @@ async function pump() {
     /* Near-silence telemetry (log-only this release): a chunk whose average
        level barely clears the noise floor is hallucination fuel. */
     const audio = flatten(job.frames);
+    /* How much speech this chunk actually holds. The step-down below compares
+       against it, because "slow" only means anything relative to the audio. */
+    jobAudioMs = (audio.length / SAMPLE_RATE) * 1000;
     let sum = 0;
     for (let i = 0; i < audio.length; i += 4) sum += audio[i] * audio[i];
     const meanRms = Math.sqrt(sum / Math.ceil(audio.length / 4));
@@ -1845,15 +1893,33 @@ async function pump() {
       const sentences = collapseLoops(splitSentences(text));
 
       let targets = orderedTargets();
-      if (ageSec > 12) {
+      /*
+       * SKIPPING A TRANSLATION ONLY BUYS SOMETHING IF THERE IS A QUEUE TO DRAIN.
+       *
+       * Dropping the translations of stale audio is how the app catches up: the
+       * listener has moved on, so the work is wasted and the time is better spent on
+       * the sentences behind it. That reasoning needs a backlog. With nothing waiting
+       * behind it, skipping costs the room a sentence and saves nothing at all.
+       *
+       * MEASURED: a campus Mac took one 16 s stall — two Screen Sharing sessions
+       * competing for the machine — and skipped that sentence's translations with an
+       * EMPTY queue. The room lost a line of the sermon to a catch-up that had
+       * nothing to catch up on.
+       */
+      if (ageSec > 12 && queue.length > 0) {
         targets = [];
         state.captionOnlyFinals++;
-        log('audio ' + ageSec.toFixed(0) + 's stale — caption only, translations skipped', 'err');
+        state.captionOnlyAt = Date.now();
+        log('audio ' + ageSec.toFixed(0) + 's stale with ' + queue.length + ' waiting — ' +
+            'caption only, translations skipped', 'err');
+      } else if (ageSec > 12) {
+        /* Late, but nothing is behind it: say so, and translate it anyway. */
+        log('audio ' + ageSec.toFixed(0) + 's late, but nothing is queued — translating it anyway', 'warn');
       } else if (ageSec > 6) {
         const listening = new Set((state.byLanguage || []).map((r) => r.lang));
         const before = targets.length;
         targets = targets.filter((t) => listening.has(t));
-        if (targets.length < before) state.degradedFinals++;
+        if (targets.length < before) { state.degradedFinals++; state.degradedAt = Date.now(); }
       }
 
       /* The segment's own loudness, measured once and shared by every sentence
@@ -1908,11 +1974,56 @@ async function pump() {
      * talk, and the backlog only grows — step down before the service is lost
      * rather than after. Measured on the Intel laptop: 12-25 s, every sentence.
      */
-    if (took > 8000) {
+    /*
+     * SLOWER THAN THE AUDIO IT COVERS — not slower than some constant.
+     *
+     * This used to fire only past 8 s, on the reasoning that 8 s meant slower
+     * than people talk. That was true when a chunk could be 8 s long. Chunks
+     * are capped at HARD_CUT_MAX, 5.5 s, so a 4.5 s chunk taking 4.3 s is
+     * already losing ground and never came near the old threshold.
+     *
+     * MEASURED on a campus M4 Pro: every sentence 4.2-4.7 s against roughly
+     * 4-5 s of audio, a queue 46 deep and a lag of 152 s still climbing — with
+     * nobody listening and no language being translated, so this was
+     * transcription alone. The one mechanism built to catch exactly that could
+     * not fire, because 4.4 s is not 8 s.
+     *
+     * The queue test is what separates "this machine cannot keep up" from one
+     * slow sentence: a backlog that is three deep and being fed faster than it
+     * drains does not recover on its own.
+     */
+    /*
+     * SLOWER THAN THE AUDIO IT COVERS, AND LANDING LATE.
+     *
+     * This used to fire only past 8 s, on the reasoning that 8 s meant slower
+     * than people talk. That held when a chunk could be 8 s long; chunks are
+     * capped at HARD_CUT_MAX, 5.5 s, so a 4.4 s sentence is already losing and
+     * never came near the old threshold.
+     *
+     * Both numbers below were measured on a campus M4 Pro, and the second one
+     * only after a first attempt got it wrong. 0.85, because that Mac ran 4.4 s
+     * against 4.5 s of audio — nominally FASTER than real time while losing
+     * badly, since the work outside this timer (segmenting, the preview model
+     * sharing the GPU, the scheduler) loses on its own.
+     *
+     * The second test was a queue depth, and watching that Mac go red disproved
+     * it: the queue sat at 2-3 while the lag held at 15 s and the verdict was
+     * already red. A machine can be persistently late without ever stacking work
+     * up — it finishes each sentence just slightly too slowly, for ever. So the
+     * test is how far behind the speaker the sentence actually landed, which is
+     * what the operator is complaining about in the first place.
+     */
+    /* A model that loaded seconds ago has not had a fair sentence yet. */
+    const settling = state.steppedDownAt && (Date.now() - state.steppedDownAt) < STEP_SETTLE_MS;
+    const behindMs = Date.now() - job.at;
+    const losingGround = jobAudioMs > 0 && took > jobAudioMs * 0.85 && behindMs > 8000;
+    if (!settling && (took > 8000 || losingGround)) {
       state.slowRun = (state.slowRun || 0) + 1;
       if (state.slowRun >= 3 && !respawning && lighterAsr(state.asrModelInUse || state.cfg.asrModel)) {
         state.slowRun = 0;
-        log('transcription is slower than people are speaking — stepping down to a lighter model', 'err');
+        log('transcription is slower than people are speaking (' + took + ' ms for ' +
+            Math.round(jobAudioMs) + ' ms of audio, ' + Math.round(behindMs / 1000) + 's behind) — ' +
+            'stepping down to a lighter model', 'err');
         respawnWorker(state.asrWorker);
       }
     } else state.slowRun = 0;
@@ -2423,6 +2534,11 @@ function monitorTranslation(cid, lang, text) {
  * It says it once per episode, and names both numbers so the operator can act
  * on it rather than go looking.
  */
+/* How long a freshly swapped speech model is left alone. Long enough to cover a
+   cold first inference, short enough that a genuinely hopeless model still falls
+   through the ladder within a sentence or two. */
+const STEP_SETTLE_MS = 20000;
+
 const DEAF_AFTER_MS = 90000;
 setInterval(() => {
   try {
@@ -2662,6 +2778,7 @@ function goLive(on) {
   state.live = on;
   if (on) {
     state.degradedFinals = 0; state.captionOnlyFinals = 0; state.fragmentFinals = 0;
+    state.captionOnlyAt = 0; state.degradedAt = 0;
     /* The watchdog measures from the start of the service, not from the last
        sentence of the previous one. */
     state.lastFinalAt = Date.now();
@@ -2736,6 +2853,10 @@ function reportStatus() {
       queue: queue.length,
       degradedFinals: state.degradedFinals,
       captionOnlyFinals: state.captionOnlyFinals,
+      /* WHEN, not just how many — the verdict needs to know whether this is
+         happening now or happened once an hour ago. */
+      captionOnlyAt: state.captionOnlyAt || 0,
+      degradedAt: state.degradedAt || 0,
       fragmentFinals: state.fragmentFinals || 0,
       cloud: state.cloudStats || null
     }
@@ -2931,6 +3052,102 @@ $('optDynamics').addEventListener('change', async (e) => {
          : 'voice dynamics off — every sentence at the voice\u2019s own level', 'ok');
 });
 
+/*
+ * ─────────────────────────── updates from our own forge ───────────────────
+ *
+ * Asked for, never volunteered. A background check that pops "an update is
+ * available" three minutes into a sermon is a hazard, not a feature, so this
+ * runs only when the operator presses the button and downloads only when they
+ * press the second one.
+ */
+function fmtMB(n) { return (Number(n || 0) / 1048576).toFixed(0) + ' MB'; }
+
+let updFound = null;
+
+async function checkForUpdate() {
+  const btn = $('updCheck');
+  btn.disabled = true;
+  $('updState').textContent = 'Checking…';
+  $('updFound').hidden = true;
+  try {
+    const r = await window.engineHost.updateCheck();
+    const here = r.current ? (r.current.version + ' \u00b7 ' + r.current.build) : 'this build';
+    if (!r.ok) { $('updState').textContent = r.error || 'Could not reach the update server.'; return; }
+    if (!r.latest) { $('updState').textContent = r.note || 'Nothing published yet. Running ' + here + '.'; return; }
+    const there = r.latest.tag;
+    if (!r.newer) { $('updState').textContent = 'Up to date \u2014 running ' + here + ', newest is ' + there + '.'; return; }
+    if (!r.latest.asset) {
+      $('updState').textContent = there + ' is newer, but it has no installer attached.';
+      return;
+    }
+    updFound = r.latest;
+    $('updState').textContent = 'Update available.';
+    $('updWhat').textContent = r.latest.title + ' \u2014 ' + fmtMB(r.latest.asset.size) +
+      '. You are on ' + here + '.';
+    $('updNotes').textContent = (r.latest.notes || '').trim() || 'No notes.';
+    $('updNotesBox').hidden = !r.latest.notes;
+    $('updFound').hidden = false;
+    $('updProgress').textContent = '';
+    $('updGet').disabled = false;
+  } catch (e) {
+    $('updState').textContent = 'Could not check: ' + e.message;
+  } finally {
+    btn.disabled = false;
+  }
+}
+
+async function downloadUpdate() {
+  if (!updFound || !updFound.asset) return;
+  const btn = $('updGet');
+  btn.disabled = true;
+  $('updProgress').textContent = 'Starting…';
+  try {
+    const r = await window.engineHost.updateDownload(updFound.asset.url, updFound.asset.name);
+    if (!r.ok) { $('updProgress').textContent = r.error || 'Download failed.'; btn.disabled = false; return; }
+      $('updProgress').textContent = 'Downloaded. Installing\u2026';
+      log('update ' + updFound.tag + ' downloaded to ' + r.path, 'ok');
+
+      /*
+       * Straight into the install. The app quits, a helper swaps the bundle and
+       * reopens it — so from here the window closing IS the update working, and
+       * saying so before it happens stops that looking like a crash.
+       */
+      $('updWhat').textContent = 'Installing ' + updFound.tag + '. Church Translate will close and ' +
+        'reopen by itself in a few seconds. Your settings, models and voices are untouched.';
+      const ins = await window.engineHost.updateInstall(r.path);
+      if (!ins || !ins.ok) {
+        /* Still useful: the file is downloaded, so fall back to the manual route
+           rather than leaving them with nothing. */
+        $('updProgress').textContent = (ins && ins.error) || 'Could not install.';
+        $('updWhat').textContent = 'Downloaded, but it could not install itself. Quit Church ' +
+          'Translate, open the disk image in Downloads, and drag the app over the one in Applications.';
+        try { await window.engineHost.updateReveal(r.path); } catch (_) {}
+        btn.disabled = false;
+      }
+  } catch (e) {
+    $('updProgress').textContent = 'Download failed: ' + e.message;
+    btn.disabled = false;
+  }
+}
+
+$('updCheck').addEventListener('click', checkForUpdate);
+$('updGet').addEventListener('click', downloadUpdate);
+if (window.engineHost.onUpdateProgress) {
+  window.engineHost.onUpdateProgress((p) => {
+    if (!p || !p.total) { $('updProgress').textContent = fmtMB(p && p.got); return; }
+    $('updProgress').textContent = Math.round((p.got / p.total) * 100) + '%  (' +
+      fmtMB(p.got) + ' of ' + fmtMB(p.total) + ')';
+  });
+}
+
+/* The token is a credential, so it is saved on blur rather than per keystroke
+   and never logged. */
+$('updToken').addEventListener('change', async () => {
+  const v = $('updToken').value.trim();
+  state.cfg = await window.engineHost.setConfig({ updateToken: v });
+  $('updState').textContent = v ? 'Token saved.' : 'Token cleared.';
+});
+
 $('revealCache').addEventListener('click', () => window.engineHost.revealCache());
 
 /* ---- carrying a setup to another campus ------------------------------- */
@@ -3583,6 +3800,7 @@ async function boot() {
   const savedSens = Number(state.cfg && state.cfg.sensitivity);
   state.sensitivity = Number.isFinite(savedSens) && savedSens > 0 ? savedSens : 3.5;
   applyThreshFromConfig();
+  try { $('updToken').value = (state.cfg && state.cfg.updateToken) || ''; } catch (_) {}
   refreshCacheInfo();
 
   /* Start the download first and do not wait for it. It is the slow part, it

app/index.html

@@ -380,6 +380,30 @@
 
           <hr>
 
+          <h3 class="subhead">Updates</h3>
+          <p class="hint">Checks this church’s own server, not the internet — so it works at a venue
+            with no route out. Nothing downloads on its own and nothing installs itself: this app is
+            often live in front of a congregation.</p>
+          <div class="row">
+            <button class="btn" id="updCheck">Check for an update</button>
+            <span class="hint" id="updState">—</span>
+          </div>
+          <div id="updFound" hidden>
+            <p class="hint" id="updWhat"></p>
+            <div class="row">
+              <button class="btn" id="updGet">Download and install</button>
+              <span class="hint" id="updProgress"></span>
+            </div>
+            <details id="updNotesBox"><summary class="hint">What changed</summary>
+              <pre class="updnotes" id="updNotes"></pre></details>
+          </div>
+          <label for="updToken" style="margin-top:.75rem">Access token</label>
+          <input id="updToken" type="password" placeholder="only while the repository is private"
+                 autocomplete="off" spellcheck="false">
+          <p class="hint">Stays on this Mac. It is not carried by <b>Save settings to a file</b>.</p>
+
+          <hr>
+
           <label style="margin-top:.75rem">Models</label>
           <p class="hint" id="modelInfo">—</p>
           <div class="row">

app/main.js

@@ -232,6 +232,22 @@ const DEFAULT_CONFIG = {
      browser sandbox — see ensureTranslator() below for why. */
   mtDtype: 'fp32',
   autoConnect: true,
+  /*
+   * WHERE UPDATES COME FROM.
+   *
+   * The church's own Gitea, not the internet: a campus Mac may have no route
+   * out, and an installer that cannot be fetched at the venue is worse than no
+   * updater at all. Nothing here is checked automatically — this app is often
+   * mid-service, and a download starting on its own is the last thing anyone
+   * wants. The operator presses the button.
+   *
+   * `token` is only needed while the repository is private, and is deliberately
+   * NOT in PORTABLE_APP: it is this machine's credential and must not ride along
+   * in an exported settings file.
+   */
+  updateForge: 'http://chronicles-ab.waterschurch.org:3000',
+  updateRepo: 'christian/church-translate',
+  updateToken: '',
   /* Setup-pane tuning that used to live only in the DOM. */
   sensitivity: 3.5,
   /*
@@ -424,6 +440,11 @@ function startServer() {
     execPath: nodeBin,
     env: Object.assign({}, process.env, {
       CT_FFMPEG: ffmpegPathForChildren(),
+      /* So /healthz can report the APP’s version rather than only an asset
+         hash. A campus Mac’s build was otherwise unknowable from the network,
+         which made confirming an update a matter of inference. */
+      CT_APP_VERSION: appBuild().version,
+      CT_APP_BUILD: appBuild().build,
       PORT: String(port),
       DATA_DIR: dataDir(),
       /* Where the spoken clips land, so the server can hand them to phones. */
@@ -2076,6 +2097,266 @@ app.on('before-quit', (e) => {
   if (ttsChild) { try { ttsChild.kill(); } catch (_) {} }
 });
 
+/*
+ * ─────────────────────────── updates from our own forge ───────────────────
+ *
+ * Gitea's release API, read directly. Deliberately NOT electron-updater: that
+ * wants a static feed (latest-mac.yml) published beside the artifacts, and the
+ * build here is ad-hoc signed with no Team ID, so a silent in-place swap would
+ * trip Gatekeeper on a machine nobody is sitting at. Checking, downloading and
+ * telling the operator where the file is does the useful 90% and cannot leave
+ * a campus with a half-replaced app.
+ *
+ * Tags are `v<version>-<build>` — v3.23.2-112. The build number is what really
+ * orders two releases of the same version, which is why it is compared second
+ * and numerically.
+ */
+function parseTag(tag) {
+  const m = String(tag || '').match(/^v?(\d+)\.(\d+)\.(\d+)(?:[-.](\d+))?/);
+  if (!m) return null;
+  return { major: +m[1], minor: +m[2], patch: +m[3], build: m[4] === undefined ? 0 : +m[4] };
+}
+
+/* Positive when a is newer than b. */
+function compareBuilds(a, b) {
+  if (!a) return -1;
+  if (!b) return 1;
+  for (const k of ['major', 'minor', 'patch', 'build']) {
+    if (a[k] !== b[k]) return a[k] > b[k] ? 1 : -1;
+  }
+  return 0;
+}
+
+function forgeRequest(urlStr, token, cb) {
+  let u;
+  try { u = new URL(urlStr); } catch (e) { return cb(e); }
+  const mod = u.protocol === 'https:' ? require('https') : http;
+  const headers = { 'Accept': 'application/json', 'User-Agent': 'church-translate' };
+  if (token) headers.Authorization = 'token ' + token;
+  const req = mod.get(urlStr, { headers, timeout: 15000 }, (res) => {
+    /* Gitea hands out asset downloads as a redirect to /attachments/<uuid>. */
+    if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
+      res.resume();
+      return forgeRequest(new URL(res.headers.location, urlStr).toString(), token, cb);
+    }
+    let body = '';
+    res.setEncoding('utf8');
+    res.on('data', (d) => { body += d; });
+    res.on('end', () => cb(null, res.statusCode, body));
+  });
+  req.on('timeout', () => { req.destroy(new Error('the forge did not answer within 15s')); });
+  req.on('error', (e) => cb(e));
+}
+
+ipcMain.handle('update:check', async () => {
+  const cfg = loadConfig();
+  const forge = String(cfg.updateForge || '').replace(/\/+$/, '');
+  const repo = String(cfg.updateRepo || '');
+  const here = appBuild();
+  const current = parseTag('v' + here.version + '-' + here.build);
+  if (!forge || !repo) return { ok: false, error: 'No update server is configured.', current: here };
+
+  return await new Promise((resolve) => {
+    forgeRequest(forge + '/api/v1/repos/' + repo + '/releases/latest', cfg.updateToken, (err, status, body) => {
+      if (err) return resolve({ ok: false, error: err.message, current: here });
+      /*
+       * A PRIVATE REPOSITORY ANSWERS 404, NOT 401 — Gitea hides its existence
+       * from a caller who cannot see it. Measured against this church's own
+       * forge: with no token the releases endpoint returns 404, so reporting
+       * "no releases published yet" told the operator the opposite of the
+       * truth and pointed them away from the one thing that would fix it.
+       * Without a token, 404 is ambiguous; say so and name the remedy.
+       */
+      if (status === 404) {
+        return resolve(cfg.updateToken
+          ? { ok: true, current: here, latest: null, newer: false, note: 'No releases published yet.' }
+          : { ok: false, current: here,
+              error: 'Nothing found there. If the repository is private, set an access token below.' });
+      }
+      if (status === 401 || status === 403) {
+        return resolve({ ok: false, current: here,
+          error: 'The update server refused the request. If the repository is private, set an access token below.' });
+      }
+      if (status !== 200) return resolve({ ok: false, error: 'Update server returned ' + status, current: here });
+      let rel;
+      try { rel = JSON.parse(body); } catch (_) { return resolve({ ok: false, error: 'Unreadable answer from the update server.', current: here }); }
+      const there = parseTag(rel.tag_name);
+      const asset = (rel.assets || []).find((a) => /\.dmg$/i.test(a.name || ''));
+      resolve({
+        ok: true,
+        current: here,
+        latest: { tag: rel.tag_name, title: rel.title || rel.name || rel.tag_name,
+                  notes: rel.body || '', published: rel.published_at || '',
+                  asset: asset ? { name: asset.name, size: asset.size, url: asset.browser_download_url } : null },
+        newer: compareBuilds(there, current) > 0
+      });
+    });
+  });
+});
+
+/*
+ * Download to the Downloads folder and reveal it. Not installed automatically:
+ * replacing a running app from inside itself needs a helper process, and doing
+ * that on a machine that might be live in front of a congregation is not a
+ * trade worth making. The operator drags it across when the room is empty.
+ */
+ipcMain.handle('update:download', async (_e, url, name) => {
+  const cfg = loadConfig();
+  const dest = path.join(app.getPath('downloads'), String(name || 'church-translate.dmg').replace(/[^\w.\-]/g, '_'));
+  return await new Promise((resolve) => {
+    let u;
+    try { u = new URL(String(url)); } catch (e) { return resolve({ ok: false, error: 'Bad download address.' }); }
+    /*
+     * PIN THE DOWNLOAD TO THE CONFIGURED FORGE, don't merely check it.
+     *
+     * Gitea answers with whatever address IT was reached on, which is not
+     * necessarily the one configured here: this church's forge is reachable as
+     * chronicles-ab.waterschurch.org and as 10.3.8.90, and the release API
+     * hands back the IP form. A plain same-host check therefore REFUSED a
+     * perfectly good installer — measured, on the very first real release.
+     *
+     * Rebuilding the URL on the configured origin keeps the property that
+     * matters (this only ever fetches from the server the operator configured,
+     * so a tampered API answer cannot redirect it elsewhere) while surviving a
+     * forge that knows itself by more than one name.
+     */
+    let base;
+    try { base = new URL(cfg.updateForge); } catch (_) { return resolve({ ok: false, error: 'The update server address is not valid.' }); }
+    u = new URL(u.pathname + u.search, base.origin);
+
+    const go = (target, hops) => {
+      if (hops > 5) return resolve({ ok: false, error: 'Too many redirects.' });
+      const mod = target.startsWith('https:') ? require('https') : http;
+      const headers = { 'User-Agent': 'church-translate' };
+      if (cfg.updateToken) headers.Authorization = 'token ' + cfg.updateToken;
+      mod.get(target, { headers, timeout: 60000 }, (res) => {
+        if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
+          res.resume();
+          return go(new URL(res.headers.location, target).toString(), hops + 1);
+        }
+        if (res.statusCode !== 200) { res.resume(); return resolve({ ok: false, error: 'Download failed (' + res.statusCode + ').' }); }
+        const total = Number(res.headers['content-length'] || 0);
+        let got = 0, lastPush = 0;
+        const out = fs.createWriteStream(dest);
+        res.on('data', (d) => {
+          got += d.length;
+          const now = Date.now();
+          if (now - lastPush > 400 && win && !win.isDestroyed()) {
+            lastPush = now;
+            try { win.webContents.send('update:progress', { got, total }); } catch (_) {}
+          }
+        });
+        res.pipe(out);
+        out.on('finish', () => out.close(() => {
+          /* A truncated installer is worse than none: it looks present and
+             fails at the venue. Refuse it rather than leave it lying there. */
+          if (total && got !== total) {
+            try { fs.unlinkSync(dest); } catch (_) {}
+            return resolve({ ok: false, error: 'The download ended early and was discarded.' });
+          }
+          resolve({ ok: true, path: dest, bytes: got });
+        }));
+        out.on('error', (e) => resolve({ ok: false, error: e.message }));
+      }).on('error', (e) => resolve({ ok: false, error: e.message }))
+        .on('timeout', function () { this.destroy(new Error('the download stalled')); });
+    };
+    go(u.toString(), 0);
+  });
+});
+
+/*
+ * ─────────────────────────── installing it, in place ──────────────────────
+ *
+ * An app cannot replace itself while it is running, so the swap is done by a
+ * small script that outlives us: it waits for this process to exit, moves the
+ * old bundle aside, copies the new one in, and relaunches.
+ *
+ * MOVED ASIDE, NOT DELETED. If the copy fails half way — a full disk, a bad
+ * download, the volume going away — the old app is put straight back and
+ * relaunched. The failure mode this is written against is a campus Mac with no
+ * working app and a service in an hour, which is far worse than not updating.
+ *
+ * No sudo: /Applications is admin-writable on macOS and the bundle belongs to
+ * the operator, so nothing here needs a password. If that is ever not true the
+ * copy fails, the old app comes back, and the log says so.
+ */
+ipcMain.handle('update:install', async (_e, dmgPath) => {
+  const dmg = String(dmgPath || '');
+  if (!dmg || !fs.existsSync(dmg)) return { ok: false, error: 'That installer is no longer on disk.' };
+
+  /* Never mid-service. Quitting the app stops every listener's captions, and an
+     update is never worth that — the operator stops translation first. */
+  if (engineIsLive) {
+    return { ok: false, error: 'Translation is live. Press Stop translating first, then install.' };
+  }
+
+  const appPath = app.getPath('exe').replace(/\/Contents\/MacOS\/[^/]+$/, '');
+  if (!/\.app$/.test(appPath)) {
+    return { ok: false, error: 'This copy is not running from an .app bundle, so it cannot replace itself.' };
+  }
+
+  const tmp = path.join(require('os').tmpdir(), 'ct-install-' + process.pid + '.sh');
+  const mount = '/Volumes/ChurchTranslateUpdate';
+  const aside = appPath + '.replacing-' + process.pid;
+  const script = [
+    '#!/bin/bash',
+    '# Written by Church Translate to swap itself. Safe to delete.',
+    'set -u',
+    'DMG=' + JSON.stringify(dmg),
+    'APP=' + JSON.stringify(appPath),
+    'ASIDE=' + JSON.stringify(aside),
+    'MNT=' + JSON.stringify(mount),
+    'PID=' + process.pid,
+    'LOG="$HOME/Library/Application Support/Church Translate/data/logs/install.log"',
+    'mkdir -p "$(dirname "$LOG")"',
+    'say(){ echo "$(date "+%Y-%m-%d %H:%M:%S") $*" >> "$LOG"; }',
+    'say "installing $DMG into $APP"',
+    '# wait for the app to actually be gone (up to 30s)',
+    'for i in $(seq 1 60); do kill -0 "$PID" 2>/dev/null || break; sleep 0.5; done',
+    'if kill -0 "$PID" 2>/dev/null; then say "app did not exit; aborting"; exit 1; fi',
+    'hdiutil detach "$MNT" -quiet 2>/dev/null',
+    'if ! hdiutil attach "$DMG" -nobrowse -quiet -mountpoint "$MNT"; then say "could not mount"; open -a "$APP"; exit 1; fi',
+    'NEW="$MNT/Church Translate.app"',
+    '# refuse to touch the installed app unless the replacement is really there',
+    'if [ ! -f "$NEW/Contents/Info.plist" ]; then say "disk image has no app"; hdiutil detach "$MNT" -quiet; open -a "$APP"; exit 1; fi',
+    'rm -rf "$ASIDE"',
+    'if ! mv "$APP" "$ASIDE"; then say "could not move the old app aside"; hdiutil detach "$MNT" -quiet; open -a "$APP"; exit 1; fi',
+    'if cp -R "$NEW" "$APP"; then',
+    '  xattr -dr com.apple.quarantine "$APP" 2>/dev/null',
+    '  say "installed $( /usr/libexec/PlistBuddy -c "Print :CFBundleVersion" "$APP/Contents/Info.plist" 2>/dev/null )"',
+    '  rm -rf "$ASIDE"',
+    'else',
+    '  say "copy failed — putting the old app back"',
+    '  rm -rf "$APP"; mv "$ASIDE" "$APP"',
+    'fi',
+    'hdiutil detach "$MNT" -quiet 2>/dev/null',
+    'open -a "$APP"',
+    'say "relaunched"',
+    'rm -f "$0"'
+  ].join('\n');
+
+  try {
+    fs.writeFileSync(tmp, script, { mode: 0o755 });
+  } catch (e) {
+    return { ok: false, error: 'Could not prepare the installer: ' + e.message };
+  }
+
+  /* Detached and unref'd, or it dies with us the moment we quit. */
+  try {
+    const child = spawn('/bin/bash', [tmp], { detached: true, stdio: 'ignore' });
+    child.unref();
+  } catch (e) {
+    return { ok: false, error: 'Could not start the installer: ' + e.message };
+  }
+
+  /* Give the renderer a moment to show the message before we go. */
+  forceQuit = true;
+  setTimeout(() => { try { app.quit(); } catch (_) {} }, 600);
+  return { ok: true };
+});
+
+ipcMain.handle('update:reveal', (_e, p2) => { try { shell.showItemInFolder(String(p2)); return true; } catch (_) { return false; } });
+
 ipcMain.handle('config:get', () => loadConfig());
 ipcMain.handle('config:set', (_e, patch) => {
   const merged = saveConfig(patch);

app/package.json

@@ -2,7 +2,7 @@
   "name": "church-translate-engine",
   "productName": "Church Translate",
   "version": "3.23.2",
-  "buildNumber": 112,
+  "buildNumber": 121,
   "private": true,
   "description": "Live sermon translation — self-contained: captures audio, transcribes, translates, and serves the congregation page itself",
   "main": "main.js",
@@ -106,7 +106,7 @@
       ]
     },
     "asar": false,
-    "buildVersion": "112",
+    "buildVersion": "121",
     "win": {
       "target": [
         {

app/preload.js

@@ -21,6 +21,11 @@ contextBridge.exposeInMainWorld('engineHost', {
 
   /* Translation lives in the main process; these are its only two verbs. */
   mtEnsure:     ()      => ipcRenderer.invoke('mt:ensure'),
+  updateCheck:  ()      => ipcRenderer.invoke('update:check'),
+  updateDownload: (u, n) => ipcRenderer.invoke('update:download', u, n),
+  updateReveal: (p)     => ipcRenderer.invoke('update:reveal', p),
+  updateInstall: (p)    => ipcRenderer.invoke('update:install', p),
+  onUpdateProgress: (fn) => ipcRenderer.on('update:progress', (_e, p) => fn(p)),
   settingsExport: ()    => ipcRenderer.invoke('settings:export'),
   settingsImport: ()    => ipcRenderer.invoke('settings:import'),
   audioDevices: ()      => ipcRenderer.invoke('audio:devices'),

app/server/server.js

@@ -500,16 +500,57 @@ const ENGINE_STALE_MS = 12000;
  * with one plain sentence does. Computed here so every operator surface —
  * in-app pane, browser page — shows the same verdict.
  */
+/*
+ * WHEN IT IS BEHIND, SAY WHAT IS LOADING IT.
+ *
+ * Translation cost is roughly linear in the number of active languages, through
+ * ONE NLLB process — measured, five languages in 1.9 s. So the single most
+ * useful fact about a machine that is falling behind is how many languages it
+ * is carrying, and that number was computed one function away and never shown.
+ * A campus reported "constantly behind, in the red" and the cause took an
+ * afternoon to find; it was eight languages on hardware tuned against one.
+ */
+/* How long a stumble keeps colouring the verdict. Long enough that a real
+   problem stays visible while someone walks to the desk, short enough that a
+   one-off does not outlive the sentence after it. */
+const TROUBLE_WINDOW_MS = 120000;
+
+function loadNote() {
+  const n = activeLanguages().length;
+  if (n < 2) return '';
+  return ' Carrying ' + n + ' languages' +
+         (config.allowAllLanguages ? ', and listeners may add more' : '') + '.';
+}
+
 function healthVerdict(connected, stale) {
   const es = state.engineStatus || {};
   if (!connected)               return { level: 'red',   detail: 'The translation app is not connected.' };
   if (stale)                    return { level: 'red',   detail: 'The translation app has stopped responding.' };
   if (!state.live)              return { level: 'green', detail: 'Ready.' };
   if (!es.modelsReady)          return { level: 'amber', detail: 'Models are still loading.' };
-  if ((es.lagMs || 0) > 12000 || (es.captionOnlyFinals || 0) > 0)
-    return { level: 'red',   detail: 'Falling badly behind — some sentences are captions only.' };
-  if ((es.lagMs || 0) > 6000 || (es.queue || 0) >= 2 || (es.degradedFinals || 0) > 0)
-    return { level: 'amber', detail: 'Running behind — idle languages may be trimmed to catch up.' };
+  /*
+   * RECENTLY, not ever.
+   *
+   * captionOnlyFinals is a running total for the whole service, and testing it
+   * for "more than zero" meant a single stale sentence pinned the desk's light
+   * red until the next restart. MEASURED: a campus Mac took one 16 s stall at
+   * 14:25 — caused by two Screen Sharing sessions competing for the machine —
+   * recovered immediately to a 421 ms lag with an empty queue, and still showed
+   * red half an hour later. An operator who looks up and sees red wants to know
+   * what is wrong NOW; a light that cannot go back to green teaches them to
+   * ignore it.
+   *
+   * So the totals still travel, for the record and for the operator page, but
+   * the verdict asks when it last happened.
+   */
+  const recentCaptionOnly = es.captionOnlyAt && (Date.now() - es.captionOnlyAt) < TROUBLE_WINDOW_MS;
+  if ((es.lagMs || 0) > 12000 || recentCaptionOnly)
+    return { level: 'red',
+             detail: 'Falling badly behind — some sentences are captions only.' + loadNote() };
+  const recentDegraded = es.degradedAt && (Date.now() - es.degradedAt) < TROUBLE_WINDOW_MS;
+  if ((es.lagMs || 0) > 6000 || (es.queue || 0) >= 2 || recentDegraded)
+    return { level: 'amber',
+             detail: 'Running behind — idle languages may be trimmed to catch up.' + loadNote() };
   if (es.cloud && es.cloud.parked)
     return { level: 'amber', detail: 'Natural voices are down — the built-in voices have taken over.' };
   /* A fallback in the last minute means one language just changed voice
@@ -1076,7 +1117,22 @@ const server = http.createServer(async (req, res) => {
   }
   if (req.method === 'OPTIONS') { res.writeHead(origin ? 204 : 405).end(); return; }
 
-  if (p === '/healthz') { sendJSON(res, 200, { ok: true, live: state.live, build: BUILD }); return; }
+  /*
+   * The APP's version, plainly, to anyone who can reach this port.
+   *
+   * `build` here has always been a hash of the listener-page assets — a
+   * cache-busting token, not a version — so the only way to tell what a campus
+   * Mac was running was to install something and watch the hash move. With an
+   * updater in the field that is the first question anyone asks, so answer it.
+   */
+  if (p === '/healthz') {
+    sendJSON(res, 200, {
+      ok: true, live: state.live, build: BUILD,
+      version: process.env.CT_APP_VERSION || '',
+      appBuild: process.env.CT_APP_BUILD || ''
+    });
+    return;
+  }
 
   // --- Companion / Stream Deck control -------------------------------
   //  GET or POST, token in ?key= or the X-Control-Token header.

app/tools/diag.sh

@@ -0,0 +1,39 @@
+#!/bin/bash
+# Church Translate — one-shot diagnostic for "it is behind / in the red".
+# Read-only: reads the log and asks the running app for its status. Changes nothing.
+L="$HOME/Library/Application Support/Church Translate/data/logs/app.log"
+C="$HOME/Library/Application Support/Church Translate/data/config.json"
+
+echo "=== MACHINE ==="
+system_profiler SPHardwareDataType 2>/dev/null | grep -E "Model Name|Chip|Total Number of Cores|Memory:" | sed 's/^ */  /'
+echo "  macOS $(sw_vers -productVersion)"
+
+echo
+echo "=== MEMORY PRESSURE / SWAP ==="
+memory_pressure 2>/dev/null | grep -Ei "free percentage|pageouts" | sed 's/^/  /'
+sysctl -n vm.swapusage 2>/dev/null | sed 's/^/  swap: /'
+
+echo
+echo "=== WHAT THE APP CHOSE AT STARTUP ==="
+grep -E "GPU features \(speech\)|no GPU acceleration|no fp16|loading the speech model|loading the translation model|speech model ready|translation model ready|models could not load" "$L" 2>/dev/null | tail -8 | sed 's/^/  /'
+
+echo
+echo "=== THROUGHPUT — is it slower than people talk? ==="
+echo "  slow-sentence events (>4s): $(grep -c 'slow sentence' "$L" 2>/dev/null)"
+grep "slow sentence" "$L" 2>/dev/null | tail -12 | sed 's/^/  /'
+grep -E "stepping down to a lighter" "$L" 2>/dev/null | tail -3 | sed 's/^/  /'
+
+echo
+echo "=== THE RED CONDITION ==="
+echo "  caption-only events: $(grep -c 'caption only' "$L" 2>/dev/null)"
+grep -E "caption only|stale — caption only" "$L" 2>/dev/null | tail -6 | sed 's/^/  /'
+
+echo
+echo "=== LIVE STATUS ==="
+TOK=$(grep -o '"controlToken"[[:space:]]*:[[:space:]]*"[^"]*"' "$C" 2>/dev/null | sed 's/.*"\([^"]*\)"$/\1/')
+curl -s -m 5 "http://127.0.0.1:3100/api/control/status?key=$TOK" 2>/dev/null | head -c 1500
+echo
+
+echo
+echo "=== RECENT ERRORS ==="
+grep "\[app\] err" "$L" 2>/dev/null | grep -v "natural voice fell back" | tail -12 | sed 's/^/  /'