4 Commits
Author SHA1 Message Date
christianandClaude Fable 5.1 81da657094 One voice per listener, and the voice process stops stalling itself
Build 127. Reported from a live test: part of the translation came out in a
different voice on an Android phone, and iPhones were skipping and repeating.
Three investigations against the campus Mac's own log and transcript, one
test harness driving the voice process directly, and the answer is not what
it sounded like.

WHAT THE MACHINE DID. One voice process ran all afternoon. Every language
kept a single resident engine from its first sentence (14:54:26) to STOP
(15:23:18); 2,800 clips, 560 per language, every one filed under its own
language and spoken by its own bundle. No synthesis-side voice change was
possible with that configuration. What the log shows at the time of the
complaint instead:

  - Someone on the English stream heard the ROOM MICROPHONE for 21 s
    (15:09:10-32). English was the first button on every phone's list, forced
    there by the page whatever the operator had chosen (fixed in build 125).
  - A listener tapped Chinese at 15:11:17. The 121 MB zh model was fetched
    and unpacked THREE TIMES AT ONCE on the voice process's only thread.
    Every stream stalled for about a minute, 26 sentences were dropped across
    all five languages, and the catch-up rushed the Spanish voices to their
    +16% cap. Same voice, sounding wrong.

WHAT WAS WRONG, AND IS NOW FIXED.

The live path filed every engine as an "audition". speak() resolved the
language's default voice, which always has a bundle, and passed it as
`explicit`, so the key was 'audition:'+bundle for every ordinary sentence
while the preload warmed copies under the language name that no sentence
ever read. Five languages asked for nine engines against a cap of six; every
service's first sentence built its real engines on the spot (1.3-2 s); the
operator's voice save flushed nothing that mattered. Engines are now keyed
by bundle and shared by every language, variant and audition that uses one,
auditions take at most two slots of their own and can never evict a live
engine, and the cap is 8, sized to the largest campus from measurement
(~165 MB per engine once it has spoken, 1.3 GB for eight).

Voice installs ran on the thread that answers every sentence, and were not
de-duplicated. They now run in a worker thread (voice-install.js), one per
bundle however many ask, staged and renamed into place whole so a half-
written bundle can never pass isInstalled().

Listener rows are { code, count } and engine.js read { lang, n } in five
places, so the set of "languages someone is on" was always empty — and a late
sentence keeps speech only for those. Under load the app dropped EVERY
language's speech instead of only the unchosen ones. Late became silent.

HLS segment numbers restarted at 1 for every new stream object, under a
'max-age=60, immutable' header. A phone back from the lock screen could be
answered from its cache with the previous stream's audio under the new
number: an earlier sentence, replayed. Numbers are seeded from the clock.

In clips mode a clip still downloading when the listener switched language
landed and played in the voice they had left; the frame now carries its
language and voice and the page checks both when the bytes land. In stream
mode the old playlist stayed loaded through a switch and three things could
restart it; it is stopped, not paused. A listener's alternate voice no longer
loses a round trip to the default on the original-text toggle.

One genuine wrong-voice path, dormant today because natural voices are off on
this campus: a variant waiting on another variant's premium synthesis
inherited that variant's Piper FALLBACK, in that variant's bundle — spa_ES
received the byte-identical Mexican clip and its own voice never loaded. A
follower now shares a fallback only when it is its own voice.

An audition (the Voices card's play button) played on the system default
output, which on the campus Mac is the Dante card feeding the hardware
channels. It now plays on another output, or refuses and says why.

VERIFIED: a harness drives the voice process through the baseline, a batch
of four auditions during live sentences, a voices save mid-stream, and nine
live languages against the cache — every clip spoken by its own bundle, and
auditions of a live voice now cost 78-311 ms with nothing rebuilt.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 16:05:15 -04:00
christianandClaude Opus 5 95d1a7b9ec Stop the stream seeking against itself, and stop the desk scrolling itself
Build 126. Two reports, unrelated to each other.

IPHONES REPEATING AND SKIPPING PARTS OF EVERY SENTENCE.

DRIFT_LAND was 0.6 s and one segment is 0.697 s, so the position the player
was being held at sat INSIDE the newest segment — a point it very often did
not have yet. It got dragged to the edge, ran out of audio and stalled. Then
'stalled' and 'waiting' were both wired to seekLive(), and a seek on a live
HLS stream makes the element re-buffer, which raises those same two events. So
the correction caused the stall that caused the correction. On Safari, where
HLS is native and a seek lands on a segment boundary, each turn of that loop
replays or drops a piece of a sentence — which is exactly what was described.

Three changes, all of them about not fighting the player: those two events no
longer seek, the target is three segments back (~2.1 s, what an HLS player
picks for itself on a live playlist), and a correction now has 5 s to settle
before another may run. Costs about 1.5 s of extra delay. A listener a second
further behind is following the service; one whose audio keeps jumping
backwards is not.

NOT YET CONFIRMED ON A REAL IPHONE — the mechanism is read off the code and
the measured segment length, and the next service is the test.

CANNOT SCROLL THE MONITOR WHILE LIVE.

Four boxes ended their update with box.scrollTop = box.scrollHeight,
unconditionally, so every sentence yanked the view back to the live edge and
reading back was impossible. They now follow the bottom only while the
operator is already at the bottom, and resume when they return — nothing to
press either way. Trimming the top is held off while they are scrolled away
too, since dropping old lines moves the text being read; the cap comes back
the moment they return, with a ceiling so nothing grows all service.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-15 15:25:42 -04:00
christianandClaude Opus 5 acfca90836 The English switch did nothing, so English could not be turned off
Build 125. Reported from the desk: every language on the listener page can be
switched off except English.

Exactly right, and it was not subtle. renderFeatured() unshifted the SOURCE
language into the list on every render, whatever the operator had chosen:

    if (list.indexOf(app.cfg.sourceLanguage) === -1)
      list.unshift(app.cfg.sourceLanguage);

So English was not merely on, it was always FIRST — the leading button on the
page — on a desk whose saved list was ["spa_Latn","por_Latn","por_PT","spa_ES"]
with English deliberately off and allowAll false. The operator had turned it
off, the config recorded that it was off, and the page put it back anyway.

The reasoning behind it was sound: people do want the English captions for
accessibility rather than translation. But that is a choice for the desk, not
for this function, and the switch to make it is already there in Settings next
to every other language. English is in the catalogue (languages.js:19) and the
operator window reads that same file, so turning it back on is one click.

A control that silently refuses is worse than no control. The operator's list
is now the list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-15 15:17:15 -04:00
christianandClaude Opus 5 7291ff48f7 Save QR image stranded the window on a picture of itself
Build 124. Reported from the booth: the button did not download anything and
there was no way back off the QR.

Both halves are the same fault. The window is served from the asset port and
the code comes off the congregation port, so the link the button built was
CROSS-ORIGIN — and a cross-origin <a download> has its download attribute
ignored and NAVIGATES instead. So the file never saved, and the window went to
the image. The app has no back button, no reload item and no address bar, so
that is a one-way trip: a headless booth Mac showing a full-screen QR with the
operator locked out of the desk. It fetches the code and downloads a blob now,
which is same-origin and actually saves.

The second change is the one that matters more. Nothing was stopping the app
window being replaced by any navigation at all, and the only reason this had
never bitten before is that nothing had tried. will-navigate now refuses to
replace the app page, logs what it refused, and sends anything external to the
browser. Our own pages are unaffected — the operator console and the poster
open as their own windows through setWindowOpenHandler, which is a different
path entirely.

Found because the build number went to the wrong field: buildNumber is the
source of truth and build.buildVersion is synced from it by predist, which
caught it and rebuilt correctly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-15 14:14:45 -04:00
8 changed files with 585 additions and 132 deletions
+155 -20
View File
@@ -195,6 +195,76 @@ const state = {
degradedAt: 0 degradedAt: 0
}; };
/*
* THE SHAPE OF A LISTENER ROW. The server sends { code, count } — see
* listenerCounts() in server.js — and this file read { lang, n } in five
* places, so every set built from the rows was EMPTY: no language ever counted
* as being listened to. Mostly harmless, except in the one place it mattered.
* When a sentence arrives late, speech is kept only for the languages someone
* is actually on, and "someone is on" was never true for any of them — so
* under load the app dropped the speech of EVERY language, for everyone,
* instead of only the languages nobody had chosen. Late became silent.
* MEASURED on a campus Mac, 2026-10-15 15:11-15:12: 26 sentences dropped
* across five languages during a stall, every one of them with listeners
* present. Both spellings are read, so a server one build older or newer
* still counts.
*/
function rowLang(r) { return r ? (r.code || r.lang || '') : ''; }
function rowCount(r) { return r ? (Number(r.count != null ? r.count : r.n) || 0) : 0; }
/*
* FOLLOW THE BOTTOM ONLY WHILE THE OPERATOR IS AT THE BOTTOM.
*
* Every one of these boxes used to end its update with
* `box.scrollTop = box.scrollHeight`, unconditionally. While the service is
* running that fires on every sentence, so scrolling back to re-read what was
* just said was impossible: the box yanked itself to the live edge a fraction
* of a second later, every time. Reported from the desk as "can't scroll on
* what's being said when it's live", which is exactly what it was.
*
* A box follows the live edge until the operator scrolls away from it, and
* starts following again when they come back to it. Nothing has to be pressed
* to get out, and nothing has to be pressed to get back in.
*
* The slack matters. A programmatic jump to the bottom also raises a scroll
* event, and it lands AT the bottom, so sticking survives its own scrolling.
* Sub-pixel layout and a growing last line can leave a couple of pixels
* behind, which with a zero threshold would unstick the box on its own output.
*/
/*
* Trimming the TOP of a box the operator is reading moves the text under
* their eyes — the other half of "it keeps jumping". While they are scrolled
* away the cap is suspended, so the lines they are looking at stay put; when
* they return to the live edge the backlog is trimmed in one go. The ceiling
* keeps a box that is left scrolled up all service from growing without end.
*/
function trimTop(box, cap) {
if (!box) return;
const held = box.dataset.stick === '0';
const limit = held ? cap * 6 : cap;
while (box.children.length > limit) box.removeChild(box.firstChild);
}
const STICK_SLACK_PX = 48;
const stickWired = new WeakSet();
function followBottom(box) {
if (!box) return;
if (!stickWired.has(box)) {
stickWired.add(box);
box.dataset.stick = '1';
box.addEventListener('scroll', () => {
const atBottom =
box.scrollHeight - box.scrollTop - box.clientHeight <= STICK_SLACK_PX;
box.dataset.stick = atBottom ? '1' : '0';
}, { passive: true });
}
/* Scrolled away: leave the view exactly where they put it. New content
grows BELOW the viewport, so not touching scrollTop is what holds the
reader's place — there is nothing else to do here. */
if (box.dataset.stick !== '0') box.scrollTop = box.scrollHeight;
}
// ────────────────────────────────────────────────────────── logging // ────────────────────────────────────────────────────────── logging
function log(msg, kind) { function log(msg, kind) {
@@ -208,8 +278,8 @@ function log(msg, kind) {
if (kind) line.className = kind; if (kind) line.className = kind;
line.textContent = new Date().toLocaleTimeString() + ' ' + msg; line.textContent = new Date().toLocaleTimeString() + ' ' + msg;
el.appendChild(line); el.appendChild(line);
while (el.children.length > 200) el.removeChild(el.firstChild); trimTop(el, 200);
el.scrollTop = el.scrollHeight; followBottom(el);
} }
function hero(stateName, title, body) { function hero(stateName, title, body) {
@@ -1017,10 +1087,15 @@ function preloadNewVoices() {
const want = state.targets.filter((l) => const want = state.targets.filter((l) =>
state.spokenLanguages.indexOf(l) !== -1 && !preloadedVoices.has(l)); state.spokenLanguages.indexOf(l) !== -1 && !preloadedVoices.has(l));
if (!want.length) return; if (!want.length) return;
want.forEach((l) => preloadedVoices.add(l));
/* One at a time in the child anyway; cap the request so a burst of joins /* One at a time in the child anyway; cap the request so a burst of joins
cannot queue up a re-download marathon. */ cannot queue up a re-download marathon.
window.engineHost.ttsPreload(want.slice(0, 4)); Only what is SENT counts as warmed. Every wanted language used to be
marked and only the first four sent, so a fifth was never preloaded and
built its engine on its first sentence instead — por_PT, at all five
starts on the Mini on 2026-10-15. Six is the voice process's own cap. */
const batch = want.slice(0, 6);
batch.forEach((l) => preloadedVoices.add(l));
window.engineHost.ttsPreload(batch);
} }
/* The translation model reports its download from the main process. */ /* The translation model reports its download from the main process. */
@@ -1031,7 +1106,10 @@ if (window.engineHost && window.engineHost.onTtsReady) {
state.spokenDefaults = info.defaults || state.spokenDefaults || {}; state.spokenDefaults = info.defaults || state.spokenDefaults || {};
log('built-in voices ready — ' + state.spokenLanguages.length + ' languages', 'ok'); log('built-in voices ready — ' + state.spokenLanguages.length + ' languages', 'ok');
/* Warm whatever people are already reading, so the first sentence of a /* Warm whatever people are already reading, so the first sentence of a
service is not the one that pays for the download. */ service is not the one that pays for the download. A voice process
that has just (re)started holds no engines, whatever this page warmed
in the last one — so forget that first. */
preloadedVoices.clear();
preloadNewVoices(); preloadNewVoices();
}); });
window.engineHost.onTtsProgress((p) => log('voice ' + p.bundle + ': ' + p.status)); window.engineHost.onTtsProgress((p) => log('voice ' + p.bundle + ': ' + p.status));
@@ -1725,9 +1803,9 @@ async function pumpPartial() {
* still gets a preview. * still gets a preview.
*/ */
const heard = (state.byLanguage || []) const heard = (state.byLanguage || [])
.filter((r) => r && r.lang && r.lang !== state.source && (r.n || 0) > 0) .filter((r) => r && rowLang(r) && rowLang(r) !== state.source && rowCount(r) > 0)
.sort((a, b) => (b.n || 0) - (a.n || 0)) .sort((a, b) => (b.n || 0) - (a.n || 0))
.map((r) => r.lang) .map(rowLang)
.filter((l) => state.targets.indexOf(l) !== -1); .filter((l) => state.targets.indexOf(l) !== -1);
const want = (heard.length ? heard : orderedTargets()).slice(0, PARTIAL_MAX_LANGS); const want = (heard.length ? heard : orderedTargets()).slice(0, PARTIAL_MAX_LANGS);
/* Never queue behind a finished sentence — a final in flight is worth more /* Never queue behind a finished sentence — a final in flight is worth more
@@ -1753,7 +1831,7 @@ async function pumpPartial() {
*/ */
function orderedTargets() { function orderedTargets() {
const counts = {}; const counts = {};
for (const row of state.byLanguage || []) counts[row.lang] = row.n || 1; for (const row of state.byLanguage || []) counts[rowLang(row)] = rowCount(row) || 1;
return state.targets return state.targets
.filter((t) => t !== state.source) .filter((t) => t !== state.source)
.sort((a, b) => (counts[b] || 0) - (counts[a] || 0)); .sort((a, b) => (counts[b] || 0) - (counts[a] || 0));
@@ -1916,7 +1994,7 @@ async function pump() {
/* Late, but nothing is behind it: say so, and translate it anyway. */ /* Late, but nothing is behind it: say so, and translate it anyway. */
log('audio ' + ageSec.toFixed(0) + 's late, but nothing is queued — translating it anyway', 'warn'); log('audio ' + ageSec.toFixed(0) + 's late, but nothing is queued — translating it anyway', 'warn');
} else if (ageSec > 6) { } else if (ageSec > 6) {
const listening = new Set((state.byLanguage || []).map((r) => r.lang)); const listening = new Set((state.byLanguage || []).map(rowLang));
const before = targets.length; const before = targets.length;
targets = targets.filter((t) => listening.has(t)); targets = targets.filter((t) => listening.has(t));
if (targets.length < before) { state.degradedFinals++; state.degradedAt = Date.now(); } if (targets.length < before) { state.degradedFinals++; state.degradedAt = Date.now(); }
@@ -2116,7 +2194,7 @@ setInterval(() => {
}, 30000); }, 30000);
function translateSentence(cid, sentence, targets, gainDb, silent) { function translateSentence(cid, sentence, targets, gainDb, silent) {
const listening = new Set((state.byLanguage || []).map((r) => r.lang)); const listening = new Set((state.byLanguage || []).map(rowLang));
for (const lang of targets) { for (const lang of targets) {
if ((mtDepth[lang] || 0) > 6 && !listening.has(lang)) { if ((mtDepth[lang] || 0) > 6 && !listening.has(lang)) {
state.degradedFinals++; state.degradedFinals++;
@@ -2471,8 +2549,8 @@ function monitorFinal(cid, text) {
monRows.set(cid, el); monRows.set(cid, el);
/* Neither the map nor the box may grow for the length of a service. */ /* Neither the map nor the box may grow for the length of a service. */
while (monRows.size > 60) monRows.delete(monRows.keys().next().value); while (monRows.size > 60) monRows.delete(monRows.keys().next().value);
while (box.children.length > 24) box.removeChild(box.firstChild); trimTop(box, 24);
box.scrollTop = box.scrollHeight; followBottom(box);
} }
/* /*
@@ -2514,7 +2592,7 @@ function monitorTranslation(cid, lang, text) {
row.classList.add('dropped'); row.classList.add('dropped');
} }
const box = $('monitor'); const box = $('monitor');
if (box) box.scrollTop = box.scrollHeight; followBottom(box);
} }
/* /*
@@ -2581,9 +2659,9 @@ function showMonitor(text, isPartial, langCount) {
sub.textContent = langCount + ' language' + (langCount === 1 ? '' : 's') + ' · ' + state.lagMs + ' ms'; sub.textContent = langCount + ' language' + (langCount === 1 ? '' : 's') + ' · ' + state.lagMs + ' ms';
el.appendChild(sub); el.appendChild(sub);
box.appendChild(el); box.appendChild(el);
while (box.children.length > 30) box.removeChild(box.firstChild); trimTop(box, 30);
} }
box.scrollTop = box.scrollHeight; followBottom(box);
} }
// ────────────────────────────────────────────────────────── church server // ────────────────────────────────────────────────────────── church server
@@ -3668,17 +3746,35 @@ $('copyAddr').addEventListener('click', () => {
* is made in their own design tool. The built-in poster page is still there * is made in their own design tool. The built-in poster page is still there
* at /poster for anyone who wants it; it is just not what this button does. * at /poster for anyone who wants it; it is just not what this button does.
*/ */
$('saveQr').addEventListener('click', () => { $('saveQr').addEventListener('click', async () => {
const base = shareBase(); const base = shareBase();
if (!base) { log('no address to encode yet', 'err'); return; } if (!base) { log('no address to encode yet', 'err'); return; }
const a = document.createElement('a'); /*
a.href = 'http://127.0.0.1:' + state.local.port + '/qr.svg?url=' + * Fetch the code and hand the browser a BLOB rather than linking straight at
* the server. This window is served from the asset port and the QR comes off
* the congregation port, so a plain <a download> is CROSS-ORIGIN — the
* download attribute is ignored and the window navigates to the image
* instead. With no back button, no reload item and no address bar, that
* strands the operator on a full-screen QR with no way out; build 123 did
* exactly that to a booth Mac. A blob: URL is same-origin, so it downloads.
*/
const url = 'http://127.0.0.1:' + state.local.port + '/qr.svg?url=' +
encodeURIComponent(base + '/'); encodeURIComponent(base + '/');
try {
const r = await fetch(url, { cache: 'no-store' });
if (!r.ok) throw new Error('the server answered ' + r.status);
const href = URL.createObjectURL(await r.blob());
const a = document.createElement('a');
a.href = href;
a.download = 'church-translate-qr.svg'; a.download = 'church-translate-qr.svg';
document.body.appendChild(a); document.body.appendChild(a);
a.click(); a.click();
a.remove(); a.remove();
log('saved the QR for ' + base + '/', 'ok'); setTimeout(() => URL.revokeObjectURL(href), 10000);
log('saved the QR for ' + base + '/ to Downloads', 'ok');
} catch (err) {
log('could not save the QR: ' + err.message, 'err');
}
}); });
$('openOperator').addEventListener('click', () => { $('openOperator').addEventListener('click', () => {
window.open('http://127.0.0.1:' + state.local.port + '/operator', '_blank'); window.open('http://127.0.0.1:' + state.local.port + '/operator', '_blank');
@@ -4572,6 +4668,40 @@ function renderVoiceList() {
} }
} }
/*
* WHERE AN AUDITION PLAYS: never on the card that feeds the room.
*
* An <audio> element with no sink plays on the system default output, and on
* the production Mini that IS the Dante Virtual Soundcard (system_profiler:
* "Default Output Device: Yes") — the same card that carries Spanish on its
* third channel (outputMap spa_Latn -> [2]). So ▶ there put the sample line
* onto the Dante network, service or no service. The phones were never
* exposed: an audition comes back to this page as bytes and never becomes a
* clip the server can see.
*
* Which of the card's channels it reached is Chromium's mono up-mix and the
* Dante routing, neither of which this page controls, so it does not try to
* pick a "safe" channel. It plays on another output, or not at all, and says
* why.
*/
async function auditionSink() {
const routedId = ($('outDevice') && $('outDevice').value) || '';
const routedLabel = (state.cfg && state.cfg.outputDeviceLabel) || '';
if (!routedId && !routedLabel) return ''; // no hardware output in use
const strip = (l) => String(l || '').toLowerCase().replace(/^default\s*-\s*/, '').trim();
let outs = [];
try {
outs = (await navigator.mediaDevices.enumerateDevices()).filter((d) => d.kind === 'audiooutput');
} catch (_) { /* refused below rather than guessed */ }
const card = strip(((outs.find((d) => d.deviceId === routedId) || {}).label) || routedLabel);
const isCard = (d) => (!!routedId && d.deviceId === routedId) || (!!card && strip(d.label) === card);
const def = outs.find((d) => d.deviceId === 'default');
if (def && !isCard(def)) return ''; // the system default is somewhere else
const other = outs.find((d) => d.deviceId !== 'default' && d.deviceId !== 'communications' && !isCard(d));
if (other) return other.deviceId;
throw new Error('the only output here is the one feeding the hardware channels — auditions stay off it');
}
async function auditionVoice(lang, v, btn, msg) { async function auditionVoice(lang, v, btn, msg) {
if (auditionBusy) return; if (auditionBusy) return;
auditionBusy = true; auditionBusy = true;
@@ -4596,9 +4726,14 @@ async function auditionVoice(lang, v, btn, msg) {
msg.className = 'vmsg'; msg.className = 'vmsg';
} }
if (instruct) msg.textContent = 'with your direction…'; if (instruct) msg.textContent = 'with your direction…';
/* Where it will play is settled BEFORE the voice process builds anything:
an audition that cannot play should cost the service nothing. */
const sink = await auditionSink();
const out = await window.engineHost.ttsAudition({ lang, voice: v, instruct }); const out = await window.engineHost.ttsAudition({ lang, voice: v, instruct });
if (!auditionAudio) auditionAudio = new Audio(); if (!auditionAudio) auditionAudio = new Audio();
if (!out.wav) throw new Error('that voice produced no audio'); if (!out.wav) throw new Error('that voice produced no audio');
if (typeof auditionAudio.setSinkId === 'function') await auditionAudio.setSinkId(sink);
else if (sink) throw new Error('this build cannot keep auditions off the hardware output');
auditionAudio.src = 'data:audio/wav;base64,' + out.wav; auditionAudio.src = 'data:audio/wav;base64,' + out.wav;
/* play() rejects on autoplay policy or a decode failure; either way the /* play() rejects on autoplay policy or a decode failure; either way the
operator needs the reason, not a silent button. */ operator needs the reason, not a silent button. */
+21
View File
@@ -662,6 +662,27 @@ function createWindow() {
: '/index.html'; : '/index.html';
win.loadURL(assetOrigin + page); win.loadURL(assetOrigin + page);
/*
* NOTHING replaces the app in this window. There is no back button, no
* reload item and no address bar, so a window that leaves the app page is a
* window the operator cannot get back from — which is precisely how build
* 123 stranded a booth Mac on a full-screen QR image. Our own pages still
* open as their OWN windows through setWindowOpenHandler below; this only
* refuses to replace the app itself, and sends anything external to the
* browser where it belongs.
*/
win.webContents.on('will-navigate', (e, url) => {
if (url === assetOrigin + page) return; /* reloading ourselves is fine */
e.preventDefault();
console.error('[main] refused to navigate the app window to ' + url);
let local = false;
try {
const u = new URL(url);
local = (u.hostname === '127.0.0.1' || u.hostname === 'localhost');
} catch (_) { local = false; }
if (!local) shell.openExternal(url).catch(() => {});
});
win.webContents.on('did-finish-load', () => console.log('[main] page loaded')); win.webContents.on('did-finish-load', () => console.log('[main] page loaded'));
win.webContents.on('did-fail-load', (_e, code, desc, url) => win.webContents.on('did-fail-load', (_e, code, desc, url) =>
console.error('[main] page FAILED to load: ' + code + ' ' + desc + ' ' + url)); console.error('[main] page FAILED to load: ' + code + ' ' + desc + ' ' + url));
+3 -2
View File
@@ -2,7 +2,7 @@
"name": "church-translate-engine", "name": "church-translate-engine",
"productName": "Church Translate", "productName": "Church Translate",
"version": "3.23.2", "version": "3.23.2",
"buildNumber": 123, "buildNumber": 127,
"private": true, "private": true,
"description": "Live sermon translation — self-contained: captures audio, transcribes, translates, and serves the congregation page itself", "description": "Live sermon translation — self-contained: captures audio, transcribes, translates, and serves the congregation page itself",
"main": "main.js", "main": "main.js",
@@ -50,6 +50,7 @@
"vendor/ort-wasm-simd-threaded.jsep.mjs", "vendor/ort-wasm-simd-threaded.jsep.mjs",
"server/**/*", "server/**/*",
"tts-service.js", "tts-service.js",
"voice-install.js",
"kokoro-sidecar.js", "kokoro-sidecar.js",
"voices.js", "voices.js",
"pt-variants.js", "pt-variants.js",
@@ -106,7 +107,7 @@
] ]
}, },
"asar": false, "asar": false,
"buildVersion": "123", "buildVersion": "127",
"win": { "win": {
"target": [ "target": [
{ {
+15 -1
View File
@@ -219,7 +219,21 @@ class LanguageStream {
this.dropTimes = []; // when, for a "recently" view on the operator page this.dropTimes = []; // when, for a "recently" view on the operator page
this.building = []; // frames of the segment being filled this.building = []; // frames of the segment being filled
this.segments = []; // { seq, body, frames } this.segments = []; // { seq, body, frames }
this.nextSeq = 1; /*
* SEGMENT NUMBERS MUST NEVER REPEAT, and starting at 1 made them repeat.
* A stream object is built when a language's first listener arrives and
* dropped when the last one leaves, so within one service the same
* language can get a fresh stream several times — and a phone that came
* back from the lock screen, or a second listener, asked for
* spa_Latn/1.aac again. The segments are served 'max-age=60, immutable',
* which is the right header for a segment that is never reused and the
* wrong one for a number that is: for up to a minute the browser answered
* from its cache with the PREVIOUS stream's audio under the new number —
* a sentence from earlier in the sermon, replayed. Numbering from the
* clock makes every incarnation's numbers new; HLS only asks that they
* increase by one within a playlist, which they still do.
*/
this.nextSeq = Date.now();
this.timer = null; this.timer = null;
this.lastTick = 0; this.lastTick = 0;
this.carry = 0; // fractional frames owed from the last tick this.carry = 0; // fractional frames owed from the last tick
+113 -14
View File
@@ -163,10 +163,21 @@
function renderFeatured() { function renderFeatured() {
var wrap = $('featured'); var wrap = $('featured');
wrap.textContent = ''; wrap.textContent = '';
/*
* THE OPERATOR'S LIST IS THE LIST. This used to unshift the source
* language in unconditionally, on the reasoning that plenty of people want
* the English captions for accessibility rather than translation — which
* is true, and still available, but it is not this code's call to make.
* The operator has a switch for every language including English, and for
* the source language alone that switch did nothing: English was put back
* on every render, so a desk that had deliberately turned it off still
* offered it as the first button on the page. A control that silently
* refuses is worse than no control.
*
* Wanting English captions is now one click in Settings, in the same list
* as every other language, and it stays where it is put.
*/
var list = (app.cfg.featured || []).slice(); var list = (app.cfg.featured || []).slice();
/* The source language belongs in the list too — plenty of people want the
English captions for accessibility, not translation. */
if (list.indexOf(app.cfg.sourceLanguage) === -1) list.unshift(app.cfg.sourceLanguage);
list.forEach(function (code) { list.forEach(function (code) {
var b = langButton(code); var b = langButton(code);
if (b) wrap.appendChild(b); if (b) wrap.appendChild(b);
@@ -464,6 +475,7 @@
drift: null, drift: null,
watchdog: null, // catches engines that stall on HLS without erroring watchdog: null, // catches engines that stall on HLS without erroring
everPlayed: false, // this engine has PROVEN it can play HLS everPlayed: false, // this engine has PROVEN it can play HLS
clipsForVoice: false, // on clips because of an alternate voice, not because HLS failed
// clip-fallback state // clip-fallback state
ctx: null, ctx: null,
keepAlive: null, keepAlive: null,
@@ -508,7 +520,16 @@
/* After any interruption the player resumes where it left off and stays /* After any interruption the player resumes where it left off and stays
behind live forever — no error, no stall, just permanently late. Every behind live forever — no error, no stall, just permanently late. Every
one of these events is a chance to catch back up. */ one of these events is a chance to catch back up. */
['playing', 'stalled', 'waiting', 'canplay', 'loadedmetadata'].forEach(function (ev) { /*
* 'stalled' and 'waiting' are deliberately NOT in this list. A seek on a
* live HLS stream makes the element re-buffer, which raises exactly those
* two events — so seeking from them is a loop that feeds itself: seek,
* stall, seek, stall. On Safari, where HLS is played natively and a seek
* lands on a segment boundary, each turn of that loop replays or skips
* part of a sentence. That is the "it repeats a bit two or three times
* then moves on" reported from iPhones.
*/
['playing', 'canplay', 'loadedmetadata'].forEach(function (ev) {
a.addEventListener(ev, function () { seekLive(); }); a.addEventListener(ev, function () { seekLive(); });
}); });
@@ -574,9 +595,13 @@
/* Give up on streaming and use per-sentence clips. The clip path was already /* Give up on streaming and use per-sentence clips. The clip path was already
unlocked inside the tap precisely so this is possible after the gesture has unlocked inside the tap precisely so this is possible after the gesture has
been spent — otherwise a browser that lies about HLS would end up silent. */ been spent — otherwise a browser that lies about HLS would end up silent. */
function fallBackToClips() { function fallBackToClips(forVoice) {
if (srv.mode === 'clips') return; if (srv.mode === 'clips') return;
srv.mode = 'clips'; srv.mode = 'clips';
/* Remember WHY. Only a page that left the stream for an alternate voice
returns to it when the listener goes back to the default — see
applyVoiceMode(). */
srv.clipsForVoice = !!forVoice;
stopStream(); stopStream();
if (srv.keepAlive) srv.keepAlive.play().catch(function () {}); if (srv.keepAlive) srv.keepAlive.play().catch(function () {});
try { console.warn('live stream unsupported here — using per-sentence clips'); } catch (_) {} try { console.warn('live stream unsupported here — using per-sentence clips'); } catch (_) {}
@@ -588,8 +613,28 @@
* and 4s of tolerance plus a 5s check interval meant real drift could reach * and 4s of tolerance plus a 5s check interval meant real drift could reach
* nine seconds before anyone looked. * nine seconds before anyone looked.
*/ */
var DRIFT_MAX = 1.5; /*
var DRIFT_LAND = 0.6; * DRIFT_LAND WAS 0.6 S, AND ONE SEGMENT IS 0.697 S.
*
* So the target sat INSIDE the newest segment — a point the player very
* often did not have yet. It would be dragged up to the edge, run out of
* audio, stall, and the stall would trigger another correction. A phone
* cannot hold a position closer to live than the segments arriving.
*
* Three segments back (~2.1 s) is what HLS players choose for themselves on
* a live playlist, and it is the smallest target with enough buffered audio
* to survive one late segment. The tolerance is wider than one segment too,
* so an ordinary arrival wobble is not read as drift.
*
* It costs about a second and a half of extra delay. A listener a second
* further behind is following the service; a listener whose audio keeps
* jumping backwards is not.
*/
var DRIFT_MAX = 3.0;
var DRIFT_LAND = 2.1;
/* A correction needs time to settle before its own result is judged. */
var SEEK_COOLDOWN_MS = 5000;
var lastSeekAt = 0;
/* /*
* How far behind the live edge this listener is SUPPOSED to be. * How far behind the live edge this listener is SUPPOSED to be.
@@ -616,6 +661,11 @@
var want = driftTarget(); var want = driftTarget();
var behind = (edge - a.currentTime) - want; // + = too late, - = too early var behind = (edge - a.currentTime) - want; // + = too late, - = too early
if (!force && Math.abs(behind) <= DRIFT_MAX) return; if (!force && Math.abs(behind) <= DRIFT_MAX) return;
/* One correction at a time. Without this, timeupdate (four times a
second) can fire a fresh seek while the last one is still buffering,
and the element never gets a chance to settle anywhere. */
if (!force && Date.now() - lastSeekAt < SEEK_COOLDOWN_MS) return;
lastSeekAt = Date.now();
/* /*
* Land just BEHIND the target, never on the edge. Seeking exactly to the * Land just BEHIND the target, never on the edge. Seeking exactly to the
* live edge leaves no buffered audio at all, so the phone stalls on the * live edge leaves no buffered audio at all, so the phone stalls on the
@@ -823,14 +873,27 @@
*/ */
function applyVoiceMode() { function applyVoiceMode() {
if (srv.voice) { if (srv.voice) {
fallBackToClips(); fallBackToClips(true);
return; return;
} }
/* Back on the default: allow streaming again. fallBackToClips latches /* Back on the default: allow streaming again. fallBackToClips latches
mode to 'clips' and maybeStartStream refuses to act on that, so the mode to 'clips' and maybeStartStream refuses to act on that, so the
latch has to be released here or the listener stays on clips for the latch has to be released here or the listener stays on clips for the
rest of the service. */ rest of the service. */
if (srv.mode === 'clips' && hlsSupported() && srv.unlocked && srv.serverHello) { /*
* Only when the alternate is what put us on clips. Every engine reconnect
* sends every listener a 'voices' frame, and an engine that cannot play
* HLS used to be handed back to the stream by it — eight seconds of
* dropped clips until the watchdog demoted it again.
*
* And silence the alternate's clips that are still scheduled. Left to
* play out they overlapped the shared stream as it came up: the voice
* the listener left and the default together, for up to the six seconds
* the clip queue holds.
*/
if (srv.mode === 'clips' && srv.clipsForVoice && hlsSupported() && srv.unlocked && srv.serverHello) {
stopClips();
srv.clipsForVoice = false;
srv.mode = ''; srv.mode = '';
maybeStartStream(); maybeStartStream();
} }
@@ -864,22 +927,52 @@
function switchAudioLanguage(lang) { function switchAudioLanguage(lang) {
stopClips(); // the old language stops NOW, mid-word stopClips(); // the old language stops NOW, mid-word
if (srv.mode === 'stream' && srv.el) { if (srv.mode === 'stream' && srv.el) {
/* Pause the old stream; the new one starts when the server's hello /*
confirms the new language's stream exists. */ * STOP the old stream, do not merely pause it; the new one starts when
try { srv.el.pause(); } catch (_) {} * the server's hello confirms the new language's stream exists.
if (srv.retry) { clearTimeout(srv.retry); srv.retry = null; } *
* Paused, the element still pointed at the OLD playlist until that
* hello, and three things restart a paused element without asking: the
* 8 s watchdog's retry (startStream(srv.lang, true), and srv.lang was
* still the old language), coming back from a locked screen
* (visibilitychange plays it), and the lock-screen play button. Each
* put the previous language's voice back in the listener's ear.
* stopStream() is what the back button already does, and the hello's
* maybeStartStream() brings the new language up from it.
*/
stopStream();
} }
setMediaSession(); setMediaSession();
} }
// ------------------------------------------------- per-clip fallback path // ------------------------------------------------- per-clip fallback path
/*
* Is this clip for the language AND voice this page is on right now?
*
* Asked twice — when the clip is announced and again when its bytes land —
* because the fetch and the decode take a few hundred milliseconds, and
* stopClips() can only silence what is already scheduled, not what is still
* downloading. A listener who switched language in that window heard the
* previous language's sentence AFTER the switch; for Spanish that is one
* sentence of the Mexican voice in a Spaniard's ear, or the reverse. The
* name carries the language (p<seq>-spa_ES.aac); the server also sends lang
* and voice in the frame, and a frame without them is judged on the name.
*/
function clipIsOurs(frame) {
var lang = frame.lang || (String(frame.name).match(/-([a-z]{3}_[A-Za-z]+)\.aac$/) || [])[1];
if (lang !== app.lang) return false;
if ('voice' in frame && String(frame.voice || '') !== String(srv.voice || '')) return false;
return true;
}
/* Fetch, decode and schedule one clip so it follows the previous one. Only /* Fetch, decode and schedule one clip so it follows the previous one. Only
used where native HLS is missing. */ used where native HLS is missing. */
function playClip(frame) { function playClip(frame) {
var name = frame && frame.name; var name = frame && frame.name;
if (srv.mode === 'stream') return; // the stream already carries it if (srv.mode === 'stream') return; // the stream already carries it
if (!tts.on || !srv.ctx || !name) return; if (!tts.on || !srv.ctx || !name) return;
if (!clipIsOurs(frame)) return;
fetch('/audio/' + encodeURIComponent(name)) fetch('/audio/' + encodeURIComponent(name))
.then(function (r) { return r.ok ? r.arrayBuffer() : Promise.reject(new Error('missing')); }) .then(function (r) { return r.ok ? r.arrayBuffer() : Promise.reject(new Error('missing')); })
@@ -891,6 +984,8 @@
}) })
.then(function (audio) { .then(function (audio) {
if (!tts.on || !srv.ctx) return; if (!tts.on || !srv.ctx) return;
/* Landed after a switch of language, voice or mode: not ours any more. */
if (srv.mode === 'stream' || !clipIsOurs(frame)) return;
var src = srv.ctx.createBufferSource(); var src = srv.ctx.createBufferSource();
src.buffer = audio; src.buffer = audio;
src.connect(srv.ctx.destination); src.connect(srv.ctx.destination);
@@ -1379,7 +1474,11 @@
} }
/* The playlist is guaranteed to exist from this moment — the server /* The playlist is guaranteed to exist from this moment — the server
creates the stream when this listener registers. */ creates the stream when this listener registers. */
maybeStartStream(); /* Not for a listener on an alternate: the shared stream is the
operator's default voice, and starting it here played that voice
until the server's 'voices' reply moved the page to clips. */
if (srv.voice) fallBackToClips(true);
else maybeStartStream();
/* The picker was rendered before this socket existed, when we still had /* The picker was rendered before this socket existed, when we still had
to assume the device's own voice. Now we know better — re-render, or to assume the device's own voice. Now we know better — re-render, or
a listener keeps staring at a voice chooser that controls nothing. */ a listener keeps staring at a voice chooser that controls nothing. */
+22 -6
View File
@@ -1763,12 +1763,20 @@ wss.on('connection', (ws, req) => {
if (!m.voice) row.audio[m.lang] = m.name; if (!m.voice) row.audio[m.lang] = m.name;
delayed((who) => { delayed((who) => {
const frame = JSON.stringify({
t: 'audio', id: row.id, name: m.name, seconds: m.seconds || 0
});
/* Which voice this clip IS. '' is the operator's default, which is /* Which voice this clip IS. '' is the operator's default, which is
what a listener who never opened the chooser is on. */ what a listener who never opened the chooser is on. */
const clipVoice = String(m.voice || ''); const clipVoice = String(m.voice || '');
/*
* The frame says which language and voice it carries, and the page
* checks both again when the clip's bytes land (clipIsOurs() in
* listen.js). The routing below is decided at SEND time; a listener
* who switches language or voice while a clip is still downloading
* would otherwise hear it after the switch, in the voice they left.
*/
const frame = JSON.stringify({
t: 'audio', id: row.id, name: m.name, seconds: m.seconds || 0,
lang: m.lang, voice: clipVoice
});
for (const [ws, meta] of state.listeners) { for (const [ws, meta] of state.listeners) {
if (ws.readyState !== 1) continue; if (ws.readyState !== 1) continue;
if (!who(meta)) continue; if (!who(meta)) continue;
@@ -2054,14 +2062,22 @@ wss.on('connection', (ws, req) => {
if (m.t === 'lang') { if (m.t === 'lang') {
const meta = state.listeners.get(ws); const meta = state.listeners.get(ws);
if (!meta) return; if (!meta) return;
meta.lang = String(m.lang || config.sourceLanguage); const next = String(m.lang || config.sourceLanguage);
/* /*
* A voice belongs to a language. Carrying a Spanish speaker across to * A voice belongs to a language. Carrying a Spanish speaker across to
* Portuguese would ask the engine for a voice that language does not * Portuguese would ask the engine for a voice that language does not
* have, so switching language always returns the listener to the * have, so switching language always returns the listener to the
* default and hands them the new language's options. * default and hands them the new language's options.
*
* Only a CHANGE of language, though. The page re-sends the language it
* is already on to have its captions re-rendered (the original-text
* toggle does exactly that), and resetting the voice there put a
* listener on an alternate back on the default for a round trip —
* until the page re-asserted its pick, they were sent the default
* voice's clips: one voice, then the other, then theirs again.
*/ */
meta.voice = ''; if (next !== meta.lang) meta.voice = '';
meta.lang = next;
pushLanguagesToEngine(); pushLanguagesToEngine();
/* /*
* builtInVoice MUST be here, not just on the first hello. The page does * builtInVoice MUST be here, not just on the first hello. The page does
@@ -2073,7 +2089,7 @@ wss.on('connection', (ws, req) => {
*/ */
send(ws, { t: 'hello', live: state.live, lang: meta.lang, serviceName: config.serviceName, build: BUILD, send(ws, { t: 'hello', live: state.live, lang: meta.lang, serviceName: config.serviceName, build: BUILD,
builtInVoice: Voices.hasVoice(meta.lang), builtInVoice: Voices.hasVoice(meta.lang),
voiceOptions: voiceOptionsFor(meta.lang), voice: '', voiceOptions: voiceOptionsFor(meta.lang), voice: meta.voice || '',
scrollback: scrollbackFor(meta.lang, meta.proxied) }); scrollback: scrollbackFor(meta.lang, meta.proxied) });
broadcastOperators(); broadcastOperators();
} }
+170 -81
View File
@@ -42,9 +42,6 @@ let VOICE_DIR = process.platform === 'win32'
: path.join(os.homedir(), 'Library/Application Support/Church Translate Voices'); : path.join(os.homedir(), 'Library/Application Support/Church Translate Voices');
let CLIP_DIR = path.join(os.tmpdir(), 'church-translate-audio'); let CLIP_DIR = path.join(os.tmpdir(), 'church-translate-audio');
/* Loaded engines, one per language. Kept small: each costs ~330-400 MB of ONNX
arena regardless of the 63 MB model, so we do not hold voices nobody is
listening to. */
/* What each voice says when auditioned — a real line of scripture, so the /* What each voice says when auditioned — a real line of scripture, so the
operator hears the voice on the material it will actually read. */ operator hears the voice on the material it will actually read. */
const SAMPLE_TEXT = { const SAMPLE_TEXT = {
@@ -65,8 +62,25 @@ const SAMPLE_TEXT = {
hat_Latn: 'Senyè a se gadò mwen, mwen p ap manke anyen.' hat_Latn: 'Senyè a se gadò mwen, mwen p ap manke anyen.'
}; };
const loaded = new Map(); /*
const MAX_LOADED = 6; * Loaded engines, ONE PER BUNDLE, shared by every language, variant and
* audition that speaks with it. Kept bounded, not small: MEASURED on an M2
* Max, an engine costs ~75 MB at construction and ~165 MB once it has spoken
* (8 engines, 1.3 GB after one sentence each), not the 330-400 MB this used
* to assume. The cap is sized to the largest campus — eight languages — so a
* live engine is never evicted to make room for another live one in normal
* service; below the cap an eviction means a reload of ~0.3-0.5 s on this
* process's only thread, in front of the next sentence, every time.
*
* `live` marks an engine a sentence or the preload asked for. Auditions keep
* to MAX_AUDITION slots of their own, always go first, and can never push a
* live engine out: with every slot speaking, an auditioned voice is built,
* played once and dropped. See engineFor() for why this is keyed by bundle.
*/
const loaded = new Map(); // 'bundle/model' -> { tts, usedAt, live }
const installing = new Map(); // bundle -> Promise: one download, however many ask
const MAX_LOADED = 8;
const MAX_AUDITION = 2;
/* /*
* The operator's per-language voice choices, made by ear in the Voices * The operator's per-language voice choices, made by ear in the Voices
@@ -142,81 +156,120 @@ function isInstalled(v) {
* 359 files, same total bytes — and it is now the same code on both platforms. * 359 files, same total bytes — and it is now the same code on both platforms.
*/ */
function installVoice(v) { function installVoice(v) {
/*
* OFF THIS THREAD. The fetch, the bzip2 and the unpack all ran here, on the
* voice process's only thread, from inside engineFor() — so the first
* sentence of a language whose voice was not installed yet stalled every
* OTHER language's sentences for as long as the install took. MEASURED on a
* campus Mac, 2026-10-15 15:11: a listener tapped Chinese, the 121 MB zh
* model was fetched and unpacked three times at once on this thread, every
* stream stalled, 26 sentences were dropped and the catch-up rushed the
* Spanish voices to their +16% cap. The work now happens in a worker thread
* (voice-install.js); this thread only relays its progress and keeps
* speaking. The worker unpacks into a staging folder and renames it into
* place whole, so isInstalled() never sees a half-written bundle.
*/
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
fs.mkdirSync(VOICE_DIR, { recursive: true }); const fail = (why) => reject(new Error('could not install ' + v.bundle + ': ' + why));
const url = voices.bundleUrl(v.bundle);
reply({ type: 'voiceProgress', bundle: v.bundle, status: 'downloading' }); reply({ type: 'voiceProgress', bundle: v.bundle, status: 'downloading' });
let w;
(async () => { try {
const res = await fetch(url, { redirect: 'follow' }); const { Worker } = require('worker_threads');
if (!res.ok) throw new Error('HTTP ' + res.status); w = new Worker(path.join(__dirname, 'voice-install.js'), {
const body = Buffer.from(await res.arrayBuffer()); workerData: { url: voices.bundleUrl(v.bundle), voiceDir: VOICE_DIR, bundle: v.bundle }
/* A proxy or a captive portal returns an HTML error page with HTTP 200;
the bzip2 magic is the cheapest way to notice before unpacking it. */
if (body.length < 1024 || body.toString('latin1', 0, 3) !== 'BZh') {
throw new Error('that download was not a voice bundle');
}
reply({ type: 'voiceProgress', bundle: v.bundle, status: 'installing' });
const bunzip = require('unbzip2-stream');
const tar = require('tar-stream');
const { Readable } = require('stream');
await new Promise((res2, rej2) => {
const extract = tar.extract();
extract.on('entry', (header, stream, next) => {
/* Never write outside VOICE_DIR, whatever the archive claims. */
const dest = path.resolve(VOICE_DIR, header.name);
if (dest !== VOICE_DIR && !dest.startsWith(VOICE_DIR + path.sep)) {
stream.resume(); return next();
}
if (header.type === 'directory') {
fs.mkdirSync(dest, { recursive: true });
stream.resume(); return next();
}
fs.mkdirSync(path.dirname(dest), { recursive: true });
stream.pipe(fs.createWriteStream(dest)).on('finish', next).on('error', rej2);
}); });
extract.on('finish', res2); } catch (e) { return fail(e.message); }
extract.on('error', rej2); let settled = false;
Readable.from(body).pipe(bunzip()).on('error', rej2).pipe(extract); const done = (err) => { if (settled) return; settled = true; err ? fail(err) : resolve(); };
w.on('message', (m) => {
if (!m || !m.status) return;
if (m.status === 'error') { done(m.message || 'the installer failed'); return; }
reply({ type: 'voiceProgress', bundle: v.bundle, status: m.status });
if (m.status === 'ready') done();
}); });
w.on('error', (e) => done((e && e.message) || String(e)));
reply({ type: 'voiceProgress', bundle: v.bundle, status: 'ready' }); w.on('exit', (code) => { if (!settled) done('the installer exited (' + code + ')'); });
})().then(resolve, (err) => reject(new Error('could not install ' + v.bundle + ': ' + err.message)));
}); });
} }
async function engineFor(lang, explicit) { /*
/* Auditions key the cache by bundle so trying five voices for one language * The engine for one voice, built at most once however many ask for it.
does not evict the voice that language is actually using. */ *
const v = explicit || builtinVoiceFor(lang); * KEYED BY BUNDLE — and the old keying is what put batches of four
const key = explicit ? "audition:" + v.bundle : lang; * "audition:" loads in the production log with nobody at the Voices card.
if (loaded.has(key)) { * Engines were keyed two ways: by language ('spa_Latn') for the preload, and
const e = loaded.get(key); * 'audition:<bundle>' whenever a voice was passed in, which speak() did for
e.usedAt = Date.now(); * EVERY ordinary sentence because the resolved default always has a bundle.
return e.tts; * So the live path was filed as an audition, every bundle was built at least
* twice (five languages asked for nine engines against a cap of six), the
* preload warmed copies no sentence ever read, and the first sentence of
* every service built the real ones on the spot. MEASURED on the Mini,
* 2026-10-15: the 14:26 service's first sentence was said at 14:54:26.291
* and its four engines were ready at 26.995, 27.309, 27.833 and 28.344 —
* ~0.3-0.5 s each on this process's only thread — while what got evicted
* was the startup preload's unused copies.
*
* The engine's config is a function of the bundle alone (voices.ttsConfig;
* speed, gain and sid are applied per sentence), so sharing is exact:
* spa_Latn and spa_MX read one es_MX-ald engine, and auditioning a voice that
* is already speaking reads the live engine instead of building a second.
*/
async function engineFor(v, live, who) {
if (!v || !v.bundle) throw new Error('no built-in voice' + (who ? ' for ' + who : ''));
const key = v.bundle + '/' + (v.model || '');
/*
* One download per bundle. A language joining mid-service is asked for by
* the preload AND by its first sentences at once, and each used to fetch
* and unpack the whole bundle itself — MEASURED on the Mini: pt_BR-jeff
* three times between 11:39:05 and 11:39:50, zh-ll three times between
* 15:11:17 and 15:12:16, two 'voice timed out' each time. The unpack is staged and
* renamed into place whole (voice-install.js), so isInstalled() stays false
* until the bundle is complete — but a second asker must still wait on the
* first install, not start its own.
*/
if (!loaded.has(key) && (installing.has(v.bundle) || !isInstalled(v))) {
if (!installing.has(v.bundle)) {
installing.set(v.bundle, installVoice(v).finally(() => installing.delete(v.bundle)));
}
await installing.get(v.bundle);
} }
if (!v) throw new Error('no built-in voice for ' + lang); /* Looked up again: whoever finished waiting first has built it. Nothing can
if (!isInstalled(v)) await installVoice(v); interleave from here to the set — construction is synchronous. */
let e = loaded.get(key);
if (!e) {
if (!sherpa) sherpa = require('sherpa-onnx-node'); if (!sherpa) sherpa = require('sherpa-onnx-node');
e = { tts: new sherpa.OfflineTts(voices.ttsConfig(voicePath(v), v)), usedAt: 0, live: !!live };
const dir = voicePath(v); if (makeRoom(!!live)) loaded.set(key, e);
const tts = new sherpa.OfflineTts(voices.ttsConfig(dir, v)); log('voice ready: ' + v.bundle + (who ? ' for ' + who : '') +
(live ? '' : loaded.get(key) === e ? ' (audition)' : ' (audition, not kept: every slot is speaking)'));
/* Evict the least recently used rather than growing without bound. */
if (loaded.size >= MAX_LOADED) {
let oldest = null;
for (const [k, e] of loaded) if (!oldest || e.usedAt < loaded.get(oldest).usedAt) oldest = k;
if (oldest) { loaded.delete(oldest); log('evicted voice ' + oldest); }
} }
e.usedAt = Date.now();
/* An audition never demotes a live engine; a sentence promotes an auditioned one. */
if (live) e.live = true;
return e.tts;
}
loaded.set(key, { tts, usedAt: Date.now() }); /*
log('voice ready: ' + key + ' (' + v.bundle + ')'); * Room for one more engine, or false when an audition should not take one.
return tts; * Least recently used goes first, as before — auditioned engines before live
* ones, and a live engine only ever gives way to another live one.
*/
function makeRoom(live) {
const stalest = (wantLive) => {
let k = null;
for (const [key, x] of loaded) if (x.live === wantLive && (!k || x.usedAt < loaded.get(k).usedAt)) k = key;
return k;
};
const evict = (k) => { log('evicted voice ' + k + (loaded.get(k).live ? '' : ' (audition)')); loaded.delete(k); };
const auditions = Array.from(loaded.values()).filter((x) => !x.live).length;
if (!live && auditions >= MAX_AUDITION) evict(stalest(false));
if (loaded.size < MAX_LOADED) return true;
const k = stalest(false) || (live ? stalest(true) : null);
if (!k) return false;
evict(k);
return true;
} }
// ------------------------------------------------------------- cloud voices // ------------------------------------------------------------- cloud voices
@@ -928,6 +981,9 @@ async function speak(job) {
text, Math.round((Number(job.gainDb) || 0) * 2) / 2, pinRate text, Math.round((Number(job.gainDb) || 0) * 2) / 2, pinRate
].join('\u0001'); ].join('\u0001');
/* Set when the leader's result was a Piper fallback in ANOTHER variant's
voice — this job then speaks its own fallback, straight to Piper. */
let piperOnly = false;
if (key) { if (key) {
const hit = synthCache.get(key); const hit = synthCache.get(key);
if (hit) { if (hit) {
@@ -941,8 +997,23 @@ async function speak(job) {
counted as one; the key is deleted below so the NEXT utterance counted as one; the key is deleted below so the NEXT utterance
tries the natural voice again. */ tries the natural voice again. */
if (premiumPick && done.source !== premiumPick.provider) { cloudStats.fallback++; fallbackAt = Date.now(); } if (premiumPick && done.source !== premiumPick.provider) { cloudStats.fallback++; fallbackAt = Date.now(); }
/*
* ...but only a fallback in THIS language's own Piper voice. The key
* names the PREMIUM voice, and spa_Latn, spa_ES and spa_MX share it
* (KOKORO_LANG maps all three to 'e', both Portugueses to 'p'). Their
* Piper fallbacks they do not share: spa_ES falls back to davefx,
* spa_Latn to ald, por_PT to miro, por_Latn to jeff. Cloning the
* leader's fallback put the Mexican voice in the Castilian stream for
* that sentence — measured in a harness against this file: spa_ES's
* clip was the byte-identical ald file and davefx never even loaded.
* So the follower still skips the sidecar that just missed (the
* variants still switch together), but in its own voice.
*/
if (done.source !== 'piper' || done.bundle === (voice.bundle || '')) {
return cloneClip(done, job.lang, started); return cloneClip(done, job.lang, started);
} }
piperOnly = true;
}
} catch (_) { /* the first caller failed — make our own */ } } catch (_) { /* the first caller failed — make our own */ }
} }
} }
@@ -956,7 +1027,7 @@ async function speak(job) {
* alternate skips the premium path entirely — otherwise cloudSynth would * alternate skips the premium path entirely — otherwise cloudSynth would
* fall back to the operator's own pick and the page would lie. * fall back to the operator's own pick and the page would lie.
*/ */
const cloudOut = (alt && alt.engine === 'builtin') const cloudOut = (piperOnly || (alt && alt.engine === 'builtin'))
? null ? null
: await cloudSynth(job.lang, text, (alt && alt.engine !== 'builtin') ? alt : null, { speed: kokoroSpeed, deadline }); : await cloudSynth(job.lang, text, (alt && alt.engine !== 'builtin') ? alt : null, { speed: kokoroSpeed, deadline });
if (cloudOut) { if (cloudOut) {
@@ -984,13 +1055,16 @@ async function speak(job) {
source: cloudOut.provider source: cloudOut.provider
}; };
} }
if (!(alt && alt.engine === 'builtin') && cloudVoiceFor(job.lang)) { /* A piperOnly job's fallback was already counted at the cache hit. */
if (!piperOnly && !(alt && alt.engine === 'builtin') && cloudVoiceFor(job.lang)) {
cloudStats.fallback++; cloudStats.fallback++;
fallbackAt = Date.now(); fallbackAt = Date.now();
if (cloudStats.fallback === 1 || cloudStats.fallback % 10 === 0) pushStats('fallback'); if (cloudStats.fallback === 1 || cloudStats.fallback % 10 === 0) pushStats('fallback');
} }
const tts = await engineFor(job.lang, voice.bundle ? voice : null); /* Live: a sentence is speaking with this voice, so no audition may take
its engine. A listener's built-in alternate counts — someone is on it. */
const tts = await engineFor(voice.bundle ? voice : null, true, job.lang);
if (!sherpa) sherpa = require('sherpa-onnx-node'); if (!sherpa) sherpa = require('sherpa-onnx-node');
/* /*
@@ -1067,6 +1141,9 @@ async function speak(job) {
const seconds = secs; const seconds = secs;
return { return {
source: 'piper', source: 'piper',
/* WHICH Piper voice spoke it. A follower may share a fallback only when
it is its own voice — see the cache hit above. */
bundle: voice.bundle || '',
/* What the sentence ended on — the stream gives a full stop a full breath /* What the sentence ended on — the stream gives a full stop a full breath
and a mid-thought fragment only a beat. */ and a mid-thought fragment only a beat. */
terminal: text.slice(-1), terminal: text.slice(-1),
@@ -1080,8 +1157,12 @@ async function speak(job) {
}; };
}; };
/* A piperOnly job is a fallback by construction. It must not stand in the
cache as a leader, or the next identical line would inherit the miss
without trying the natural voice — the chorus case below. */
const cacheKey = piperOnly ? null : key;
const promise = produce(); const promise = produce();
if (key) synthCache.set(key, { promise, at: Date.now() }); if (cacheKey) synthCache.set(cacheKey, { promise, at: Date.now() });
try { try {
const out = await promise; const out = await promise;
/* /*
@@ -1091,10 +1172,10 @@ async function speak(job) {
* miss would have re-created the very voice switch this file exists to * miss would have re-created the very voice switch this file exists to
* prevent (measured in the harness before shipping, not after). * prevent (measured in the harness before shipping, not after).
*/ */
if (key && premiumPick && out && out.source !== premiumPick.provider) synthCache.delete(key); if (cacheKey && premiumPick && out && out.source !== premiumPick.provider) synthCache.delete(cacheKey);
return out; return out;
} catch (e) { } catch (e) {
if (key) synthCache.delete(key); if (cacheKey) synthCache.delete(cacheKey);
throw e; throw e;
} }
} }
@@ -1187,11 +1268,15 @@ process.on('message', async (msg) => {
if (msg.type === 'voices') { if (msg.type === 'voices') {
/* The operator changed a per-language voice choice. */ /* The operator changed a per-language voice choice. */
voiceChoice = msg.choices || {}; voiceChoice = msg.choices || {};
/* Drop cached engines for languages whose choice changed, so the next /*
sentence uses the new voice rather than the one already resident. */ * Nothing to drop. Engines are keyed by bundle, so a changed choice
for (const k of Array.from(loaded.keys())) { * resolves to a different engine on the next sentence by itself, and
if (!String(k).startsWith("audition:")) loaded.delete(k); * the one it replaced ages out like any other. The loop that was here
} * only ever deleted the preload's per-language copies (the live path was
* filed under 'audition:'); under bundle keys it would delete the live
* engines too, and every "Use it" would make the next sentence rebuild
* every language's voice.
*/
reply({ type: 'voicesOk', count: Object.keys(voiceChoice).length }); reply({ type: 'voicesOk', count: Object.keys(voiceChoice).length });
return; return;
} }
@@ -1222,7 +1307,9 @@ process.on('message', async (msg) => {
} finally { voiceChoice = saved; } } finally { voiceChoice = saved; }
} else { } else {
if (!sherpa) sherpa = require('sherpa-onnx-node'); if (!sherpa) sherpa = require('sherpa-onnx-node');
const tts = await engineFor(msg.lang, spec.bundle ? spec : null); /* Not live: shares the engine when this voice is already speaking,
otherwise takes an audition slot — never a live one. */
const tts = await engineFor(spec.bundle ? spec : builtinVoiceFor(msg.lang), false, 'audition');
const audio = tts.generate({ text, sid: spec.sid || 0, speed: spec.speed || 1.0 }); const audio = tts.generate({ text, sid: spec.sid || 0, speed: spec.speed || 1.0 });
if (spec.gain && spec.gain !== 1) { if (spec.gain && spec.gain !== 1) {
for (let i = 0; i < audio.samples.length; i++) { for (let i = 0; i < audio.samples.length; i++) {
@@ -1256,7 +1343,9 @@ process.on('message', async (msg) => {
/* Warm the languages people are actually listening to, so the first /* Warm the languages people are actually listening to, so the first
sentence of a service is not the one that pays for the download. */ sentence of a service is not the one that pays for the download. */
for (const lang of msg.langs || []) { for (const lang of msg.langs || []) {
try { await engineFor(lang); } catch (e) { log('preload failed for ' + lang + ': ' + e.message); } /* Keyed exactly as speak() keys it, so this warms the engine the first
sentence will read, not a copy beside it. */
try { await engineFor(builtinVoiceFor(lang), true, lang); } catch (e) { log('preload failed for ' + lang + ': ' + e.message); }
} }
reply({ type: 'preloaded', langs: msg.langs || [] }); reply({ type: 'preloaded', langs: msg.langs || [] });
return; return;
+78
View File
@@ -0,0 +1,78 @@
/*
* voice-install.js — download and unpack ONE voice bundle, in a worker thread.
*
* This work used to run on the voice process's own event loop, from inside
* engineFor(), the moment a sentence arrived for a language whose voice was
* not installed. MEASURED on a campus Mac, 2026-10-15 15:11: a listener tapped
* Chinese, three parallel downloads and unpacks of the 121 MB zh model ran at
* once on the thread that has to answer every sentence, every other language
* stalled for about a minute, 26 sentences were dropped, and the catch-up
* pushed the Spanish voices to their +16% rush cap — reported from the pews
* as the voice "changing". bzip2 in pure JS is CPU work; it has no business on
* that thread.
*
* So the whole fetch-check-unpack now happens here, off the loop, and
* tts-service.js de-duplicates it per bundle. The archive is unpacked into a
* staging folder and renamed into place only when complete, so a half-written
* bundle can never pass isInstalled() and be handed to sherpa.
*/
'use strict';
const { parentPort, workerData } = require('worker_threads');
const fs = require('fs');
const path = require('path');
const { url, voiceDir, bundle } = workerData;
(async () => {
fs.mkdirSync(voiceDir, { recursive: true });
const res = await fetch(url, { redirect: 'follow' });
if (!res.ok) throw new Error('HTTP ' + res.status);
const body = Buffer.from(await res.arrayBuffer());
/* A proxy or a captive portal returns an HTML error page with HTTP 200;
the bzip2 magic is the cheapest way to notice before unpacking it. */
if (body.length < 1024 || body.toString('latin1', 0, 3) !== 'BZh') {
throw new Error('that download was not a voice bundle');
}
parentPort.postMessage({ status: 'installing' });
const bunzip = require('unbzip2-stream');
const tar = require('tar-stream');
const { Readable } = require('stream');
const staging = path.join(voiceDir, '.installing-' + bundle + '-' + process.pid);
fs.rmSync(staging, { recursive: true, force: true });
fs.mkdirSync(staging, { recursive: true });
await new Promise((resolve, reject) => {
const extract = tar.extract();
extract.on('entry', (header, stream, next) => {
/* Never write outside the staging folder, whatever the archive claims. */
const dest = path.resolve(staging, header.name);
if (dest !== staging && !dest.startsWith(staging + path.sep)) {
stream.resume(); return next();
}
if (header.type === 'directory') {
fs.mkdirSync(dest, { recursive: true });
stream.resume(); return next();
}
fs.mkdirSync(path.dirname(dest), { recursive: true });
stream.pipe(fs.createWriteStream(dest)).on('finish', next).on('error', reject);
});
extract.on('finish', resolve);
extract.on('error', reject);
Readable.from(body).pipe(bunzip()).on('error', reject).pipe(extract);
});
/* The archive's top-level folder is the bundle name — that is what
voicePath() opens. Swap it into place in one rename. */
const unpacked = path.join(staging, bundle);
if (!fs.existsSync(unpacked)) throw new Error('the archive did not contain ' + bundle);
const final = path.join(voiceDir, bundle);
fs.rmSync(final, { recursive: true, force: true });
fs.renameSync(unpacked, final);
fs.rmSync(staging, { recursive: true, force: true });
parentPort.postMessage({ status: 'ready' });
})().catch((err) => {
parentPort.postMessage({ status: 'error', message: (err && err.message) || String(err) });
});