christian/church-translate
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases 11 Wiki Activity

Church Translate, as it runs at the campuses

Browse Source 
Live sermon translation into 100+ languages, running entirely on one Mac per
campus: audio in, voice detection, Whisper on the GPU, NLLB-200 in its own
process, and a built-in web server the congregation reads from on their
phones. No audio leaves the building.

This is the source only. What used to sit beside it — 14 GB of speech and
translation models, 8 GB of voice bundles, 6 GB of natural voices, the Electron
caches, and the app data holding the operator PIN and every service transcript
— is machine state and downloaded assets, and stays in the NAS backup where it
belongs. The vendored node, cloudflared and Windows ffmpeg binaries are out too
(533 MB, re-fetched for packaging); the browser-side runtime beside them is
kept, so a checkout still serves the congregation page.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
christian
2026-09-23 15:41:21 -04:00
co-authored by Claude Opus 5
commit c9eef0672c
118 changed files with 36031 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/server/live-audio.js
+676
View File
@@ -0,0 +1,676 @@
/*
* live-audio.js — one continuous audio stream per language, as live HLS.
*
* WHY THIS EXISTS, because the obvious design is the one that fails:
*
* The natural thing is to send each sentence to the phone as its own little
* clip and play it. That is what this app did, and it cannot survive a locked
* screen. WebKit decides background behaviour from the media TYPE: an
* AudioContext is MediaType::WebAudio, which is restricted in the background,
* while an <audio> element is MediaType::Audio, which is not restricted at all.
* Web Audio is on the wrong side of that line by construction — no amount of
* keep-alive trickery moves it.
*
* Worse, iOS keeps a locked page's JavaScript and its WebSocket alive only
* while the page is continuously AUDIBLE. Stop making sound and a ten-second
* timer suspends the whole process. Ordinary pauses in a sermon are longer than
* ten seconds, so any per-sentence design loses its socket mid-service and
* cannot get it back until someone unlocks the phone.
*
* So the stream never stops. When nobody is speaking it carries encoded
* silence, which WebKit counts as audible (it never inspects sample values),
* which holds the process alive, which keeps the next sentence arriving. Idle
* silence costs about 1.9 kbit/s — 264 bytes per second per language.
*
* The audio also arrives over plain HTTP fetches driven by the media stack
* rather than over the WebSocket, so even if the socket does die while locked,
* the voice keeps playing and only the captions pause.
*
* A happy side effect: an audible media element puts the page in the
* MediaPlayback audio category instead of AmbientSound, so the ringer switch
* stops silencing it. Listeners with their phone on silent could not hear this
* app at all before.
*/
'use strict';
const fs = require('fs');
const path = require('path');
const { encodeAdts, PRIMING_FRAMES } = require('./aac-encode.js');
/* 1024 samples per AAC frame is fixed by the codec. At 22050 Hz that is
46.44 ms, so ~21 frames make a segment just under a second. Short segments
keep the live edge close; HLS players start roughly three segments back. */
/*
* Segment and window sizes are TIME — each stream derives frame counts from
* its own sample rate. A fixed count that gives 0.7s at 22050 Hz gives 0.35s
* at 44100 and shrinks retention into rotate-out failures. 0.70s segments
* (down from 0.975) move the live edge ~0.3-0.5s closer to now.
*/
const SEGMENT_SEC = 0.70;
/* ~12s of window. 8 (~7.8s) left players almost no slack: one brief stall and
the segment they wanted next had already rotated out, which reads as a fatal
error. Segments are a few hundred bytes of silence or 1-2 KB of speech, so
the memory cost of the extra headroom is nothing. */
const WINDOW_SEC = 11;
/*
* How much slack the continuous (microphone) feed carries. Paid once as delay,
* and it is the difference between a hiccup being absorbed and a hiccup being
* a hole in the middle of a word. The hardware output path does not go through
* here at all, so this costs the assistive-listening packs nothing.
*/
const CONTINUOUS_CUSHION_MS = 700;
/* A persistently jittery feed grows its cushion rather than skipping, but not
past this — beyond about two seconds the delay is its own problem. */
const CONTINUOUS_CUSHION_MAX_MS = 2100;
/*
* Extra retention, in seconds, on top of the window.
*
* A delayed listener is served the SAME segments as the room, just starting
* further back — there is only one sliding window per language and nothing to
* send "late" to one listener over HTTP. So the window has to still contain
* what they are about to ask for: retention must cover the stream delay as
* well as the normal player slack, or their next segment has already rotated
* out and the page reports a fatal error.
*
* Cheap enough not to think about: audio is 48 kbps, so a 60-second reserve is
* about 360 KB per language — under 5 MB for a dozen languages.
*/
let reserveSec = 0;
/*
* The stream plays at exactly 1x real time, so anything that queues faster than
* that falls behind FOREVER — the gap never closes on its own. Translated
* speech routinely runs longer than the original (Spanish especially), so
* without a cap the voice ends up minutes behind the preacher by the end of a
* sermon while every caption is current.
*
* When the backlog passes MAX, drop the OLDEST whole sentences until it is back
* under TARGET. Oldest, because in live translation the stale sentence is the
* worthless one — and whole sentences, because chopping mid-word is worse than
* skipping. The caption for anything dropped is already on screen.
*/
/*
* MEASURED against a real service (2026-09-13, 377 sentences replayed through
* this very module): at 6/3 the dropper threw away 73 sentences — 19.4% of the
* sermon — and the median dropped sentence was 3.0 s old. It was not
* trimming stale drift; it was firing on ordinary same-breath bursts, where
* one final becomes 2-4 sentences whose clips land together. The stream then
* drained in under 2 s and sat empty until the next clip: the "skip" and the
* "quiet" were two halves of one event.
*
* Two changes, both measured on the replay. Wider thresholds (9/6): 73 -> 28.
* Plus the rule below — only a sentence whose CAPTION is at least
* STALE_CAPTION_SEC old may be dropped: 73 -> 16 (4.2%), and every one of
* those was genuinely stale. Gating on time since push instead was tried and
* rejected: after any supply stall the flushed backlog is "fresh" by push
* time, and that variant played 94 s of stale audio.
*/
const MAX_BACKLOG_SEC = 9;
const TARGET_BACKLOG_SEC = 6;
const STALE_CAPTION_SEC = 6;
/*
* Silence appended after each sentence. Sentences are synthesised separately
* and butted together with no gap at all, which is what makes a stream of them
* sound blocky and rushed — real speech breathes between sentences.
*
* But the pause is AUDIO, and audio is exactly what we are short of when
* running behind: 320ms on a 2s sentence is another 16% of drift. So the pause
* is a luxury spent only when the stream is keeping up. Caught up, it breathes;
* behind, it gets on with it. That is what a human interpreter does too.
*/
const SENTENCE_GAP_MS = 320;
/* Raised with TARGET, or the breath between sentences would be squeezed to
nothing on 97% of sentences at the new steady state (measured). */
const GAP_COMFORT_SEC = 1.5; // below this backlog, take the full pause
const GAP_SQUEEZE_SEC = 4.0; // above this, no pause at all
/* How often the pump wakes. Not once per frame — that would be 21 timers a
second per language for no benefit. It emits however many frames the wall
clock says are due. */
const TICK_MS = 240;
const SILENCE_DIR = path.join(require('os').tmpdir(), 'church-translate-silence');
// ------------------------------------------------------------------ ADTS
/*
* Split an ADTS file into whole frames. Every frame starts with a 12-bit sync
* word and carries its own length, so a stream is just frames concatenated —
* which is exactly what makes this container splice-able and MP4 not.
*/
function adtsFrames(buf) {
const out = [];
let off = 0;
while (off + 7 <= buf.length) {
if (buf[off] !== 0xff || (buf[off + 1] & 0xf0) !== 0xf0) break;
const len = ((buf[off + 3] & 0x03) << 11) | (buf[off + 4] << 3) | (buf[off + 5] >> 5);
if (len < 7 || off + len > buf.length) break;
out.push(buf.subarray(off, off + len));
off += len;
}
return out;
}
function adtsSampleRate(buf) {
const TABLE = [96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000];
if (buf.length < 4 || buf[0] !== 0xff) return 0;
return TABLE[(buf[2] & 0x3c) >> 2] || 0;
}
/*
* One frame of encoded silence at a given rate, made once and reused forever.
* Encoded through server/aac-encode.js like everything else — same reasoning as the
* clip encoder in tts-service.js.
*/
const silenceCache = new Map();
function silenceFrame(sampleRate) {
if (silenceCache.has(sampleRate)) return silenceCache.get(sampleRate);
fs.mkdirSync(SILENCE_DIR, { recursive: true });
const wav = path.join(SILENCE_DIR, 'sil-' + sampleRate + '.wav');
const aac = path.join(SILENCE_DIR, 'sil-' + sampleRate + '.aac');
const n = sampleRate; // one second is plenty to slice from
const b = Buffer.alloc(44 + n * 2);
b.write('RIFF', 0); b.writeUInt32LE(36 + n * 2, 4); b.write('WAVE', 8); b.write('fmt ', 12);
b.writeUInt32LE(16, 16); b.writeUInt16LE(1, 20); b.writeUInt16LE(1, 22);
b.writeUInt32LE(sampleRate, 24); b.writeUInt32LE(sampleRate * 2, 28);
b.writeUInt16LE(2, 32); b.writeUInt16LE(16, 34); b.write('data', 36); b.writeUInt32LE(n * 2, 40);
fs.writeFileSync(wav, b);
/* 48k to MATCH tts-service.js's clip encoder. At 32 kbps afconvert silently
RESAMPLES 44100 Hz input down to 32000, so the Chinese stream would splice
44.1 kHz speech against 32 kHz silence — mismatched-rate frames in one
stream, which players handle by glitching. */
encodeAdts(wav, aac); // keeps the WAV's own rate — see aac-encode.js
const frames = adtsFrames(fs.readFileSync(aac));
/* Skip the first couple — the encoder's priming frames are not steady state. */
const frame = frames[Math.min(3, frames.length - 1)];
silenceCache.set(sampleRate, frame);
try { fs.unlinkSync(wav); } catch (_) {}
return frame;
}
// ---------------------------------------------------------------- streams
function segmentsToKeep() {
return Math.max(6, Math.round((WINDOW_SEC + reserveSec) / SEGMENT_SEC));
}
class LanguageStream {
constructor(lang, sampleRate, onLog) {
this.lang = lang;
this.sampleRate = sampleRate;
this.onLog = onLog;
this.frameMs = (1024 / sampleRate) * 1000;
this.framesPerSegment = Math.max(4, Math.round(SEGMENT_SEC * sampleRate / 1024));
this.keepSegments = segmentsToKeep();
this.pending = []; // [{frames, gap, id, secs, at}] — one entry per sentence; `at` is its CAPTION time
this.pendingFrames = 0; // total frames queued, for the backlog check
this.onSentence = null; // fires with (id, secs) as a sentence starts playing out
this.onDrop = null; // fires with ([ids], secondsDropped) when the trimmer discards sentences
this.dropped = 0; // sentences discarded by the trimmer, this service
this.droppedSec = 0;
this.dropTimes = []; // when, for a "recently" view on the operator page
this.building = []; // frames of the segment being filled
this.segments = []; // { seq, body, frames }
this.nextSeq = 1;
this.timer = null;
this.lastTick = 0;
this.carry = 0; // fractional frames owed from the last tick
}
start() {
if (this.timer) return;
/*
* Seed one segment of silence immediately, so /live/<lang>.m3u8 exists from
* the first moment a player can ask for it. Without this there is a ~1-2s
* window after a listener picks a language where the playlist 404s — and a
* 404 at exactly that moment is indistinguishable from "this device cannot
* play HLS", which once sent pages to the clip fallback for good.
*/
if (!this.segments.length) {
const sil = silenceFrame(this.sampleRate);
for (let i = 0; i < this.framesPerSegment; i++) this.building.push(sil);
this.closeSegment();
}
this.lastTick = Date.now();
this.carry = 0;
this.timer = setInterval(() => this.tick(), TICK_MS);
if (this.timer.unref) this.timer.unref();
}
stop() {
if (this.timer) clearInterval(this.timer);
this.timer = null;
this.continuous = false; // re-prime the cushion on the next service
this.underruns = 0;
this.reprimes = 0;
this.pending.length = 0;
this.pendingFrames = 0;
this.building.length = 0;
/* Drop the window too. Keeping it would serve a playlist that never
advances, and a player reaching the end of a frozen live playlist stalls
rather than failing — which is much harder to notice than a 404. */
this.segments.length = 0;
}
/* Queue a sentence. Frames go out at the stream's own pace, so a long
sentence simply occupies the next few seconds of the stream. */
/*
* Continuous feed (the live microphone): no sentence gap, no read-along
* announce, and an exact-sample debt ledger. Each 0.7s chunk arrives with
* 2112 samples of encoder priming; stripping two whole AAC frames removes
* 2048 of them, and the remaining +64 per chunk would stretch an hour of
* feed by ~20 seconds — the backlog trimmer would then "fix" that with
* audible chunk drops. When the debt reaches a full frame, one extra
* (mostly-priming) frame is stripped and the ledger settles.
*/
pushContinuous(buf) {
const rate = adtsSampleRate(buf);
if (rate && rate !== this.sampleRate) {
this.sampleRate = rate;
this.frameMs = (1024 / rate) * 1000;
this.framesPerSegment = Math.max(4, Math.round(SEGMENT_SEC * rate / 1024));
this.pending.length = 0;
this.pendingFrames = 0;
this.building.length = 0;
}
/*
* A JITTER CUSHION, primed once.
*
* The mic feed produces exactly real time — 0.7s of audio every 0.7s — so
* supply equals demand with nothing spare, and the steady-state backlog
* measured only ~800ms. tick() inserts SILENCE whenever the queue runs dry,
* so any hiccup longer than that cushion — a busy event loop, a WebSocket
* that bunched two chunks, a GC pause — punches an audible hole straight
* through the middle of a word. On a sentence-shaped TTS stream a hole
* lands in a pause and nobody notices; on a continuous feed it is a skip.
*
* MEASURED with a metronome-perfect feed and no network: 0 underruns, 800ms
* backlog. That is healthy but thin, and the real path has neither of those
* luxuries.
*
* Priming silence costs that much added delay ONCE, at the start, and buys
* the same amount of jitter tolerance for the whole service.
*/
if (!this.continuous) {
this.continuous = true;
this.underruns = 0;
this.cushionMs = CONTINUOUS_CUSHION_MS;
this.reprimes = 0;
}
/*
* SELF-HEALING, and this is the part that matters.
*
* A cushion primed once is spent once: the first hiccup big enough to
* empty the queue eats it, and every hiccup after that is an audible hole
* because the buffer never comes back. MEASURED — two stalls (500ms and
* 900ms late) took the backlog from 1472ms to 704ms permanently, and the
* stream reported 15 underrun frames.
*
* So top up whenever the queue is found EMPTY at push time. Empty means
* silence has already gone out — the audio is lost either way — and adding
* the cushion back is the difference between losing it once and losing it
* every time. Never top up a queue that still holds audio; that would
* insert silence into the middle of a word to fix a problem that has not
* happened.
*
* A feed that keeps stalling earns a bigger cushion, up to a ceiling: more
* delay is a fair price for not being punched full of holes, but it cannot
* grow without limit.
*/
if (this.pendingFrames === 0) {
if (this.reprimes > 0) {
this.cushionMs = Math.min(CONTINUOUS_CUSHION_MAX_MS, this.cushionMs + 350);
}
const sil0 = silenceFrame(this.sampleRate);
const cushion = Math.round(this.cushionMs / this.frameMs);
const pad = [];
for (let i = 0; i < cushion; i++) pad.push(sil0);
this.pending.push({ frames: pad, gap: 0, id: null, secs: 0 });
this.pendingFrames += pad.length;
this.reprimes++;
this.onLog('live mic feed for ' + this.lang + ': ' +
(this.reprimes === 1 ? 'primed ' : 'refilled ') +
Math.round(cushion * this.frameMs) + 'ms of jitter cushion' +
(this.reprimes > 1 ? ' (refill #' + this.reprimes + ')' : ''));
}
/* Encoder priming, from aac-encode.js — 2 frames under afconvert, 1 under
ffmpeg. The +64 remainder below is the same under both (measured). */
let strip = PRIMING_FRAMES;
this.sampleDebt = (this.sampleDebt || 0) + 64;
if (this.sampleDebt >= 1024) { strip += 1; this.sampleDebt -= 1024; }
let frames = adtsFrames(buf);
if (frames.length > strip + 2) frames = frames.slice(strip);
if (!frames.length) return;
this.pending.push({ frames, gap: 0, id: null, secs: (frames.length * 1024) / this.sampleRate });
this.pendingFrames += frames.length;
/* No trimBacklog here: a live feed produces exactly real time by
construction (the debt ledger guarantees it), so the backlog stays a
constant one-chunk bubble. */
}
push(buf, terminal, id, captionAt) {
const rate = adtsSampleRate(buf);
if (rate && rate !== this.sampleRate) {
/* A voice was swapped underneath us. Splicing mismatched rates produces
chipmunk audio, so rebuild the stream instead. */
this.onLog('live audio for ' + this.lang + ' switching ' + this.sampleRate + ' -> ' + rate);
this.sampleRate = rate;
this.frameMs = (1024 / rate) * 1000;
this.framesPerSegment = Math.max(4, Math.round(SEGMENT_SEC * rate / 1024));
this.pending.length = 0;
this.pendingFrames = 0;
this.building.length = 0;
}
/*
* The encoder prepends priming to every clip — afconvert 2112 samples,
* ffmpeg 1024 (both measured; see aac-encode.js) — and ADTS
* has no edit list to declare it, so the decoder plays it as dead silence
* at every sentence boundary — 96-142 ms per join. Dropping the first two
* frames removes 2048 of those samples; frame 2 keeps frame 1 as its MDCT
* overlap partner, so the speech onset survives (measured error confined to
* the first 1024 samples, RMS 10-21 LSB). Dropping THREE destroys the onset
* (-5 dBFS error, 12-38% energy loss). Do not raise this number.
*/
let frames = adtsFrames(buf);
if (frames.length > PRIMING_FRAMES + 2) frames = frames.slice(PRIMING_FRAMES);
if (!frames.length) return;
/* The gap rides with the sentence so that dropping one drops its pause too. */
/*
* A full stop earns a full breath; a fragment the VAD cut mid-thought gets
* only a beat, because a sentence-final pause after half a sentence is
* exactly the halting rhythm listeners hear as "blocky".
*/
const ended = terminal === '.' || terminal === '!' || terminal === '?' ||
terminal === '\u3002' || terminal === undefined;
const gapMs = this.gapMsForBacklog() * (ended ? 1 : 0.4);
const gap = Math.round(gapMs / this.frameMs);
const sil = silenceFrame(this.sampleRate);
const secs = (frames.length * 1024) / this.sampleRate; // speech only, sans gap
for (let i = 0; i < gap; i++) frames.push(sil);
/* `at` is when the sentence was SAID (the caption's timestamp), not when
its audio arrived — the difference is the whole dropper fix above. */
this.pending.push({ frames, gap, id, secs, at: Number(captionAt) || Date.now() });
this.pendingFrames += frames.length;
this.trimBacklog();
}
backlogSeconds() { return (this.pendingFrames * 1024) / this.sampleRate; }
gapMsForBacklog() {
const b = this.backlogSeconds();
if (b <= GAP_COMFORT_SEC) return SENTENCE_GAP_MS;
if (b >= GAP_SQUEEZE_SEC) return 0;
/* Taper rather than switch, so the rhythm does not lurch at a threshold. */
const t = (b - GAP_COMFORT_SEC) / (GAP_SQUEEZE_SEC - GAP_COMFORT_SEC);
return Math.round(SENTENCE_GAP_MS * (1 - t));
}
trimBacklog() {
if (this.backlogSeconds() <= MAX_BACKLOG_SEC) return;
/*
* Shorten the breathing room before losing anyone's words. A clipped pause
* costs a little naturalness; a dropped sentence costs the sentence. Only
* once every pause is gone is dropping the right answer.
*/
for (let i = 0; i < this.pending.length && this.backlogSeconds() > TARGET_BACKLOG_SEC; i++) {
const p = this.pending[i];
const trim = Math.min(p.gap, p.frames.length - 1);
if (trim <= 0) continue;
p.frames.length -= trim;
this.pendingFrames -= trim;
p.gap -= trim;
}
if (this.backlogSeconds() <= MAX_BACKLOG_SEC) return;
let dropped = 0;
let droppedSec = 0;
const ids = [];
while (this.pending.length > 1 && this.backlogSeconds() > TARGET_BACKLOG_SEC) {
/* Never cut the sentence that is coming out of the speaker mid-word —
if the head has started playing, the oldest DROPPABLE one is next. */
const at = this.pending[0].announced ? 1 : 0;
if (at >= this.pending.length) break;
/* Only a sentence the preacher said STALE_CAPTION_SEC ago is stale enough
to lose. If the oldest droppable one is fresher than that, everything
behind it is fresher still — stop, and let the hurry work it down. */
if (Date.now() - (this.pending[at].at || 0) < STALE_CAPTION_SEC * 1000) break;
const gone = this.pending.splice(at, 1)[0];
this.pendingFrames -= gone.frames.length;
droppedSec += (gone.frames.length * 1024) / this.sampleRate;
if (gone.id != null) ids.push(gone.id);
dropped++;
}
if (dropped) {
this.dropped += dropped;
this.droppedSec += droppedSec;
this.dropTimes.push(Date.now());
if (this.dropTimes.length > 200) this.dropTimes.splice(0, this.dropTimes.length - 200);
if (this.onDrop) { try { this.onDrop(ids, droppedSec); } catch (_) {} }
}
if (dropped) {
this.onLog('live audio ' + this.lang + ': dropped ' + dropped +
' stale sentence(s) to catch up (backlog now ' +
this.backlogSeconds().toFixed(1) + 's)');
}
}
tick() {
const now = Date.now();
const elapsed = now - this.lastTick;
this.lastTick = now;
/* Frames due since the last wake, carrying the fraction forward so the
stream does not drift slow. */
const exact = elapsed / this.frameMs + this.carry;
let due = Math.floor(exact);
this.carry = exact - due;
/* After a long stall (a laptop sleeping) do not dump a minute of silence
into the buffer — just resume at the live edge. */
if (due > this.framesPerSegment * 2) due = this.framesPerSegment * 2;
const sil = silenceFrame(this.sampleRate);
for (let i = 0; i < due; i++) {
let frame = sil;
while (this.pending.length && !this.pending[0].frames.length) this.pending.shift();
if (!this.pending.length && this.continuous) {
/*
* The queue ran dry on a feed that is supposed to be unbroken. This is
* the skip, and it used to happen silently — the stream simply played
* a little silence and nobody could tell it from the room being quiet.
* Counted and reported, because "it skips sometimes" is not something
* anyone can act on.
*/
this.underruns = (this.underruns || 0) + 1;
if (this.underruns === 1 || this.underruns % 25 === 0) {
this.onLog('live mic feed for ' + this.lang + ': ' + this.underruns +
' underrun frame(s) — audio arriving slower than real time');
}
}
if (this.pending.length) {
const head = this.pending[0];
/* The first frame of a sentence leaving the queue is the moment it
starts existing as sound — which is what the read-along highlight on
the phones needs to know. */
if (!head.announced) {
head.announced = true;
if (this.onSentence && head.id != null) {
try { this.onSentence(head.id, head.secs); } catch (_) {}
}
}
frame = head.frames.shift();
this.pendingFrames--;
}
this.building.push(frame);
if (this.building.length >= this.framesPerSegment) this.closeSegment();
}
}
closeSegment() {
const body = Buffer.concat(this.building);
this.segments.push({ seq: this.nextSeq++, body, frames: this.building.length });
this.building = [];
while (this.segments.length > this.keepSegments) this.segments.shift();
}
segmentDuration() { return (this.framesPerSegment * 1024) / this.sampleRate; }
/*
* `backSec` is how far behind the live edge this listener should start. 0 for
* the room; the stream delay for someone who came through the tunnel. Same
* playlist, same segments, different starting position — which is the only
* way to delay one listener of a shared HLS window and not the others.
*/
playlist(backSec) {
if (!this.segments.length) return null;
const dur = this.segmentDuration();
/* Never ask for a position older than the window actually holds, or the
player seeks to something that has rotated out and stalls. */
const held = Math.max(0, (this.segments.length - 1) * dur);
const back = Math.min(Math.max(0, Number(backSec) || 0) + 1.2, held);
const lines = [
'#EXTM3U',
'#EXT-X-VERSION:3',
'#EXT-X-TARGETDURATION:' + Math.ceil(dur),
'#EXT-X-MEDIA-SEQUENCE:' + this.segments[0].seq,
/* Without this a player starts three segments back and stays there for
the whole service. Measured: 4.92 s of lag becomes 1.61 s. */
'#EXT-X-START:TIME-OFFSET=-' + back.toFixed(2) + ',PRECISE=YES'
];
for (const s of this.segments) {
lines.push('#EXTINF:' + dur.toFixed(3) + ',');
/* Relative to the PLAYLIST's directory, which is /live/ — so the language
has to be in the path or the player asks for /live/17.aac and gets a
404. The playlist itself is /live/<lang>.m3u8, not /live/<lang>/. */
lines.push(this.lang + '/' + s.seq + '.aac');
}
return lines.join('\n') + '\n';
}
segment(seq) {
const s = this.segments.find((x) => x.seq === seq);
return s ? s.body : null;
}
}
// ----------------------------------------------------------------- facade
class LiveAudio {
constructor(onLog) {
this.streams = new Map();
this.onLog = onLog || (() => {});
}
/* Default rate is a starting guess; the first real clip corrects it. */
ensure(lang, sampleRate) {
let s = this.streams.get(lang);
if (!s) {
s = new LanguageStream(lang, sampleRate || 22050, this.onLog);
s.onSentence = (id, secs) => {
if (this.onSentenceStart) { try { this.onSentenceStart(lang, id, secs); } catch (_) {} }
};
s.onDrop = (ids, secs) => {
if (this.onDropped) { try { this.onDropped(lang, ids, secs); } catch (_) {} }
};
this.streams.set(lang, s);
this.onLog('live audio stream open for ' + lang);
}
s.start();
return s;
}
push(lang, buf, sampleRate, terminal, id, captionAt) {
const s = this.ensure(lang, sampleRate || adtsSampleRate(buf));
s.push(buf, terminal, id, captionAt);
}
/*
* What the trimmer has thrown away, per language — the number that was
* invisible for a whole service. `recent` is drop EVENTS in the last minute,
* which is what an operator can act on; totals are for the log.
*/
stats() {
const out = {};
const cut = Date.now() - 60000;
for (const [lang, s] of this.streams) {
if (!s.dropped && !s.underruns) continue;
out[lang] = {
dropped: s.dropped,
droppedSec: +s.droppedSec.toFixed(1),
recent: s.dropTimes.filter((t) => t > cut).length,
underruns: s.continuous ? (s.underruns || 0) : undefined
};
}
return out;
}
/* Per-language backlog in seconds — how far the voice is behind the text.
The engine slows or quickens synthesis from this, like an interpreter. */
backlog() {
const out = {};
for (const [lang, s] of this.streams) {
if (s.timer) out[lang] = +s.backlogSeconds().toFixed(2);
}
return out;
}
/*
* Called whenever the set of languages anyone is listening to changes.
* OPENS a stream for every active language rather than only restarting ones
* that already exist — the stream has to be audible before the first sentence,
* not after it, or the phone is suspended before that sentence can arrive.
*/
retain(langs, rateFor) {
const keep = new Set(langs);
for (const lang of keep) this.ensure(lang, rateFor ? rateFor(lang) : 22050);
for (const [lang, s] of this.streams) {
if (keep.has(lang)) continue;
if (s.timer) { s.stop(); this.onLog('live audio stream idle for ' + lang); }
}
}
stopAll() { for (const [, s] of this.streams) s.stop(); }
playlist(lang, backSec) {
const s = this.streams.get(lang);
return s ? s.playlist(backSec) : null;
}
/*
* Keep enough history to serve a listener who is `sec` behind the live edge.
* Called whenever the stream delay changes, and applied to streams that
* already exist as well as new ones.
*/
reserve(sec) {
const n = Math.max(0, Math.min(300, Math.round(Number(sec) || 0)));
if (n === reserveSec) return;
reserveSec = n;
const keep = segmentsToKeep();
for (const st of this.streams.values()) {
st.keepSegments = keep;
/* Shrinking takes effect on the next rotate; growing must not drop
anything that is already held. */
while (st.segments.length > keep) st.segments.shift();
}
this.onLog('audio window now ' + ((WINDOW_SEC + n)).toFixed(0) + 's (delay reserve ' + n + 's)');
}
segment(lang, seq) { const s = this.streams.get(lang); return s ? s.segment(seq) : null; }
}
module.exports = { LiveAudio, adtsFrames, adtsSampleRate, silenceFrame };