- config.json is NA Ingest's live settings: the READY hours (Tue 14-22, Wed 9-19, Sun 5-14) and the
auto-fix hours (Tue 14-22, Wed 9-21, Sun 6:30-14) set on the settings page.
- Settings page: a draft carries what its page started from. A tab that loaded the settings before
they changed (an Apply elsewhere, a program update) now has only its own edits put on top of the
live settings; before, it saved its whole older copy and Apply quietly undid the newer values
(it took ready.prop_host out twice today).
- install.sh refuses (exit 4) to replace live settings that came from an Apply (their fingerprint is
a history entry's sha_after) unless --replace-config; --keep-config still updates code only.
- Tests pin their own auto-fix and READY hours, so the campus can change its own; the exit-timeout
test expects 310 s (the Tascam outlet power-cycle).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- For every item in the plan's ProPresenter playlist: the macros its slides fire (read on the
ProP Mac from Configuration/Macros and the .pro files), the RossTalk "CC page/row/col" each
macro sends to Companion, the Companion buttons that one presses in turn, down to Resolume's
/composition/columns/N/connect. Each such column must hold a clip on some layer (one whose
ignore-column-trigger is not On), and Resolume must be on the Service deck.
- COL N EMPTY / COL N MISSING name the column and the items that use it. The hold only puts
Resolume back on its deck; it never plays a clip. "Something is playing" is no longer checked.
- ready.prop_host replaces ready.led_clip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ON/OFF on every key, check, fix, Restart and READY key, computer, MIDI link, HyperDeck,
Dante device, watched Companion connection, switcher and Spotify check, outlet fix, and one
main switch each for auto-fix, the auto switch and the READY page. Switched off = kept in
the settings, grey OFF on the deck, never checked or run (effective_config in the watchdog).
- Settings › Auto switch: a plain summary, what counts as down (each trigger editable in place:
address, program and computer, Dante device, the button it presses, timings), the hours
folded away, the buttons; + Button, + What counts as down, Remove.
- Pick a Companion button by clicking it on its page, drawn like Companion (read-only API:
/api/companion/pages and /api/companion/page).
- STREAM STB / STREAM FOH Restart keys removed (the Studio B page has its own); the alert names
the switch-back button instead.
- Dante screen: shows the DANTE key's device list again (it picked STB AUDIO's one device).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new fix kind "companion_press" presses any Companion button over its HTTP API (page, row, col).
A fix marked "failover" runs by itself within seconds of its check failing: inside the "failover"
windows (Sunday 06:30-13:30), even while ALARMS is muted, once per fault, never back the other way.
The hidden STB AUDIO key (inside AUDIO) watches Studio B's ping every second, Pro Tools over SSH
every 2 s and its Dante device; each check's fix is the FOH Backup press. The keys say ON FOH BACKUP
until STREAM STB on the Restart page switches back. Settings page: the press builder, the failover
tick and undo, the Failover calendar, help entry "A fallback button".
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Keys › READY page: + READY key (tap to confirm, a checklist line the page checks itself, or
any check), and the same check/fix editor the Status page keys have; delete a key.
- Settings › READY page: when gaps turn red (calendar), every address and folder it looks at,
and the lists the slides must match.
- Help: "How do I add…", step by step: a Companion connection, a ping, a program, a fix,
auto-fix, a checklist line, where READY looks, a computer, undo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- DIGICO (in the AUDIO group): Companion's DiGiCo_FOH connection, the digico-relay helper on
NA Ingest (hold AUDIO restarts it), and a ping to the console at 10.1.60.100 (the OSC
connection reads good even with the console off).
- Settings page: "+ Key" on an unwatched Companion connection asks which group key to put it
in when the Status page has no free spot (it used to make a key nobody could see); the
connection field of a check suggests Companion's connection names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- readiness.py looks every 2 minutes (never while a HyperDeck records) at this weekend's PCO plan
and what is in place for it: the Ableton set (songs, mix, routing, lighting cues), ProPresenter
(every PCO item, MC announcements, baptism names, timelines, backgrounds), the LTC note of every
song against the timecode the tracks play, the Band Lineup TV, Resolume's LED cross, and the
tracks and ProP bundles for the locations.
- READY key on the Status page (row 2, col 7; SPOTIFY now shows inside SHOW) opens page 24.
Amber until done, blinking red from Saturday noon; not part of ALL. Tap keys for BOH mix,
mics and stage reset every Monday.
- Holds fix what can be fixed: re-sync the lineup, put the cross back up, write LTC into PCO,
copy tracks to locations, look again. Hold READY fixes everything.
- Settings page: a READY page tab (live keys, drag to move, why each key is amber), and Apply
re-imports page 24.
- Auto-fix: "while_muted" fixes keep running with ALARMS muted (Spotify playlist).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The committed config.json is the campus as applied on NA Ingest; the READY-page work still in
progress in this checkout is left out of it. The outlet fix now says, when the Dante card does not
come back, that only the unit's power button resets it (verified: a 45 s cut does not), and still
reconnects Companion.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A draft started before someone else's Apply used to carry that older config and would have put
the newer changes back. Now every read of the draft replays only the draft's own changes onto the
live settings (base config from history), so the review lists exactly what you changed. Values
read in words (windows as days and hours), long ones fold open with was/now, commands are no
longer red, and warnings the live config already had are not repeated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
settings/outlet.py talks to Tuya's cloud with the campus's own project keys (kept in outlets.json
beside the config, 0600, never in the repo): list, status, on/off, and 'cycle <outlet> --ping
<address>' which leaves a unit that answers alone, cuts power 8 s, and waits for the address to
answer; refused while a HyperDeck is recording. A builder 'Power-cycle a smart outlet' makes the
fix from fields; the TASCAM key's ping check now holds to it (verified live: the hung DA-6400
answered again 26 s after the cycle and Companion reconnected). Settings > Outlets: keys, the
outlet list with live state and on/off, + Outlet from the account's devices. Calendar in 12-hour time.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Top bar with icon navigation (Overview, Keys, Computers, Settings, Logs, History, Help); no side
menu; every explanation moved into a hover hint; the [how] lines no longer shift the layout.
- One Settings page with sections: Playlist, Auto-fix (a paintable weekly calendar + which fixes),
Changes wait (calendar), HyperDecks, MIDI links, Computers, Dante, Switcher, Companion (with the
names check), Network, Site & PIN, Program & campus, Advanced (every field by screen).
- Fixes live under Keys as a tab; the jobs became sections or + buttons where they belong.
- Editing is switched on with a PIN (digits) plus a first name for History; the overview's
status line became chips; 'Needs attention' instead of a person's name, and it no longer fires
after a page-only Apply (page-only.json marker); last change comes from History.
- Deployed to NA Ingest; the campus PIN set there and in config.json.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every screen was swept in the browser on NA Ingest with Editing on (31 screens, no script
errors); the job dialogs ran taller than a phone screen, so dialogs now scroll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A launchd-run python3 gets 'Operation not permitted' on ~/Library/CloudStorage; sshd does not. The
page now lists and reads the folder over ssh to localhost with the panel's own key when the local
read is blocked (the Ingest Mac's Setup card gains that authorise step). Pick the newest verified
on NA Ingest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Site setup > Companion lists every connection, Dante device and MIDI session the keys use that
this campus does not report, with a map-to choice that renames it across checks, restart keys
and fixes. The ATEM section explains the Full Disk Access step when the cloud folder looks empty
to the background page. Verified on NA Ingest; Install the staged folder tested live there too.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Overview jobs: Change the playlist, Add or remove a HyperDeck, Add a MIDI link, Add a computer,
Make a PC program startable; 'Add a key for it' on unwatched Companion connections.
- Computers > Setup: steps with a [how] each and the set-up scripts generated per computer by
settings/scripts.py from the templates in remote-access/ (no site-shaped value left in them);
Remove this computer drops the fixes and checks that used it.
- Site setup > Site: the running and staged code, Install the staged folder (a one-shot launchd job
running install.sh --keep-config, refused while live), Export site bundle (zip), New campus from a
bundle (this campus's site block stays; every copied computer is flagged until touched).
- sites/: na -> the repo config, _template = the empty campus; install.sh --site <id>.
- Review reads appended checks as 'added'; tests for all of it.
- Deployed page-only to NA Ingest. (config.json, status_panel.py and readiness.py are left to the
READY-page work in progress in the same tree.)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keys screen: every key is a draggable chip (empty cells and the tray are drop zones), + New key and
+ New restart key dialogs, Delete this key, Companion page numbers, 'this campus has a Restart
page'. A taken cell is refused on the spot; 'Layout changed: Apply will re-import ...' comes from
the server's plan. Screens wait for a pending draft save before asking the server about it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
settings/builders.py renders a fix's command from a builder (template) and its fields; validate
refuses a config whose stored command no longer equals what its fields build; the Fixes screen and
the remote checks in the key editor rebuild through /api/build and /api/build-check. config.json
adopts templates for all 27 fixes and 5 remote checks (ProPresenter's restart loop and Ableton's
quoting normalised, same behaviour); tests/test_builders.py (174 tests pass). Deployed to NA Ingest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- settings/apply.py: the Apply pipeline (check, keep history, write, restart the watchdog, re-import
pages by the marker-only rule, recolour page keys), rollback when the watchdog does not come back,
the recording guard with the Emergency (LIVE) override, history entries and Go back / Undo.
- settings/server.py: the write routes behind the editing code (draft PUT/DELETE with 409 on a
conflict, validate, apply, restore, undo, watchdog restart/stop/start, commands to the watchdog,
the switcher file upload / pick newest / discard, gear notes).
- settings/ui/app.js: every screen grows inputs with Editing on; keys (checks, fixes, hold-always,
members), computers, fixes, site setup (schedules, Dante devices, Spotify, ATEM), Review & Apply
with live progress, History with Go back and Undo, Run this fix now.
- schema.unmask, push_page.push, nav_keys.colour; tests for all of it (163 pass).
- Verified live on NA Ingest: a page-only Apply, a restart Apply, Undo, history.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Overview (problems in the room with the hold action, the live deck, one line per computer),
Keys (both pages, what each key will say and what holding does), Computers (map with the keys
that turn red, cards with a light per thing that runs there from the watchdog's snapshot and a
read-only probe, Gear, Setup), Logs, History (the installer's copies for now), Help (four cards,
the generated key guide, cheat sheet and Where things are, glossary, Companion facts), Site setup
and Fixes read-only. The editing code is checked (with a lockout) but nothing can change yet.
Also: faults are attributed to the computers their check is about; the installer refuses while a
HyperDeck is recording (--now overrides); the Lighting PC's MIDI session is listed; the probes read
sudo's answer correctly and expand $HOME outside the quotes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- settings/server.py: read-only routes (overview, keys, computers, gear, companion, logs, history,
help facts, key guide / cheat sheet / where things are), the editing-code check with a lockout,
LAN-only binding with a per-minute rebind
- settings/probes.py: fixed read-only probes per computer; whole lists fetched and matched locally
- settings/keyguide.py: the generated teaching pages
- settings/ui: the page, every screen read-only
- status_panel.py: snapshot.json every 5 s for the page; the Companion database lookup prefers the
running build's folder and skips a database that will not open (a stale v5.1 folder was breaking it)
- config.json: every computer's record (what runs there, network ports, done-by-hand ticks), the
editing code (kept in the repo, the user's choice), an empty gear list
- install.sh: the settings page as a second LaunchAgent; settings/ and schema.py deployed
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- schema.py: every setting with type, default, help, owner screen and the rules a config must
pass; tests/test_schema.py keeps it in step with the code (every key the code reads is declared,
every default matches, check types and fix kinds agree, the glossary covers every word a key can
show, today's config still builds today's pages apart from the key notes)
- status_panel.py: VERSION + started.json; SIGTERM lets a running fix finish and starts nothing new
(RESTARTING SOON); fixes-running.json and live-now.json for the settings page; state.json keeps
the ALARMS override, acknowledgements, auto-fix tries and fix times across restarts, keyed by
what a check is about rather than its position; 'test' and 'resurvey' commands; queued commands
dropped at start; overnight windows; a HyperDeck switched off is always switched back on;
the spotify fix kind (playlist set once, in config 'spotify'); Windows remote checks batched for
cmd.exe with conditional echoes; every fix's kind and host validated; never_auto; wording and
numbers from config (site block, timing, page_hold_ms, auto-fix keep_months); hyperdeck_probe
removed; --check and --exit-timeout
- make_page.py: numbers from config, markers on every key, nav templates per Companion version,
--out; push_page.py: a page is overwritten only when every key carries a marker (foreign keys
named by position), secret-free fetch, host from the site; nav_keys.py: host from the site;
netgear.py: switches and the shared-port limit from config
- install.sh: validates with the new code first, keeps prev/ for --rollback, --keep-config,
derived ExitTimeOut, wider deploy list, waits for the start stamp
- config.json: site, timing, spotify sections; Windows hosts marked; deployed to NA Ingest and
both pages re-imported once (--force) so every key carries its marker
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Plan for the settings page on NA Ingest (every setting editable, what runs
where, multi-campus), built with a plan-and-audit workflow and consolidated
against a decisions list; the audit's 157 findings and their dispositions;
the 544-item inventory of everything configurable today.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
atem_watch.js (run with Companion's node + its atem-connection) keeps one
connection to the switcher and compares, every second, every Fairlight
source's mix option and fader and every camera's settings with
atem-reference.xml (the 2026-10-07 14:59:55 restore file). ATEM shows
ATEM AUDIO WRONG / ATEM CAMERA WRONG; the hold and Sunday auto-fix send
back only what differs. ATEM SW MIDI ON is fixed by quitting ATEM Software
Control, switching midiEnabled off and reopening it. Live check: switcher
matches the file (0 differences, 1440 camera values).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New 'pref' check reads an app setting on NA Ingest (defaults read) every
10 s; ATEM shows ATEM SW MIDI ON if midiEnabled is not 0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A LaunchAgent helper on that Mac (Accessibility) reads Options > Solo Mode
every second into a status file and ticks AFL/Latch on request. APPS
alarms PT SOLO NOT AFL (hold fixes it; in the Sunday auto-fix list) and
PT SOLO CHECK OFF when the helper is not running or not allowed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- NA Tracks and the ProPresenter Mac: one long-lived ssh each streams the
MIDI sessions (checked twice a second, sent on change) instead of an ssh
login every 5 s; a silent stream gives no verdict after 12 s.
- NA Ingest's own session is read twice a second (was every 2 s).
- MIDI links show a drop after 1 s (was 3 s); everything redraws twice a
second (was once).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At 07:45:55 the six tcp-HyperDeck helper connections (behind the 'reboot
all' button, watched by no tile) flickered together; each counted as its
own group, so NET showed a blip and auto-fix paused, though the real
HyperDeck connections and everything else stayed up.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only files named exactly '<YYYY-MM-DD> <Weekday> Auto-Fix.log' in that
folder; checked when a new day's file starts and when the watchdog starts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
~/Desktop/Auto-Fix Logs/<YYYY-MM-DD Weekday> Auto-Fix.log, written line by
line as it happens: auto-fix on/off, each problem and what was done (try n
of 2), the result, when the fault cleared, give-ups, and when it held back
(network problem, several Dante devices gone). Tests write to a temp
folder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sundays 06:30-13:30, with ALARMS on, a fault that has lasted 20 s is fixed
by itself if the fix is gentle (reconnects, HyperDeck off/on, Tascam, MIDI,
rtpMIDI, starting a closed Vista/Resolume/OBS/Ableton/Translate, DVS on one
Mac). At most 2 tries 90 s apart; key shows AUTO-FIXING. Anything that
closes a running program stays a hold (load_config refuses it).
A review before going live (2 lenses, each finding verified) confirmed and
this fixes: a Dante-wide "gone" (Ingest's view) would restart DVS on every
Mac every minute - several devices gone at once now means hands off; the
try limit reset whenever a fix made its fault blink - tries are kept until
the fault has been gone 10 min; DVS on Ingest kills ProdCom - removed from
the auto list; restarting dead connections looked like a network burst -
only a connection good in the last 30 s counts; a restart key could kill
an app auto-fix was starting; concurrent MIDI reconnects overwrote each
other - now one worker. Auto-fix errors are caught so the panel keeps
running. 105 tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bible TV PC's Wi-Fi dropped for ~30 s at 06:17 on 2026-10-11; the
hold could not reach it and showed FIX FAILED, though nothing on the PC
was broken. An ssh fix that cannot reach its machine now shows e.g.
"BIBLE APP NOT REACHABLE".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a deck restarts, Companion's HyperDeck connection does not come back
by itself and a plain restart of the connection does not help; switching
it off and on does. Each HD key's hold now does exactly that (even when it
looks green), waits 15 s and checks: DONE, STILL OFFLINE (deck pings) or
NO NETWORK. Decks are no longer rebooted by the panel; a recording deck is
left alone. Tested live on HD2 (off/on, reconnected in 17 s).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A Restart page key flashes red "HOLD TO FIX" while any check that its
fix (or a fix/connection in its "watch") covers is down, e.g. DVS
TRACKS when DANTE misses NA-Tracks.
- Ableton's start and restart reopen the set Live last opened (read from
its Log.txt before quitting; templates skipped) instead of the default
template. Tested live: ABLETON and TRANSLATE restart keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the network test SuperRack sat in NA Ingest's session as
"initializing"; the hold saw it listed and did nothing ("already").
connect_peer now drops a listed-but-not-live peer and adds it back,
building the second write from the first so a slow re-read cannot leave
it dropped. Tested live today: DVS x4, APPS and INGEST holds, NET clear,
CAPTIONS and BRIDGE restart keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The remembered network blip is a flashing amber warning (new "notice"
level, not counted as DOWN): press NET to stop the flashing, hold NET 2 s
to clear it, otherwise it goes by itself after 5 minutes (was 3, solid).
- The user moved the Restart page to page 20: config, the RESTART PAGE key
on page 19 and both pages re-pushed. A scan of all pages finds no other
button pointing at the old page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Page 23 "Restart": hold a key 2 s to restart DVS (4 Macs), ProPresenter,
Ableton, Church Translate, ProdCom, ProdDash, captions, the bridge or the
Tascam connection even when the status page says it is fine ("force",
shared 60 s guard). RESTART PAGE key on page 19, STATUS PAGE back.
- Holds carry Companion's clock; ones queued while the watchdog was down
are dropped instead of firing together later.
- TASCAM hold always reconnects and checks again after 20 s: HUNG - POWER
CYCLE IT (pings, control port hung), NO NETWORK, or turns a disabled
connection on. Tascam pinged at its Companion address.
- INGEST: ProdCom (:24480), ProdDash's page (:24500), the caption screen;
DVS on Ingest restarts ProdCom afterwards. RESI: the encoder (10.1.10.202).
- Windows starts check the app really started (FIX FAILED otherwise); OBS's
start task runs elevated (OBS is "Run as administrator"); setup script
does that for any such app and gives sshd restart-on-failure.
- push_page.py checks what is on the target page, not its name (pages 22
and 97 are named "PAGE" but hold buttons); --which restart.
- Companion custom variables are written in the request body (no URL limit).
Reviewed by a 3-lens review with per-finding verification; 7 confirmed
issues fixed. Tested live: OBS hold opens OBS; Restart page TASCAM key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lighting-midi (Companion's MIDI out through NA Ingest's session) on MIDI;
http://10.1.14.225:1025/ (ProPresenter's network page, used for the band
lineup / Prop TV) on APPS, hold restarts ProPresenter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It flickers to "warning" every 15-20 minutes and made APPS blink for
nothing. "ignore_warning" on a conn check keeps offline, error and
disabled as alarms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A power-off test today took 18-27 s to show on RF (Companion's Shure
modules' own timeout). Each Axient, ULXD and PSM receiver is now pinged
every 3 s: "AXIENT 1 NO PING" in about 6-9 s. A ping check may name a
Companion "conn"; its address is read from Companion's settings database
(read-only, every 60 s), with "host" as the fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found in today's power-off test: a Shure/PSM check reports "conn" while its
connection is down and per-channel results once it is back, so the
"AXIENT 1 OFFLINE" tracker never saw OK. The button looked green, but no
CLEAR was logged and a hold on RF would still have restarted the Shure
connections. A tracker missing from a round now gets an OK update.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Art-Net heartbeat moves from APPS to a VISTA tile ("VISTA / SILENT";
"VISTA SILENT" on SHOW and ALL), hold starts Vista if it is not running.
SHOW now groups MIDI, APPS and VISTA. Tested live before the move: closing
Vista and Resolume alarmed, and each hold brought them back (Vista 5 s,
Arena 9 s after the hold).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Hold MIDI on "LIGHT PC MIDI GONE" starts the rtpMIDI service, then its tray
app, only if they are not running (a restart drops every session).
- The Lighting PC's rtpMIDI service now has Windows recovery actions
(restart after 5 s, 5 s, 30 s); windows-setup.ps1 sets the same.
- coremidi.disconnect_peer (Disconnect in Audio MIDI Setup), used to test the
alarms: all six links dropped and came back (four by themselves, two by
the hold).
- Audio MIDI Setup need not be open: quitting it on NA Ingest left the
session connected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds Tracks -> Livebox, Tracks -> Studio B and ProPresenter -> Lighting
(read on the ProPresenter Mac over SSH), each with the hold-to-reconnect fix.
coremidi.py now recognises the network MIDI driver on macOS 26.2 too
("AppleMIDIRTPDriver"; 26.5 calls it "AppleMIDINetworkDriver"), which is why
the ProPresenter Mac's session looked empty before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
macOS's MIDINetworkSession API only reaches the default session (blank and
disabled on NA Ingest and NA Tracks), so coremidi.connect_peer writes the
named session's "apple.midirtp.session" settings with the peer added, as
Audio MIDI Setup's Connect does. Existing peers are passed back unchanged.
The watchdog remembers each peer's last address (config "address" is the
fallback) and runs it locally for NA Ingest's session or over SSH on the
Tracks Mac for NA Tracks' (FX lights). Checked live: both Macs answer
"already" while the Lighting PC is connected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tracks sends the FX lighting cues to the Lighting PC through NA Tracks'
network MIDI session. A midi_peer check may now name a "host": the watchdog
reads that Mac's sessions over SSH every 5 s, sending this folder's
coremidi.py over stdin (nothing installed there). Unreachable = no verdict;
the Tracks Mac's own ping check covers that.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its Bonjour firewall rules only allow Public; switched to Private it dropped
out of the MIDI directory and could not rejoin NA Ingest's session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tested live: restarting rtpMIDIService brought the Lighting PC's Bonjour
advert back but dropped it from the NA Ingest and NA Tracks sessions, and it
did not rejoin them. macOS only exposes the default network session to
scripts, not NA Ingest's named one, so there is no safe automatic reconnect.
MIDI now has no hold fix; the README says how to reconnect by hand.
ssh fixes run with LogLevel=ERROR so warnings stay out of the log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- OBS: start OBS on the LiveStream PC if it is not running, then reconnect
Companion. If the PC does not answer, send Wake-on-LAN and show WAKING PC.
- ARENA: start Resolume Arena on the Lighting PC if it is not running, then
reconnect Companion. VISTA SILENT: start Vista if it is not running.
A running OBS, Arena or Vista is never closed.
- MIDI (LIGHT PC): restart the rtpMIDI service; it reconnects to its saved
peers (NA Ingest, NA ProP, NA Tracks).
- Fixes may set "reconnect" (restart the check's Companion connection after
the command succeeds); hosts may set "mac"/"wake_broadcast" for Wake-on-LAN.
- windows-setup.ps1 finds OBS on any drive and Vista, and leaves Wake-on-LAN
alone when it is already on (changing it can reset the adapter).
- Tests stub the fix executors so a hold in a test never reaches a machine.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Startup-folder shortcut on the Sanctuary TV PC is switched off in
Startup apps, so the app stayed closed after a restart. The StatusPanel
task now has a logon trigger (set live on that PC over SSH too), and a
re-run of the script keeps it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SSH to media@10.6.7.13 (Windows OpenSSH, cmd.exe): kill a hung Bible App,
then run the StatusPanel\Start BibleApp task, which starts it on the
logged-in desktop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' OpenSSH firewall rule covers Private networks only, so SSH timed out
on the Bible TV PC (Wi-Fi). Open it on any profile, but only to NA Ingest's
two addresses, and print each adapter's network category in the RESULT.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add-WindowsCapability can finish without registering the sshd service until
the PC restarts; the script stopped with a raw Set-Service error. It now prints
a RESULT block saying to restart and run it again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The hold-to-fix deck probe takes each HyperDeck's IP from its Companion
connection's "ip" variable, so the config no longer repeats them.
- APPS shows "VISTA SILENT" when the Lighting PC's Art-Net poll (every 0.5 s)
is not heard for 5 s: a crashed or closed Vista shows in seconds, not the
minute macOS takes to drop a network MIDI peer.
- tascam-quiet/ builds the Tascam DA-6400 module 1.3.1-quiet: the recorder's
"Illegal Status" warning is logged once a minute, not 5 times a second.
- README: hold-to-fix table fixed and updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every 10 minutes the watchdog maps each Dante device: name and IP from
Companion, MAC from NA Ingest's ARP table, switch port and PoE state from
the Netgear switches in dante_switches, saved to dante-map.json. A DANTE hold
on a missing PoE-powered device power-cycles the port it is on now (found
live by MAC, or its last known port if it has aged out and nothing else is
there); never a port with more than 2 devices. Re-patching, adding or
renaming devices needs no config change. First live map: 19 devices, the 4
Studio B boxes on PoE.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STUDIO-B-SOLO-MON, MON, TB and TB-MON map to ports 0/14-0/17 on the Netgear
M4250 Dante switch 10.66.1.102. netgear.py poe_cycle first checks the port's
MAC table and refuses if a different device is plugged in there, then runs
poe reset. Tested 2026-10-08 13:10 on SOLO-MON: dropped off Dante and was
back about 17 s later. install.sh now copies netgear.py too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Dante network runs on two Netgear M4250 AV switches (10.66.1.100
M4250-8G2XF-PoE+, 10.66.1.102 M4250-26G4XF-PoE+), managed by Engage from NA
Ingest. netgear.py logs in over their CLI with the login stored in NA
Ingest's login keychain (service netgear-status-panel) and, read-only, finds
which port a MAC is on and what PoE a port is delivering. The Studio B AVIOs
and talkback are on 10.66.1.102 ports 0/14-0/17.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dante Virtual Soundcard on the four Macs is restarted directly; for hardware
(consoles, racks, cards, AVIOs) nothing can be restarted from here, so a hold
re-checks via a fresh Dante module connection (harmless to audio and routing).
If the device is still gone it is really off or unplugged. A check's fix can
now be "restart-connection", and a fix map can use "*" for every other sub-check.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On 2026-10-08 a hold on HD1, whose Companion connection had been disabled,
sent the deck a reboot. Now a disabled connection on any button is fixed by
enabling it, before any other fix is considered. The HyperDeck fix asks the
deck itself (transport info on port 9993): if it answers - recording or not -
only Companion's connection is restarted; only a deck that accepts a
connection and says nothing at all is rebooted; an unreachable deck is left
alone and the button shows FIX FAILED. Tested against a fake deck.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
STUDIO-A-Ingest now maps to a local dvs_ingest fix (same helper as the other
Macs, which NA Ingest also needs). A hold on a failing button that has no
automatic fix now shows NO AUTO FIX instead of NOTHING TO FIX.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
remote-access/mac-dvs-helper.sh (run once per Mac through Remote Desktop as
root) installs an admin-owned script that does what pressing Start in DVS
does (wantToRun = 1, restart the DVS service) and a sudoers rule letting the
logged-in user run only that script. The dvs_ fixes call it with sudo -n, so
until it is installed the hold shows FIX FAILED rather than pretending.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A hold now turns the button blue with FIXING <what> while the fix runs, then
CHECKING for 20 s (green if it worked, back to the alarm if not), orange FIX
FAILED if the command failed, and grey NOTHING TO FIX or JUST TRIED / WAIT 1
MIN so a hold always visibly registers. The page redraws as soon as a command
arrives instead of on the next 1 s tick.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes come from config.json only and run over SSH with NA Ingest's
status-panel key (ProPresenter restart, Ableton / Church Translate start,
reopen DVS), locally (Resi bridge, captions, ProdDash) or as a HyperDeck
reboot. A green button does nothing, each fix waits 60 s before it can run
again, and a recording deck is never rebooted. New remote checks ask the Macs
every 10 s whether their apps are running. Group buttons and INGEST now have a
hold too; tests never run a real fix or open an SSH connection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It now targets whoever is logged in at the console instead of the user the
command runs as, and hands the .ssh folder back to them when run as root.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mac-authorize.sh is a one-liner for Apple Remote Desktop's Send UNIX Command:
it authorizes NA Ingest's status-panel key for the logged-in user and prints
the account name. windows-setup.ps1 turns on Windows' OpenSSH server,
authorizes the same key, creates desktop start tasks for OBS, Arena, rtpMIDI
and the Bible app, and enables Wake-on-LAN where the adapter allows it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ping failures now read NO PING, Companion connection errors OFFLINE, and a
watched connection that is disabled in Companion shows amber DISABLED instead
of going quietly grey.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UHD Jr was switched off and the NovaStar module still reported
Connected (even after a restart), so LED stayed green. LED now also checks
that each processor answers ping; the Resi decoder (send-only RossTalk) gets
the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The converted key's "startup" page resolves to the current page on surfaces
set to "use last page", so a tap did nothing. The tap page is now
nav_tap_page in config.json (1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A tap still goes to the start page, now on release so a hold does not flash
through page 1 first. Hold time is nav_hold_ms in config.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New config setting nav_keys (now ["pagenum"]) picks which nav keys are
coloured; nav_keys.py restores the built-in key for any kind not listed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nav_templates.json holds Companion's own UP / page number / DOWN keys after
'convert to regular button'; make_page.py colours them from the ALL tile and
nav_keys.py copies them onto all 96 other pages (288 keys, recorded in
nav_locations.json; --revert restores the built-in keys). Also rewrites the
README's top half, which an earlier edit had duplicated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It uploads the page file over Companion's own API (tRPC over WebSocket, the
same calls the web UI makes) and imports it into page 19, refusing unless that
page is empty or already the status page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DANTE now alarms for all 19 devices by name. MIDI covers the Lighting PC and
SuperRack peers plus the Lighting PC and NA Tracks sessions; APPS covers
ProPresenter, Ableton, the tracks Mac and the Bible TV app. Five buttons
became three, and a check's tag names the device in the message.
dns-sd pads hours before 10 AM with a space, which the Bonjour parser did not
allow for, so every session read as gone each morning. Parsing is now its
own function with a test for both cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
