christian/status-indicators
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: settings page plan, engineering details, audit record and inventory

Browse Source 
Plan for the settings page on NA Ingest (every setting editable, what runs
where, multi-campus), built with a plan-and-audit workflow and consolidated
against a decisions list; the audit's 157 findings and their dispositions;
the 544-item inventory of everything configurable today.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
christian
2026-10-13 15:05:36 -04:00
co-authored by Claude Fable 5.1
parent 38b6d1b014
commit 0ad6fb8451
4 changed files with 1504 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/settings-plan-details.md
+420
View File
@@ -0,0 +1,420 @@
# Status Panel settings page — engineering details (appendix to settings-plan.md)
This is the reference for building the settings page; the plain-words plan is `settings-plan.md`. Every item here follows the decisions list of 2026-10-13: it keeps K1-K18 and leaves out everything C1-C13 cut (no SIGHUP partial reload or apply classes, no queued or pending jobs, no admin code or address allow-list, no training copy, no vendor ignore lists or Add/Ignore bookkeeping, no page-set probe, no live-signal triples, no wording test over the rendered DOM, no `then` chain executor, no stop flag or commands folder, no surfaces table). Where the second full revision of the plan and the final revision's sections 10-14 differed, the final revision won. It keeps the ATEM saved-file handling, the page markers, the rollback, the identity-keyed state, the Windows echo pairs, `EXPECTED_MODULE` and overnight live-service windows.
Conventions used below:
- **Apply words** (what Apply does for a changed value): **restart** = the watchdog is restarted (Apply always restarts it, about 5 s of purple, outside services); **pages** = the Status or Restart page is rebuilt and re-imported because the built page JSON, with the keys' notes stripped, differs from the last verified build (never a hand-kept list; a changed note alone never forces a re-import, D11), and the page-number buttons are re-run after any page change; **none** = page-only data (notes, gear notes): no restart, no import; **settings** = the settings agent restarts.
- **Screens**: always Overview, Keys, Computers (Map, card, Gear filter, Setup tab), Logs, History, Help; then under a divider "Set up once": Site setup (Site . Schedules . Companion . Network & Dante . Spotify . ATEM) and Fixes; all eight are always visible and readable by anyone, and the editing code is what lets you change things (K3). The top bar holds the site name, the search box (finds any key, fix, computer, device or setting by name), the unsaved-changes badge, the Editing toggle and a "?".
- **Status words**: one legend for the whole page, printed once under the top bar's "?": OK / Problem (+ one-line reason) / Can't see from here - ask at the computer / Not needed / Done by hand (who, when). Every light anywhere on the page uses these words.
- **NEW** = a config key added by this plan (its default keeps today's behaviour). **DNE** = deliberately not editable (how the panel works, not what it watches; guideline 1), documented in Help > Glossary / Where things are.
- Paths: the data dir is `~/Library/Application Support/status-panel/`; the watchdog agent is `org.waterschurch.status-panel`, the settings agent `org.waterschurch.status-settings`.
## A. Everything you can change
Values are typed once: every string that names a program, path, task or helper lives in one owner (`hosts.<alias>.runs`, `site.*`, `spotify.name`) and checks, fixes and scripts are regenerated from it. One owner screen per value; every other place shows it read-only with a link.
| Thing | Screen | Field(s) | Plain-words help | Validation / apply |
|---|---|---|---|---|
| `site.id` NEW | Site setup > Site (Advanced) | Site id | Short name for folders and files ("na"). | `[a-z0-9_-]+`; restart |
| `site.name`, `site.ingest_name`, `site.ingest_hostname`, `site.ingest_ips[]` NEW | Site setup > Site | Site name, this Mac's name, network name, addresses | Used in log headers, alert text, key notes and generated setup scripts (replaces "NA Ingest" and the IPs in windows-setup.ps1). [Check this Mac's network ports] fills the addresses. | non-empty; IPs valid; restart (notes: pages) |
| `site.settings_port` NEW | Site setup > Site (Advanced) | Port | Where this page listens: loopback plus every private-address port, narrowed by this Mac's LAN list. Default 8020. | 1024-65535; the dialog states the consequence (the page moves to the new address); settings |
| editing code (keychain item `status-settings-edit-key`, 0600 file fallback `edit-key` in the data dir) NEW | Site setup > Site (Advanced) | "editing code: set <date>", [Make a new editing code] | Random word-list code the settings agent makes on its first start in the logged-in session; shown only on this Mac's screen at `http://127.0.0.1:8020/edit-key`; browsers remember it with the person's first name. The one code: no admin code, no address list. | never shown over the LAN; the dialog says every browser must enter the new code; settings |
| `site.live_services[]` NEW | Site setup > Schedules | Live services | When a service is on: Apply, Run now, Restart the watchdog, Re-import, Check the switcher and Install are refused and your draft stays saved; auto-fix may run; alarms are always on. Defaults to `auto_fix.windows`; the Phase 0 migration writes it. | days in mon..sun; `HH:MM`; end <= start = runs past midnight (allowed); warn if empty; restart (ServiceMode is built at start) |
| built-in live signal: a watched HyperDeck that "follows the recording" is recording | Site setup > Schedules ("Also counts as live", read-only line with "Live now?") | - | Derived from every hyperdeck check with "follows the recording"; the watchdog writes `live-now.json` on change / every 5 s with a timestamp; older than 30 s = schedule only (said in Review). No connection/variable/value triple on the page; "OBS is streaming" is added as a built-in kind only if the Phase 1 spike finds the module variable. | DNE; warn when no watched deck follows the recording |
| `site.contacts` NEW | Site setup > Site | Contacts | Printed on the cheat sheet. | free text; none |
| `site.dvs_helper`, `site.solo_dir` NEW | Site setup > Site (Advanced) | DVS helper path, Studio B helper folder | One owner for paths that fixes, checks and scripts repeat; builders and templates regenerate from them. | absolute path; restart |
| `alert_webhook`, `site.alert_prefix` NEW, `timing.alert_after_seconds` NEW | Site setup > Site / Schedules (Advanced) | Alert message (web hook), message starts with, alert after | Posts "<prefix>: Companion has not answered for N s" and the all-clear; also a line for every command edit applied. Blank = off. Shown masked everywhere ("set, 44 chars"); [Change] [Send a test alert]. | http(s) or blank; 5-600; restart (captured in run()) |
| `ssh_key` | Site setup > Site (Advanced) | Login key | Private key on this Mac for every other computer; light + public half with [Copy]. [Make the login key] on the Setup tab makes the panel's own pair at a new campus (asks first). | file exists, 0600; `.pub` readable; restart |
| `companion.host/port/db` | Site setup > Companion (Advanced) | Address, port, settings database | Where Companion answers. The DB is read read-only for labels and addresses (module version only if the Phase 1 spike finds it trivially); blank = newest `vN.N/db.sqlite`. push_page and nav_keys read these too. | host non-empty; 1-65535; path exists; live probe; restart |
| `variables.state/heartbeat/command`, `variables.flash` NEW | Site setup > Companion (Advanced) | Variable names (state `sp_state`, heartbeat `sp_hb`, command `sp_cmd`, blink `flash_tick`) | Must exist in Companion: a hard error when Companion answers, a warning ("could not check") when it does not. Renaming the blink variable means renaming it inside the trigger in Companion too (said beside the field). | `[A-Za-z0-9_]+`; exists; pages + restart |
| `page`, `page_name`; `restart_page.page`, `restart_page.page_name` | Keys tab header (only home; the Companion section links here) | Page numbers and names | Name derived "<n>\nStatus" / "<n>\nRestart" unless overridden. Number change: the old page keeps its keys; Apply resets our marked keys on it (asks; if the spike shows `controls.resetControl` cannot blank a cell, Review lists "delete these N keys on page 19 by hand") and re-runs the page-number buttons. Name-only change on an existing page: rename it in Companion first (live name shown). "Pages exist" light. | int >= 1; distinct; name contains the number; live checks; pages |
| `nav_keys[]`, `nav_hold_ms`, `nav_tap_page` | Site setup > Companion (Advanced) | Page-number buttons that follow ALL, hold ms, tap page | Which of up / page number / down are coloured; a tap goes to page N; a hold jumps to the Status page. "Use last page" surface note shown. | subset of up/pagenum/down; 200-5000; page exists; pages |
| `page_hold_ms` NEW (today's HOLD_MS) | Site setup > Schedules (Advanced, Timing) | Holding a key takes | Guideline: 2 s. The watchdog reads it for the "HOLD 2s" text. | 500-5000; pages + restart |
| `page_grid` NEW (GRID) | Site setup > Companion (Advanced) | Grid | Export grid for the surface (XL 8x4 inside 0-8 x 0-5); the Keys screen draws the 8x4 by default, "Show more" shows the whole grid. | ints; every position inside; pages |
| `page_stale_seconds` NEW (STALE_SECONDS) | Site setup > Companion (Advanced) | Purple after | Keys turn purple when the watchdog is silent this long. | 5-120; pages |
| `companion.export_version`, `companion.build` NEW | Site setup > Companion (Advanced) | Export stamps | What make_page writes into page files and which nav template it picks (`companion-objects/nav_templates/<version>.json`); warning when Companion's live version differs; "Recapture the nav templates" recipe linked. Change only after a Companion upgrade. | int; string; pages |
| `service_windows[]` + window shape | not on the page (Help: deliberately not editable) | Alarms are armed | Always every day all day (guideline 1). A narrowed value in an old file loads with a Review warning. | days in mon..sun; `HH:MM`; restart |
| `auto_fix.windows[]` | derived from live services (not shown) | - | Same windows as the live services; "May run by itself on Sundays 06:30-13:30" is built from them. | derived; restart |
| `auto_fix` on/off, `after_seconds`, `max_tries`, `retry_seconds`, `pause_after_network_seconds`, `forget_seconds`, `restart_connections`, `fixes[]`, `log_dir`, `keep_months` NEW | Site setup > Schedules | Auto-fix block: tick list and "also restart Companion connections by itself" on the screen; numbers, folder and months under Advanced with defaults beside them | Gentle fixes only; a fix that closes a program needs "may close"; a "never by itself" fix is refused. | ints; fix ids exist; load_config gentle rule; folder creatable; 1-12; restart (the Engine builds its auto state at start) |
| `debounce_seconds`, `flap.count`, `flap.window_seconds` | Site setup > Schedules (Advanced, Timing) | A fault must last; flapping | One bad reading does not alarm; repeated faults show amber xN. Defaults 3 s, 3 in 600 s. | 0-60; ints; restart |
| `timing.fix_cooldown_seconds` NEW (60) | Site setup > Schedules (Advanced, Timing) | The same fix never re-runs within | Guards against hammering a device; the watchdog reads it as one `self.cooldown` for the "WAIT 1 MIN" text. | 10-600; restart |
| `network.interface/gateway/burst_connections/burst_groups/burst_window_seconds/remember_seconds` | Site setup > Network & Dante | NET key: port and gateway on the screen; blip rule under Advanced | Port from a live list; gateway blank = off; a "blip" is N connections from M groups dropping within W s; NET shows BLIP for remember_seconds. | exists in ifconfig; IP or blank; ints >= 1; restart (pollers built at start; NET note: pages) |
| `dante_switches[]`, `dante_map_seconds`, `dante_max_shared` NEW (MAX_SHARED) | Site setup > Network & Dante | Switches, survey every, never power-cycle a port carrying more than | Netgear M4250 only; empty = no map, no PoE. | IPs with telnet light; 60-86400; 1-10; restart |
| keychain `netgear-status-panel` | Site setup > Network & Dante | light + account | Present/absent and user name only; the Terminal line to store it is shown ([How to add it]). | never read or written by the page |
| `hosts.<alias>` | Computers (card) | Short name used inside the settings | Shown below the display name. Delete shows what refers to it and removes those too (cascade, listed). | `[a-z0-9_]+` unique; rename updates references; restart |
| `hosts.<alias>.ssh/mac/wake_broadcast` | Computers (card) | Log in as, wake MAC, broadcast | How this Mac logs in; wake a PC that is off (key says WAKING PC). The MIDI stream and the batched checks for that computer restart with the new address. | `user@host`; MAC 6 bytes; IP; reach light; restart |
| `hosts.<alias>.name/os/role/notes/pingable/self` NEW | Computers (card) | Display name, kind, what it does for the panel, notes for operators, does not answer pings, this Mac | For people, logs, scripts and probes; `pingable:false` skips the ping light; exactly one `self` record (no ssh), refused as a host for fixes, remote checks and MIDI links (builders switch to local for it). | os in mac/windows/other; one self; none |
| `hosts.<alias>.runs[]` NEW | Computers (card: "What runs here") | What runs here | One entry per thing, with kind `sshd`, `task`, `service`, `launchagent`, `daemon`, `app`, `web`, `helper_dvs`, `helper_solo`, `midi_session`, `artnet`, `disabled` or `other`; fields per kind: name, exe, candidates, args, at_logon, label, path, http, source, dante_name, folder, restart_on_failure, `watched` (false = listed, no light); `helper_solo` also carries `pt_bundle_id` and `solo_menu_path` (defaults = today's Pro Tools values). Feeds probes, scripts, check and fix builders: a program is named once here. | per-kind required fields; task names `[A-Za-z0-9 ]+`; Windows values cmd-safe `^[A-Za-z0-9 ._()\\:*-]+$` (no `!` or `^` either); `PROBE_SAFE` whitelist of fields that may reach a probe line; none (dependent checks/fixes regenerate -> restart) |
| `hosts.<alias>.networks[]` NEW | Computers (Ingest card, Advanced) | Ports | iface + use (LAN / Dante / switch management), filled by [Check this Mac's network ports]; the page binds 127.0.0.1 plus the IPv4 address of each LAN-use port (resolved from ifconfig at start, re-checked every minute, rebound on change); an empty list = every private-address port and a grey notice. | iface exists (light); settings (for the `self` record) |
| `hosts.<alias>.manual_done{}` NEW | Computers > Setup | Done-by-hand ticks | BIOS WoL, Public profile, Vista autoload, Wi-Fi power settings: no probe can see them, you tick them and the light says "Done by hand (Chris, 2026-10-08)". | bool + who/when; none |
| `gear.<id>.location/notes` NEW | Computers > Gear filter | Where it is, notes | Plain words for people; the device itself is defined by the check that watches it (the Gear filter is a view of the Computers list, not its own data model). | free text; none |
| `fixes.<id>` | Fixes (Advanced) | Id | Referenced by keys, restart keys and auto-fix; delete cascades with a list. | `[A-Za-z0-9_]+`; not restart-connection / dante-auto; restart |
| `fixes.<id>.kind` | Fixes (Built as) | Built as | ssh (default), local, atem, reconnect, midi_connect, netgear_poe, hyperdeck, `spotify` NEW; `FIX_KINDS` exported and validated by load_config. | enum; restart |
| `fixes.<id>.host` | Fixes | Computer | ssh kind; validated for every fix (restart keys, hold-always, MIDI links too): exists, has ssh, not the `self` record. | alias rules; restart |
| `fixes.<id>.run`, `.template`, `.params` NEW | Fixes (run under Advanced) | Generated / Custom command | zsh on Macs, cmd.exe on PCs. The builder stores template + params beside the generated `run`; the watchdog runs only `run`; every Mac value is `shlex.quote`d, every Windows value regex-checked; the dvs_ingest -> prodcom step is rendered inline by the builder (no chain executor). A raw Custom command is a "command edit": red in Review, asks for the editing code again, never run before Apply. | non-empty; `zsh -n` for local and Mac ssh commands; pkill/taskkill/killall => not gentle unless "may close"; restart |
| `fixes.<id>.timeout` | Fixes | Give up after | Default 60 s. The largest timeout sizes the watchdog's ExitTimeOut at install. | 5-600; restart |
| `fixes.<id>.short`, `.label` | Fixes | Key text, plain words | "FIXING <short>"; plain words required for every fix; the Play builder derives both from `spotify.name`. | <= 14; required; restart (a changed note alone never forces a re-import, D11) |
| `fixes.<id>.reconnect`, `.conn` | Fixes | Reconnect Companion connection after | conn required for kind reconnect; picked from live labels. | warning if not live; restart |
| `fixes.<id>.restart_ok` | Fixes | May close a running program | Lets a closing fix be gentle (atem_midi, prodcom, proddash). Removes "Run now". | bool; restart |
| `fixes.<id>.never_auto` NEW | Fixes | Never runs by itself | Generic flag (bridge: never during a service). Removes "Run now". | bool; refused in auto_fix.fixes; restart |
| `fixes.<id>.part` | Fixes | Switcher part | audio or camera. | enum; restart |
| `fixes.<id>.verify_seconds/off_seconds/hung_note` | Fixes (More settings) | Wait before judging, off time, still-bad text | reconnect / hyperdeck kinds. | 5-120; 1-10; <= 2 lines; restart |
| `fixes.<id>.switch/port/mac` | Fixes | PoE switch, port, MAC guard | netgear_poe kind. Prefer Dante auto. No Run now. | IP in switches; `\d+/\d+`; MAC; restart |
| `spotify.open_wait`, `spotify.play_wait` NEW (read by the `spotify` fix kind) | Site setup > Spotify | Play fix: open Spotify and wait N s, then wait M s before checking it plays | The fix reads playlist and name from `spotify.*` (one owner); the same for the restart key, which has no check. | 1-60; restart |
| built-ins `restart-connection`, `dante-auto` | key editor picker | "Restart its Companion connection", "Dante auto" | Always offered where they apply. No Run now. | - |
| per-sub fix map `{sub: fix, "*": fix}` | key editor (Advanced) | table per check | Sub names per type (device names live; conn/disk/rec; chN_batt/chN_rf; chN_mute; conn/error/caution). | values are fix ids or built-ins; restart |
| implicit "enable a disabled connection" | Help; HOLDING 2 s WILL line | documented | Any held key whose connection is off in Companion turns it on. | DNE |
| `tiles[].id` | Keys (Advanced) | Id | Used inside Companion; master is `all`. | `[A-Za-z0-9_]+`; unique vs restart keys; pages |
| `tiles[].label` | Keys | Label | First line and prefix of faults. | <= 10 warn; pages |
| `tiles[].pos` | Keys (drag) | Position / tray | Blank = in the tray "Shown inside the RF group (no key of their own)". | in grid; not col 0 rows 0-2; no overlap incl. jump/back pos; pages |
| `tiles[].kind` | Keys | Kind | single key / group; ALL and ALARMS are fixed chips (moved and edited, never created or deleted); NET may be removed at a campus that does not want it (at most one; D13). | exactly one ALL, one ALARMS, at most one NET; pages |
| `tiles[].members[]` | Keys | Group members | Worst member shown ("worst member + N"); hold fixes all. | ids exist; not self; a hidden key must be in a group (warn); restart (pages when hold-ability changes, detected by the build) |
| `tiles[].checks[]` | Keys | Checks | See per type. Changing a key's check list drops that key's remembered acknowledgements and auto-fix tries (said in Review). | type in CHECKS; restart (pages when the build differs) |
| `tiles[].hold_always` | Keys > When held | also when the key is green | Even when green (HD, TASCAM, SPOTIFY). | fix exists with label; pages |
| check `type` | "+ Add a check" | question first | The menu asks a question (Is a device answering? Is a program running? Is a MIDI link up? Is a recorder OK? Is a setting right? Is Spotify playing? Is Dante complete? Is the lighting console sending?) and picks the type from the answer; the 19 raw types (conn, hyperdeck, dante, shure, psm, tascam, ping, launchd, process, http, disk, midi_peer, pref, spotify, mdns, midi_rx, remote, artnet, atem) only under Advanced. | enum |
| check `conn` | check row | Companion connection | Live dropdown filtered by `EXPECTED_MODULE` for the type; a mismatch warns in Review. | required for conn/hyperdeck/dante/shure/psm/tascam; restart |
| check `fix`, `tag`, `service_only`, `debounce_seconds` | check row (When held / More settings) | Fix, tag, only during a service (Advanced; warned: guideline 1; no effect while alarms are armed all day), wait | | exists; <= 12; bool; 0-60; restart |
| check `template`, `params` NEW (remote, process, pref) | check row | Built as (A program is running on a Mac / on a Windows PC / A helper file is fresh or says X / A program is running on this Mac / A macOS preference equals / Custom) | Regenerates `cmd` / `match` / `domain,key` from what runs on the computer and `site.solo_dir`, so renaming a program in Computers updates the checks that watch it. | regenerated on save; test_builders_regenerate; restart |
| conn `ignore_warning` | check row | Ignore Companion's yellow warning | ProPresenter flicker. | bool; restart |
| hyperdeck `conn/optional/disk_service_only/full_minutes/low_minutes/rec_sync/ip` | check row | Connection, this deck may have no disk, NO DISK only in service, FULL below, LOW below, follows the recording, address override | Which HyperDecks are in use = which HD keys exist (the Add or remove a HyperDeck job also asks "may this deck have no disk?" per kept deck); "follows the recording" decks feed the live-service guard. Expects bmd-hyperdeck. | ints; IP; restart |
| dante `conn/devices{}/count_all` | Site setup > Network & Dante (only home; the check row links there) | Connection, device list, every device counts | Names from the live devices variable; key texts <= 14; the first dante check feeds the map. Expects audinate-dantecontroller. | non-empty; restart |
| shure `conn/channels/battery_err_bars/battery_warn_bars` | check row | Receiver, channels, red at <=, amber at <= | | 1-8; 0-5; err <= warn; restart |
| psm `conn/channels/muted_is_error` | check row | Receiver, channels, a muted channel is an error | | 1-8; bool; restart |
| tascam `conn` | check row | Connection | Needs the quiet module (light on the Companion section). | restart |
| ping `host/conn/name/msg/failures` | check row | Address or Companion connection, name, text, misses | With conn set, Companion's address wins. | host or conn required (fixes today's CHECK BUG); 1-10; restart |
| launchd `label/name` | check row | Helper (picker), name | | restart |
| process `match/name` (+ template/params) | check row | Program (picker from what runs on this Mac), name | | regex compiles; restart |
| http `url/msg/name/interval_seconds` | check row | Address, text, name, every N s | Any answer counts (so it is never a live signal). | URL; >= 5; restart |
| disk `path/warn_gb/err_gb` | check row | Folder, amber below, red below | | exists; err < warn; restart |
| midi_peer `session/peer/host/address` | check row / Add a MIDI link job | Session, peer (live pickers, local or over ssh, or a typed name when the Mac is off), Mac, fallback address | Macs only as host. The identity (host, session, peer) keys acknowledgements and Run now. | alias exists, is a Mac, has ssh, not self; restart |
| pref `domain/key/want/msg` (+ template/params) | check row | Domain and key (picker from `defaults domains`), wanted value, text | live `defaults read` shown | all required; restart |
| spotify `playlist/name` -> `spotify.playlist`, `spotify.name` NEW (one owner) | Site setup > Spotify (only home; the SPOTIFY key's check row links) | one link field + name | The page derives the URI and fetches the name and song count from open.spotify.com only; `spotify.name` owns the display name used by the fix label, the key guide and the notes; one permission light from the watchdog's last successful Spotify read in its log. | playlist URI regex; song-list fetch warning (private playlist); restart |
| mdns `instance/name` | check row | Bonjour MIDI name (picker) | RTP-MIDI only; the Bonjour type is fixed. | restart |
| midi_rx `source/name/max_silence_seconds` | check row | Source (picker), name, amber after | | restart |
| remote `host/cmd/msg/shell` NEW (+ template/params) | check row | Computer, built as, text | `shell` NEW: `sh` (Macs, batched with `; echo SPn=$?`) or `cmd` (Windows PCs, batched as `(cmd) >nul 2>&1 && echo SPn=0 \|\| echo SPn=1` pairs joined with `&`, never a bare `%ERRORLEVEL%`; verified once on the Lighting PC); `cmd` is generated; Custom only under Advanced with "cannot be checked before Apply"; a PC whose OpenSSH DefaultShell is not cmd.exe gets a light and cmd-batched checks are refused; a changed remote check is marked "not run before Apply" in Review. | alias exists, has ssh, not self; `sh -n` on each command and the batched line (Mac); cmd-safe regex (Windows); restart |
| artnet `source/max_silence_seconds/msg` | check row | Sender, silence, text | Port 6454 is fixed. | IP; 1-60; restart |
| atem check `part/msg` | check row | Part, text | Needs the ATEM section. | restart |
| `atem.host/node/library/reference`, `atem.compare` NEW, `atem.source_folder` NEW | Site setup > ATEM | Switcher address, patterns (resolved paths shown), saved file (upload or pick the newest from the folder), folder, what to keep in line | The upload (capped at 20 MB; XML with a DOCTYPE or entity declarations refused; parsed in the settings process with the watchdog's own parser) replaces `atem-reference.xml` atomically together with the config; its hash goes into History; the old file is kept dated under `history/atem/`; refused during a live service; Review shows "product name matches / N audio and M camera values differ from the current file" and needs the tick "I checked this is the right file for this switcher"; [Check the switcher now] reads only (3 s, a second connection, guarded). | IP; globs resolve; XML parses with the watchdog's own parser, no DOCTYPE or entity declarations, <= 20 MB; compare is a subset of MIX_OPTIONS/CCU_PARAMS; folder exists (light); restart |
| "this campus has no ATEM" switch (the `atem` section absent) | Site setup > ATEM | tick box | Hides the section; an atem check or a saved-switcher-file fix then fails validation with the key named. | cascade listed; restart |
| `restart_page.jump_pos/back_pos`, `restart_page.keys[].id/label/pos/fix/watch[]` | Keys (Restart tab) | jump/back chips; restart keys: label, position, fix, "keep flashing HOLD TO FIX while these are down" | watch = fix ids and "conn:<label>". | id unique vs tiles; fix exists; labels live-checked; pages (fix/watch: restart) |
| "this campus has no Restart page" switch (the `restart_page` section absent) | Keys (Restart tab) | tick box | make_page builds no Restart page and no jump key; the restart keys and the jump chip go with it (cascade listed). | cascade listed; pages |
| Companion connection labels | Site setup > Companion | live list with module id, status, watched by; rename-to for a vanished label; [Add a key for it] on unwatched ones | | warning only |
| custom variables / blink trigger / modules / pages exist / our pages / page-number buttons | Site setup > Companion | lights; "coloured on N of M pages" (counted when the section is opened) + [Colour them now] + [Undo]; [Re-import the pages now] | Set up once by the person who installed Companion. | by hand except nav revert and re-import |
| `companion-objects/` (trigger export, quiet Tascam module, nav templates per version) NEW | Help > Set up a new campus | shown with the import steps | Captured from NA once (Phase 1); not editable on the page. | repo files, deployed to the data dir |
| Windows program table (`runs[].kind=task`) | Computers (card) | task name, exe candidates, args, at logon | Generates windows-setup.ps1, task names in fixes and the "is running on a Windows PC" check. | cmd-safe regex; none (dependents -> restart) |
| `$Key`, `$IngestIPs` in scripts | Computers > Setup | from `ssh_key.pub` and `site.ingest_ips` | | - |
| `{{PT_BUNDLE_ID}}`, `{{SOLO_MENU_PATH}}` in the solo agent | Computers (Studio B card, Advanced) | Pro Tools bundle id, solo menu path | Differ per Pro Tools version/language; defaults are today's values. | non-empty; regenerating shows the agent needs a rebuild + Accessibility re-grant |
| python version on this Mac | Computers (Ingest card) | light | Must be 3.9-3.12: telnetlib (Netgear) is gone in 3.13. | self-check at start |
| `state.json` (acknowledged keys, auto-fix tries, fix cooldowns) NEW | Review note; Overview ALARMS line | shown, not edited | Keyed by (key, check identity): midi_peer (host, session, peer) / remote (host, cmd) / ping (host or conn) / http url / mdns instance / launchd label / process match / artnet source / else conn. Acks survive a restart and are dropped only for a key whose checks changed; one atomic writer (ServiceMode merges into it); `service-mode.json` migrated once. | the watchdog's file; the page never writes it |
| `started.json`, `live-now.json`, `fixes-running.json` NEW | Overview (watchdog line, live-service line) | shown | Written by the watchdog: `{pid, started_at, config_sha}` at start; live-now on change / every 5 s with a timestamp; the running-fix registry. Apply reads them for its success test and its waits. | the watchdog's files; the page never writes them |
| `netgear.py` CLI SWITCHES; push_page/nav_keys `--host` defaults | - | read config | | code change (Phase 0) |
| COLORS, ORDER, fault words, hold-feedback texts, poller intervals, HOLD_MAX_AGE, freshness limits, text fitting, ssh options, ports (WoL 9, 6454, 9993, 23), OS tool paths, Companion API paths, command grammar, RESTARTABLE, NAV_LOCATIONS, button style, notes markers, nav templates, ATEM parser, Spotify endpoints, Netgear driver, coremidi constants | Help > Glossary / Where things are | shown, not editable | Guideline 1 wants identical keys everywhere; these define how the panel works, not what it watches. | DNE |
## B. Computers: data model, probes, setup checklist
**The view.** The Map comes first: one row per computer, problems sorted to the top, one status word at the left from the one legend (the worst of its items: OK / Problem with its one-line reason / Can't see from here), what it does for the panel as up to three plain names then "+N", the keys that turn red if it dies, and the only per-item status shown is a problem ("Problem: rtpMIDI tray stopped"). Clicking a row opens the card: display name, its status word from the legend, "What runs here" with a light per listed item, "Used by" (keys, fixes, restart keys as links), notes for operators, and a collapsed raw **"Everything found on this computer"** list straight from the probe (every agent, task, service, login item and listening port the probe saw, shown as it came back). There is no Add/Ignore bookkeeping, no vendor list and nothing from that list is counted on Overview. The **Gear** filter shows the devices that are not computers (address, kind, where it is, watched by, light; notes typed once). The **Setup** tab holds the one-time checklist. The Overview shows one line per computer with one status word.
**The data.** `hosts` grows from `{ssh, mac, wake_broadcast}` to a computer record; the Ingest Mac is a record too (`self: true`, no ssh). The watchdog keeps using only `ssh/mac/wake_broadcast`; the rest is for the page, probes and generators. Phase 1 seeds all 8 NA computers from today's inventory. Record fields: `name`, `os`, `ssh`, `mac`, `wake_broadcast`, `role`, `notes`, `pingable`, `self`, `runs[]`, `networks[]`, `manual_done{}`. `runs[]` kinds: `sshd`, `task`, `service`, `launchagent`, `daemon`, `app`, `web`, `helper_dvs`, `helper_solo`, `midi_session`, `artnet`, `disabled`, `other` (plus `watched: false` for things that are listed but get no light).
```json
"ingest": {"self": true, "name": "NA Ingest", "os": "mac", "role": "watchdog, Companion, captions, transcription",
"runs": [
{"kind": "launchagent", "label": "org.waterschurch.status-panel", "name": "watchdog"},
{"kind": "launchagent", "label": "org.waterschurch.status-settings", "name": "Settings page"},
{"kind": "app", "path": "/Applications/Companion 5.0.7.app", "name": "Companion", "http": "http://127.0.0.1:8000/api/connections"},
{"kind": "app", "path": "~/Applications/Companion Satellite.app", "name": "Companion Satellite", "watched": false},
{"kind": "launchagent", "label": "org.waterschurch.resi-propresenter", "name": "Resi bridge"},
{"kind": "launchagent", "label": "org.waterschurch.lobby-captions", "name": "lobby captions"},
{"kind": "launchagent", "label": "org.waterschurch.lobby-captions-display", "name": "caption screen"},
{"kind": "app", "path": "/Applications/ProdCom.app", "name": "ProdCom", "http": "http://127.0.0.1:24480/api/v1/ping"},
{"kind": "app", "path": "/Applications/ProdDash.app", "name": "ProdDash", "http": "http://127.0.0.1:24500/"},
{"kind": "app", "path": "~/Applications/Spotify.app", "name": "Spotify"},
{"kind": "app", "path": "/Applications/Blackmagic ATEM Switchers/ATEM Software Control.app", "name": "ATEM Software Control"},
{"kind": "daemon", "label": "com.audinate.dante.DanteVirtualSoundcard", "name": "Dante Virtual Soundcard", "dante_name": "STUDIO-A-Ingest"},
{"kind": "helper_dvs"},
{"kind": "app", "path": "~/Library/CloudStorage/GoogleDrive-media@waterschurch.org", "name": "Google Drive (saved switcher files)", "folder": true},
{"kind": "launchagent", "label": "org.waterschurch.checklist-build-runner", "name": "Gitea runner (not the panel)", "watched": false},
{"kind": "launchagent", "label": "org.waterschurch.checklist-drive-backup", "name": "checklist backup (not the panel)", "watched": false},
{"kind": "launchagent", "label": "org.waterschurch.checklist-weekly-health", "name": "checklist health (not the panel)", "watched": false},
{"kind": "other", "name": "login items: Itsycal, Alfred 5, Google Drive, Signiant", "watched": false},
{"kind": "disabled", "label": "org.waterschurch.translate", "name": "Church Translate (must NOT run here)"}
],
"networks": [{"iface": "en0", "use": "LAN"}, {"iface": "en17", "use": "Dante"}, {"iface": "en20", "use": "Netgear management"}]},
"studiob": {"name": "Studio B Mac Studio", "os": "mac", "ssh": "studiob@10.1.13.175", "role": "Pro Tools",
"runs": [{"kind": "sshd"},
{"kind": "helper_solo", "pt_bundle_id": "com.avid.ProTools", "solo_menu_path": ["Options", "Solo Mode", "AFL"]},
{"kind": "app", "path": "/Applications/Pro Tools.app", "name": "Pro Tools"},
{"kind": "midi_session", "name": "Studio B Mac Studio (peer of NA Tracks)"}]},
"lighting": {"name": "Lighting PC", "os": "windows", "ssh": "media@10.1.10.206", "role": "Vista, Resolume Arena, rtpMIDI",
"pingable": false, "notes": "Keep the network profile Public (rtpMIDI Bonjour rules).",
"manual_done": {"public_profile": {"by": "Chris", "on": "2026-10-08"}, "vista_autoload": {"by": "Chris", "on": "2026-10-08"}},
"runs": [{"kind": "sshd"},
{"kind": "task", "name": "Start Vista", "exe": "Vista.exe", "candidates": ["*:\\Vista3\\Vista.exe"]},
{"kind": "task", "name": "Start Arena", "exe": "Arena.exe", "candidates": ["C:\\Program Files\\Resolume Arena*\\Arena.exe"]},
{"kind": "task", "name": "Start rtpMIDI", "exe": "rtpMIDI.exe", "candidates": ["C:\\Program Files (x86)\\Tobias Erichsen\\rtpMIDI\\rtpMIDI.exe"]},
{"kind": "service", "name": "rtpMIDIService", "restart_on_failure": true},
{"kind": "artnet", "name": "Vista Art-Net", "source": "10.1.10.206"}]}
```
The other NA records: **prop** (sshd, DVS daemon NA-ProPresenter-Mac, helper_dvs, app ProPresenter = newest `/Applications/ProPresenter*.app` with http :1025, midi_session "NA ProP"); **tracks** (sshd, DVS NA-Tracks, helper_dvs, app `Ableton Live 11 Suite` - the "Reopen Ableton with its last set" builder and the "A program is running on a Mac" check read the app name from here, so it is typed once; midi_session "NA Tracks" + Bonjour advert); **translation** (sshd, DVS NA-Translation, helper_dvs, app Church Translate); **livestream** (sshd, task Start OBS exe obs64.exe `--disable-shutdown-check` elevated-detected, WoL mac/broadcast, `manual_done.bios_wol`); **bibletv** (sshd, task Start BibleApp at_logon, web http://10.6.7.13:8080/, `pingable:false`, `manual_done.wifi_power`, note "Wi-Fi only; Ethernet recommended").
**Devices that are not computers** (ATEM, HyperDecks, Tascam, X32, Shure, LED, Resi boxes, SuperRack, Netgear switches, Studio B PoE boxes) have one home: the Gear filter of the Computers list, derived from the checks (address, kind, ping/connection light, keys that watch it) plus `gear.<id>.location/notes` typed by a person; the key editors, the Dante map, the ATEM section and the Help card "Which computer or device is that?" link to it.
**Liveness** (`settings/probes.py`): read-only; it only ever runs the fixed command set in probes.py, never a string from the settings file; the only parameters (`<app>`, `<helper>`, `<solo_dir>`, `<ip>`, the listed task and service names) come from `runs` / `hosts` through the schema regexes and `shlex.quote`, and `PROBE_SAFE` names which `runs` fields may reach a probe line at all. Probes never run during a fix (they wait while `fixes-running.json` is non-empty), never wake a PC, time out after 6 s per command, run in parallel threads, and cache their results with timestamps in `machines-live.json` so the page shows "last look 2 min ago" instead of waiting. They run when Computers or Overview is open (every 60 s) and on [Check computers now] / [Check this computer now]. Per computer: ping (skipped and marked "does not answer pings" when `pingable:false`), `ssh ... true` with the panel key (same `ssh_cmd` options and ControlMaster socket, so it is cheap), then one batched command per computer, by OS, that returns the full inventory. Each listed `runs` item is matched against that inventory for its light; everything else is shown raw under the collapsed "Everything found on this computer" list.
- Mac: `launchctl list` (every agent label with pid/state), `ls ~/Library/LaunchAgents /Library/LaunchAgents /Library/LaunchDaemons` (every plist), login items via `osascript -e 'tell application "System Events" to get the name of every login item'`, `pgrep -x <app>` per listed app, `ls -l /usr/local/sbin/status-panel-dvs-start /etc/sudoers.d/status-panel` + `sudo -n -l <helper>` (helper_dvs), `cat <solo_dir>/solo-mode.json` age and `trusted` (helper_solo), `pgrep -f DanteVirtualSoundcard/apec` + `defaults read /Library/Preferences/com.audinate.dante.dvs wantToRun` (daemon), `grep -c status-panel ~/.ssh/authorized_keys` (login key), `/usr/bin/python3 - < coremidi.py` sessions list (MIDI sessions and peers shown on the card, expected beside live), `ssh-keygen -F <ip>` on this Mac (host key known; "host key changed since last time" is detected from the ssh error text and offers the confirmed [Forget the old login key] = `ssh-keygen -R <ip>`).
- Windows (cmd.exe; the probes use `schtasks /Query /FO CSV` and PowerShell `Get-CimInstance`, never wmic, which is gone on Windows 11 24H2): `schtasks /Query /FO CSV` filtered to `\StatusPanel\` (every task, not only the listed ones), `tasklist /FO CSV`, `sc query type= service state= all` + `powershell -c "Get-CimInstance Win32_Service | ? StartMode -eq Auto | select Name,PathName"` filtered to non-Microsoft paths (wmic is gone on Windows 11 24H2), `powershell -c Get-NetAdapterPowerManagement` (WoL), `Get-NetFirewallRule -Name OpenSSH-Server-In-TCP | Get-NetFirewallAddressFilter` (firewall allow-list vs `site.ingest_ips`), `Get-NetConnectionProfile` (Public profile light), and the DefaultShell probe `echo %COMPUTERNAME%` (expanded = cmd.exe is the OpenSSH DefaultShell; echoed literally = PowerShell: the card shows the light and cmd-batched checks for that PC are refused until the generated script has set it).
- This Mac: everything in the Mac list plus `lsof -nP -iTCP -sTCP:LISTEN` (bound ports), http pings of the listed `http` entries, `ifconfig` per network in `networks`, keychain item attributes only (never the secret), the watchdog's Spotify permission from its log (last successful read; never an osascript from the settings process), the login key pair present, `/usr/bin/python3 --version` in 3.9-3.12, node/library glob resolution for ATEM, `pgrep -lf atem_watch.js`, `lsof -nP -iUDP:6454` (Art-Net port bound), `pgrep -lf ServerAliveInterval=2` (MIDI streams per host), `dns-sd -B _apple-midi._udp` (3 s), `atem.source_folder` mounted, Remote Login on (`systemsetup -getremotelogin`, falling back to `launchctl print system/com.openssh.sshd` when it needs admin), `arp -an` sees the Dante devices, the switches answer on port 23, `launchctl print gui/<uid>/org.waterschurch.status-panel` (agent loaded / running / last exit / exit timeout) and the age of `started.json`.
**One-time setup checklist per computer** (Setup tab; every light uses the one status legend of the whole page, printed once under the top bar's "?": OK / Problem (+ one-line reason) / Can't see from here - ask at the computer / Not needed / Done by hand (who, when); every line has a [how] with the exact steps or the generated script):
- Every Mac: Remote Login on; Ingest login key authorised (generated `mac-authorize.sh`); host key known (with "changed since last time" + [Forget the old login key]); `/usr/bin/python3` present (for MIDI reads); DVS helper + sudoers (computers with `helper_dvs`); Audio MIDI Setup sessions named as the checks expect (expected beside live; a peer added by the Add a MIDI link job shows "will be added on first hold"; a typed session shows NO SESSION until it exists); apps at the listed paths; Studio B: solo agent running, `solo-mode.json` fresh, `trusted: true` (Accessibility), swiftc present for a rebuild.
- Every Windows PC: sshd Automatic + running; OpenSSH DefaultShell is cmd.exe (light); firewall rule allows this site's Ingest IPs; key login works (both authorized_keys files); a `\StatusPanel\` task per `task` run; services' failure-restart; WoL on the adapter; BIOS WoL (Done by hand); Public profile when rtpMIDI is listed (light + Done by hand); Vista autoload (Done by hand); Wi-Fi power (Done by hand); RESULT block pasted (fills ssh user, MAC, IP); the host-key-changed line as on Macs.
- This Mac: Command Line Tools / python 3.9-3.12; watchdog agent (loaded, running, ExitTimeOut set to the derived value); settings agent; Companion checklist (variables, blink trigger, pages exist, our pages, quiet Tascam module, ATEM module, "use last page" noted); keychain item (if `dante_switches`); the watchdog's Spotify permission (if a spotify check); DVS helper + sudoers (if dvs_ingest); login key pair ([Make the login key] at a new campus); editing code present; ports for LAN / Dante / switch management up; Art-Net port bound (if artnet); saved-file folder mounted (if `atem.source_folder` is set); Remote Login on (for the MacBook's `checklist-<id>` alias); Church Translate not loaded.
**Generated scripts** (`settings/generate.py`): "Generate" fills `remote-access/mac-authorize.sh.template`, `mac-dvs-helper.sh.template`, `windows-setup.ps1.template`, `install-solo-mode.sh.template` and `solo-mode-agent.swift.template` with the placeholders `{{PUBLIC_KEY}}`, `{{INGEST_IPS}}`, `{{APPS}}`, `{{DVS_HELPER}}`, `{{SOLO_DIR}}`, `{{SOLO_LABEL}}`, `{{PT_BUNDLE_ID}}`, `{{SOLO_MENU_PATH}}` and `{{SITE}}` from the site's public key, Ingest IPs, program table, helper paths and site name; every Mac value is shell-quoted, every Windows value passes the cmd-safe regex (and contains no `!` or `^`); the Windows script also sets the OpenSSH DefaultShell to cmd.exe. Each script is shown with Copy / Download and the exact place to paste it (ARD "Send UNIX Command as root" / PowerShell as administrator / scp + ssh for Studio B; a rebuilt solo agent needs Accessibility re-granted on that Mac). The page never runs them (guideline 12). The schema check compares `site.solo_dir` against the generated strings of the Studio B checks, the solo fix and the swift template, so all three agree by construction. The templates are deployed to the data dir by `install.sh`, and `test_generate_from_data_dir` (Phase 5) proves generate.py finds and fills every one of them there.
## C. Saving and applying — the pipeline
1. **Draft.** One shared draft per site, saved to `draft.json` in the data dir (0600, like every file the settings agent writes there) on every change (a closed tab loses nothing). It carries `base_sha` (hash of the live `config.json` it started from) and who started it; the server answers `PUT /api/draft` with an ETag and `409` when another tab wrote in between: the page merges field-wise and, when the same field was touched, says "<name> changed this a moment ago - your change to <field> was not kept" and re-offers the field. When you switch Editing on and a draft exists, the page says "Sam started changes at 14:02 - [continue them] [throw them away]". The live config is untouched until Apply.
2. **Validate** (on every change and again at Apply), in process: (a) schema rules from `schema.py`: types, enums, ranges, id characters, unique ids, master id `all`, one ALARMS / at most one NET, positions in the grid and off the nav cells, no overlaps including jump/back pos, windows well-formed (overnight allowed), MACs, URLs, playlist URI, fix label present, fix kind in `FIX_KINDS`, every `host` alias exists, has ssh and is not the `self` record for every fix, remote check and MIDI link, hidden key in a group, `never_auto` not in `auto_fix.fixes`, Windows params cmd-safe, local and Mac-ssh commands pass `zsh -n`, remote check commands and the batched line the poller builds pass `sh -n`, cmd-batched checks refused for a PC whose DefaultShell light is red, every `variables.*` name exists in Companion (a hard error when Companion answers, the warning "could not check - Companion is not answering" when it does not, so a rollback is never blocked by a Companion outage); (b) `status_panel.load_config()` on the draft written to a temp file; (c) `make_page.build()` + `build_restart()` in memory; (d) soft warnings from live data: label not in Companion, computer unreachable, playlist not public, module mismatch against `EXPECTED_MODULE`, a HyperDeck without a disk becoming non-optional, `live_services` empty, no watched deck follows the recording, `service_windows` narrowed; (e) a saved switcher file that is part of the draft: the upload is capped at 20 MB, XML with a DOCTYPE or entity declarations is refused, and the file is parsed in the settings process with the watchdog's own parser so Review can show what differs from the current file.
3. **Review.** The plain-words diff from the schema (raw JSON under Details); command edits (`fixes.*.run`, `remote.cmd`, `process.match`, `pref.*`) listed in red with the generated text and "not run before Apply"; what Apply will do, computed by building both page JSONs from the live config and from the draft, stripping the keys' notes, and comparing them with the stored hash of the last verified import (a changed note alone never forces a re-import; notes catch up at the next one, D11): "the deck will go purple for about 5 seconds and come back with your changes" (always, unless every change is page-only data), "the Status page will be re-imported" when the build differs, "the page-number buttons will be re-run" when a page changes, "the key guide will be rebuilt"; the live-service state with the exact reason (window or recording deck) and "watchdog silent: using the schedule only" when `live-now.json` is older than 30 s; the foreign keys on our pages listed by position when the marker-only rule would refuse; the state note "acknowledged alarms and auto-fix counts carry over, except on keys whose check list changed" (state is keyed by (key, check identity), never by index); a note for History; one APPLY button. There is no preview run: what a key will say after the change is shown on the key's editor (WHAT THIS KEY WILL SAY), and validate already builds both pages in memory.
4. **Apply** (`settings/apply.py`, one request under a server-side file lock `apply.lock` - a second Apply is refused with "an Apply is already running"; each step logged to `apply.log`, each must succeed before the next; refused up front when `sha(config.json) != base_sha`, and refused while live unless the Emergency override is used):
1. copy the live config to `history/<timestamp>-<slug>.json` (0600) + `.meta.json` (note, by, address, diff summary with secrets masked, actions, the hash of the switcher file it ran with, the code version); the 46 existing `config.json.bak-*` files are imported once;
2. write `config.json.tmp`, validate again, `os.replace` (never a half-written file); when a new switcher file is part of the change, write `atem-reference.xml.tmp` and `os.replace` it in the same step, keeping the old one in `history/atem/<date>/`;
3. if every change is page-only data (notes, gear notes): regenerate the key guide and stop. Otherwise **restart the watchdog**: wait until `fixes-running.json` says no fix is in progress (refuse after 60 s with "a fix is running on <key>"), read the old PID from `launchctl print`, then `launchctl kickstart -k gui/<uid>/org.waterschurch.status-panel` (if the agent is not loaded, `launchctl bootstrap gui/<uid> <plist>` first) and record T. The SIGTERM handler lets a fix that is already running finish (join up to ExitTimeOut-5) and refuses new ones meanwhile (keys say RESTARTING SOON), so a running fix finishes before the process is gone. **Success** = `started.json` `{pid, started_at, config_sha}` newer than T with the sha we wrote (an old PID's file never counts; a log rotation during the wait does not fail it), no new traceback in `status-panel.err.log`, process alive after 10 s; polled up to 25 s. A fresh heartbeat (`sp_hb` > T + 2) is a second, non-fatal check reported as "Companion not answering, settings are live" when missing. At start the watchdog clears the command variable once (logged), so a hold queued before the restart is dropped and logged;
4. **only then**, when the built page JSON (notes stripped) differs from the last verified build: `make_page.py` into the data dir, then `push_page.py` (Status page, then Restart page) with the **marker-only rule**: every non-nav key on the target page must carry one of our markers (`STATUS tile`, `RESTART key`, `STATUS jump key`, `RESTART back key`, `NAV_MARKER` = "STATUS nav key"; one `MARKERS` constant in make_page.py) whatever the page's name; otherwise Review already listed the foreign keys by position and Apply refuses with the verbatim reason and "import by hand" steps; `--force` never. If the page **number** changed, the same Apply resets every control on the old page that carries our markers (tRPC `controls.resetControl` once the Phase 1 spike proves it blanks a cell; otherwise Review lists "delete these N keys on page 19 by hand" and the Needs Chris strip keeps it until done). The import is **verified** by re-fetching the page with `includeSecrets=false` and comparing control ids; build hashes (`build-hashes.json`) are stored only after that. Then, because pages changed, the **nav keys**: `nav_keys.py --dry-run` shown, then applied (walking pages by `page_number_N_name` up to 200 with the 5-miss end, writing `nav_locations.json` in the data dir);
5. verify: no ERR in the last 20 log lines; regenerate `key-guide.html`; clear `pages-out-of-sync.json`. Nothing is copied to the staging folder `~/status-panel-install/` by Apply;
6. if step 3 fails (process died, traceback, no `started.json` with our sha): **rollback, once**: restore the previous config **and the switcher file recorded in that History entry** from the history entry, kickstart again, wait with the same test, re-run the page/nav steps for the restored config when the pages had changed, and report "restored previous settings: <error>"; if the restored config fails too, leave it in place and raise the red notice on Overview ("Restored the previous settings after a failed Apply - [See why]") until the next good Apply. If step 4 fails after a successful restart: write `pages-out-of-sync.json` (what to import by hand), roll back the config and re-run the page/nav steps for the restored config; if that also fails the notice stays until the next verified import, so the deck never silently shows new pages over an old config.
5. **Live-service guard** (independent of ALARMS). "Live now" = inside a `site.live_services` window (overnight windows count past midnight) **or** any watched HyperDeck that "follows the recording" is recording, read from the watchdog's `live-now.json` (written on change / every 5 s with a timestamp; older than 30 s = schedule only, said in Review). While live, **Apply, Run this fix now, Restart the watchdog, Re-import the pages now, Check the switcher now, Install the staged folder, Go back to this and Undo the last change are refused** with the reason and when it ends ("A live service is on (HD2 is recording). Apply is not allowed until it ends at 13:30. Your change is saved as a draft."); the draft stays saved and the live `config.json` is never touched, so the watchdog, if launchd relaunches it mid-service, always starts on the config it was verified with. There is no queue and no job runner. The **Emergency override** is a dialog that states the consequence ("the deck goes purple for 5 s during the service") and asks you to type `LIVE` plus a reason; both are logged with who and the address. **Start the watchdog** while it is down is allowed during a live service; the three watchdog buttons (Start / Stop behind the typed word / Restart) live on Overview only. **Silence alarms until midnight** (the ALARMS key's override, written through the watchdog) is never guarded. Viewing, probes and logs are always allowed (guideline 10).
6. **Go back / Undo.** History > Go back to this (or Undo the last change): the chosen version becomes your draft together with the switcher file its History entry names (refused with a sentence when that file is missing), and goes through the same pipeline (validation, guard, restart, pages when the build differs), bumping `base_sha`. History is never pruned automatically; [Prune older than N months] asks with the count. ATEM files are kept dated under `history/atem/`.
7. **Program updates.** `install.sh` first runs the **new** `status_panel.py`'s `load_config` and `schema.validate` over the config it will run with and refuses on error (so a stricter validator never crash-loops the watchdog); keeps the previous code as `prev/` with `install.sh --rollback`; writes both LaunchAgents (the watchdog plist with `ExitTimeOut` derived from the config = max fix timeout + verify + 10, floor 60; the settings plist with `AbandonProcessGroup`); restarts the watchdog first and waits for `started.json`, then the settings agent last, detached; writes `apply.log` and a History entry ("program update", with the code version). Once the page exists it **never overwrites site-owned files** in the data dir (`config.json`, `atem-reference.xml`, `nav_locations.json`, the built pages) unless `--config <file>`, `--site <id>` or `--force-site-files` is given, printing a diff notice when the staged file differs (`--keep-config` is the Phase 0-1 opt-out while the copy behaviour is still the default). It deploys `settings/`, `schema.py`, `make_page.py`, `push_page.py`, `nav_keys.py`, `remote-access/*.template`, `companion-objects/` and `atem_state.js`. **Install the staged folder** from the page (Site setup > Site) runs install.sh as a one-shot launchd job, so it survives the settings agent's own restart; refused while live; the page polls the new agents' health. The staged folder still arrives by `scp` to `~/status-panel-install` from the MacBook; there is no git clone, deploy key or Pull button on the Ingest Mac. `install.sh --site <id>` deploys `sites/<id>/` when the repo has it, otherwise `sites/_template/` (D12).
## D. Safety and access
- **Who can read and change what.** The server binds IP addresses, not interface names: at start it resolves the IPv4 address of each LAN-use interface in `hosts.ingest.networks` from `ifconfig` (every private-address interface when the list is empty, with a grey notice), re-checks every minute and rebinds on change; 127.0.0.1 is always bound and is the only listener that serves `/edit-key`; never 0.0.0.0, and it answers only private ranges (10/8, 172.16/12, 192.168/16, 169.254/16, 127/8); everything else gets 403. **Everything is readable on the LAN without the code**: Overview, Keys, Computers (inventories, generated scripts, the Dante map), Logs, History (list and bodies), the key guide, Help and every Site setup screen, with the alert web hook masked. **Every change needs the editing code** (`X-Edit-Key` header equal to the code in this Mac's keychain item `status-settings-edit-key`, with the 0600 file fallback), a `Host`/`Origin` header naming this server (CSRF), and a name; lockout for 10 minutes after 5 wrong tries; every write is logged with address and name and History records "changed by <name> from <address>". The code is shown only at `http://127.0.0.1:8020/edit-key` on the Ingest Mac's own screen (404 on a LAN socket) and is rotated from Site setup > Site > Advanced. Changing the port or the code shows a dialog that states the consequence. There is no admin code, no address allow-list and no second code; the editing code is not a password for any other system and is never typed by Claude.
- **Commands never run from the page before Apply.** `fixes.*.run`, `remote.cmd`, `process.match` and `pref.*` are "command edits": generated by builders, shown in red in Review, executed only by the watchdog after Apply; a raw Custom command (Advanced) asks for the editing code again before it is accepted into the draft. Probes run only the fixed set in `probes.py` with shlex-quoted / regex-checked parameters. "Run this fix now" never calls `execute_fix` in the settings process: it sends `test <key> <fix> <identity> <time>` on the command variable (`sp_cmd`), exactly as a hold arrives, and the watchdog resolves the check from that key (deck address from the snapshot, MIDI address from the last-seen peer address, recording-deck guard, cooldown, one MIDI fix at a time), logs it as TEST, and refuses a `test` whose fix is attached to no check on that key, a netgear_poe / dante-auto / atem fix, a recording deck, or a bare `test <fix>`; it is offered only for the start/reconnect kinds (start-only Windows/Mac/helper/DVS builders, Reconnect a Companion connection, Switch a HyperDeck off and on, Reconnect a MIDI peer, Play the Spotify playlist) and never during a live service. "Check the Dante network now" sends the `resurvey` verb the same way. The page never writes a runtime file the watchdog also writes: `dante-map.json`, `live-now.json`, `fixes-running.json`, `started.json`, `state.json` and the command variable are the watchdog's alone.
- **Server surface.** `GET /api/logs/<name>` serves only names on its list; `/api/live/spotify?url=` fetches only `open.spotify.com`; ATEM uploads are capped at 20 MB, XML with a DOCTYPE or entity declarations is refused, and the file is parsed with the watchdog's own parser in the settings process; the Companion DB reader returns label, module id and `config.host` only (the module version only if the spike finds it trivially; no surfaces, never a raw row); every page export is fetched with `includeSecrets=false`.
- **Secrets:** presence only (keychain items exist + account, login key exists, public key text, alert web hook "set, N chars"). `alert_webhook` and any key named `*token`, `*secret`, `*password` are masked in every API response, diff, History meta and site bundle, and every file the settings agent writes in the data dir (`draft.json`, `history/*`, `machines-live.json`, `settings.log`, the `edit-key` fallback) is created 0600; the real value lives only in `config.json` and is re-attached on restore. The page never reads, shows or writes the Netgear password, never stores a credential for any computer or for git, and never runs `security add-generic-password` or makes a key for a person; it shows the exact line to type. The only `ssh-keygen` calls it makes are the confirmed `-R <ip>` (Forget the old login key) and the confirmed creation of the panel's own login key pair (Make the login key).
- **Never automated:** setup scripts on other computers (ARD / PowerShell by the user, guideline 12), firewall or system changes, BIOS WoL, Remote Login, macOS permissions (Automation, Accessibility), creating Companion custom variables or triggers unless the Phase 1 tRPC spike proves it safe (then only with explicit confirmation and never a full import), module imports, `push_page --force`, full Companion imports, rebooting any device, power-cycling a PoE port or pushing the switcher file from a Run button (those fixes have no Run now), restarting the Resi bridge during a service ("never by itself"), any Apply or watchdog restart during a live service without the Emergency override, git pulls on the Ingest Mac.
- **Confirmations, one style:** a dialog that states the consequence and asks once; a typed word only for the Emergency override during a live service (`LIVE` + reason) and for Stop the watchdog (`STOP`: "the deck goes purple and every alarm is off until Start"). The dialogs: deleting a computer, key, fix or check shows everything that refers to it ("Delete Lighting PC and also remove 3 fixes, 4 checks, 1 restart key?"); "Run this fix now" names the computer or device; Restart the watchdog; Go back to this / Undo; a page-number change before the old page is cleared; a saved switcher file must be ticked after its diff is shown; Undo the page-number buttons on every page (rewrites 97 pages); Prune history (with the count); Re-import the pages now; Install the staged folder (shows the staged version); Forget the old login key (names the computer); Make the login key; Make a new editing code and changing the port (state the consequence); a Custom command edit asks for the editing code again.
- **Watchdog restart safety:** the Engine keeps a registry of running fix threads (every thread started in `run_plan`, `run_force` and the test verb) and publishes it as `fixes-running.json`; its SIGTERM handler sets the stop event, refuses new fixes (hold, force, auto or test answer RESTARTING SOON on the key) and joins the registry for up to ExitTimeOut-5 s before exiting; `install.sh` writes the derived `ExitTimeOut` (max fix timeout + verify + 10, floor 60; today launchd's deadline is 5 s) into the watchdog plist before the handler ships; Apply waits until the registry is empty (60 s) before the kickstart; a HyperDeck off/on fix re-enables the connection in a `finally`, so a kill between its off and on never leaves a deck connection switched off; acknowledged keys, auto-fix tries and fix cooldowns persist in `state.json` keyed by (key, check identity) so a restart never re-runs a fix early or forgets an acknowledgement; the command variable is cleared once at start. There is no SIGHUP partial reload and no stop flag: every Apply is a full restart on the new config.
- **Guideline fit:** no per-key colour options in the schema; the legacy `service_only` flag sits under Advanced with a warning and has no effect while alarms are armed all day; the alarms window is not on the page at all (1); one schedule, labelled so only auto-fix and the change guard use it and alarms stay armed, the one sentence everywhere (2); the validator warns when a key has nothing to fix and no "also when green", and the HOLDING 2 s WILL line shows the answer (3); restart keys always have a fix and the watch list keeps them flashing (4); builders generate real, tested fixes with a guarded Run now (5, 6); debounce defaults stay 3/1/0 s (7); the gentle list is enforced by load_config and "never by itself" (8); page-number buttons re-run inside Apply whenever a page changes, and the Companion section counts "coloured on N of M pages" when opened, with [Colour them now] (9); the live-service guard uses the clock and the built-in recording signal, refuses rather than queues, leaves the live config untouched and has one logged override (10); no passwords (11); scripts only shown (12); the repo stays private (13); help texts short (14); Apply waits for proof (`started.json`, verified import) before saying done (15); History and the generated key guide replace most README tables (16).
- The settings server imports only `load_config`, `CHECKS`, `RESTARTABLE`, `FIX_KINDS`, the ATEM parser and `make_page.build` from the watchdog code (never `run_once`), so a bug in the page cannot take the panel down; it runs as the GUI user with no sudo, as a separate process, so the page keeps working while the watchdog restarts or when the panel is purple.
## E. Architecture: files, server routes, watchdog changes
```
schema.py the registry: top-level keys, site/host/run/gear records, 19 check types, 8 fix kinds, tile and
restart-key fields; per field: type, default, enum/range/regex, required-when, help, example,
shows-on-key, screen (one owner screen), advanced flag, secret flag, page-only flag (notes, gear notes).
DEFAULTS, GLOSSARY (grouped by action), SUBS (per-type sub names), KEY_RECORD ((receiver kind, key) -> record),
IDENTITY (per check type: which fields key the persisted state), EXPECTED_MODULE (module id per check type),
BUILDERS (fix and check templates with param rules and quoting; the dvs_ingest second step rendered inline),
QUESTIONS (the question-first "Add a check" menu), PROBE_SAFE (runs fields that may reach a probe line),
validate(cfg, live=None) -> errors, warnings; plain_diff(a, b); mask_secrets(obj); phrases(cfg) (the
"may run by itself ..." and window sentences built from the config)
settings/__init__.py adds the parent folder to sys.path so status_panel/make_page import from the data dir
settings/server.py stdlib ThreadingHTTPServer bound to IP addresses, not interface names: 127.0.0.1 always (the only listener
that serves /edit-key) plus the IPv4 address of each LAN-use interface in hosts.ingest.networks resolved
from ifconfig at start (every private-address interface when the list is empty), re-checked every minute
and rebound on change; one thread per address; editing code + Origin check on every write; reads open
GET / /app.js /style.css /key-guide.html /cheat.html /where.html GET /edit-key (127.0.0.1 only)
GET /api/schema /api/config (masked) /api/draft (ETag) PUT /api/draft (If-Match, 409 with the field) DELETE /api/draft
POST /api/validate /api/apply (lock; refused while live unless the body carries the Emergency word + reason)
POST /api/restore/<id> /api/undo
POST /api/watchdog/command (test <key> <fix> <identity> <time> | resurvey, sent on the command variable like a hold)
POST /api/watchdog/start|stop|restart (launchctl; the buttons live on Overview only; guarded, Stop behind the typed word, Start allowed while down)
GET /api/history /api/history/<id> /api/history/atem
GET /api/live/overview (deck from sp_state, heartbeat, state.json summary, live-now with age, started.json, launchd state)
GET /api/live/companion (connections with module id and status, lights, page-number buttons coloured on N of M)
GET /api/live/machines POST /api/live/machines/probe GET /api/live/dante-map GET /api/live/gear
GET /api/live/interfaces /api/live/midi?host= /api/live/launchd /api/live/processes /api/live/bonjour
GET /api/live/spotify?url= (open.spotify.com only) GET /api/live/atem-state (guarded)
GET /api/logs/<name> (whitelist) POST /api/atem-reference (upload, capped, no DOCTYPE) POST /api/atem-reference/pick-newest
GET /api/generate/<script>/<alias> POST /api/hosts/<alias>/forget-host-key (asks)
POST /api/companion/nav-keys /api/companion/nav-revert /api/companion/reimport POST /api/program/install (one-shot launchd job)
GET /api/site/export (bundle, masked) POST /api/site/import (Phase 6)
settings/render.py server-side rendering of the screens from the config, the schema and the live data
settings/probes.py per-OS fixed probe commands and parsers incl. the full-inventory lists and the DefaultShell probe (section B),
machines-live.json cache; no vendor lists
settings/apply.py history (with the switcher-file hash and code version), atomic write of config + xml, base_sha check, the
apply.lock file lock (a second Apply is refused with "an Apply is already running"), fix-in-progress wait, kickstart (bootstrap when not loaded), the started.json success test, make/push/nav,
verified import + build hashes, old-page reset or by-hand list, pages-out-of-sync, auto-rollback once
(config + xml, pages re-run), the live guard that refuses with the reason, the Emergency override;
also `python3 settings/apply.py <config> --note "..."` (CLI)
settings/generate.py fills remote-access/*.template with site data (quoted / regex-checked)
settings/keyguide.py builds key-guide.html, cheat.html, where.html from config + schema
settings/ui/index.html app.js style.css vanilla JS, no framework; the grid editor is plain DOM drag/drop; the top bar holds the site name,
the search box (finds any key, fix, computer, device or setting by name), the unsaved-changes badge, the
Editing toggle and the "?" with the one status legend; strings in ui/strings.json (wording test)
remote-access/*.template mac-authorize.sh, mac-dvs-helper.sh, windows-setup.ps1, install-solo-mode.sh, solo-mode-agent.swift with
{{PUBLIC_KEY}}, {{INGEST_IPS}}, {{APPS}}, {{DVS_HELPER}}, {{SOLO_DIR}}, {{SOLO_LABEL}}, {{PT_BUNDLE_ID}}, {{SOLO_MENU_PATH}}, {{SITE}}
companion-objects/ flash_tick_500.companionconfig, tascam-da-6400-1.3.1-quiet.tgz, nav_templates/<version>.json
sites/<id>/, sites/_template/ (Phase 6; until then config.json stays at the repo root)
tests/fixture-config.json frozen copy of today's NA config for the behaviour tests
install.sh + --site / --config / --keep-config / --force-site-files / --rollback; validates the config with the NEW code
before bootout; keeps prev/; deploys settings/, schema.py, make_page.py, push_page.py, nav_keys.py,
remote-access/*.template, companion-objects/, atem_state.js; writes both LaunchAgents (watchdog plist with the
derived ExitTimeOut, settings plist with AbandonProcessGroup); restarts the watchdog first and waits for
started.json, the settings agent last (detached); never overwrites site-owned files once the page exists;
prints the /edit-key address; writes apply.log + a History entry
tests/test_schema.py test_wording.py test_apply.py test_probes.py test_settings.py (beside the existing tests/test_status_panel.py)
```
The page tools (`make_page.py`, `push_page.py`, `nav_keys.py`) run from and write to the **data dir** on the Ingest Mac (they load `HERE/config.json`; `nav_locations.json` and the `.companionconfig` outputs live there). The data dir holds, besides today's `config.json`, `atem-reference.xml`, `atem-reference.json`, `dante-map.json`, `nav_locations.json`, the two built pages, `status-panel.log`, `status-panel.err.log` and the `config.json.bak-*` files: `draft.json`, `history/` (+ `history/atem/<date>/`), `apply.log`, `apply.lock` (the server-side Apply lock: a second Apply is refused with "an Apply is already running"), `machines-live.json`, `settings.log`, `key-guide.html`, `build-hashes.json`, `fixes-running.json`, `live-now.json`, `started.json`, `state.json` (replaces `service-mode.json`, migrated once), `pages-out-of-sync.json`, `edit-key` (when the keychain is unavailable), and the deployed `remote-access/*.template` and `companion-objects/`. Every file the settings agent writes there (`draft.json`, `history/*`, `machines-live.json`, `settings.log`, the `edit-key` fallback) is created 0600.
**Code changes in the watchdog** (`status_panel.py`, additive, keeps its `.get(key, literal)` defaults): a `VERSION` constant, the "status panel running (code X)" line and `started.json` `{pid, started_at, config_sha}` at start; new keys read with defaults (`site.{id,name,ingest_name,ingest_hostname,ingest_ips,contacts,alert_prefix,live_services (defaults to auto_fix.windows; the migration writes it),dvs_helper,solo_dir,settings_port}`, `timing.{fix_cooldown_seconds,alert_after_seconds}`, `page_hold_ms`, `page_grid`, `page_stale_seconds`, `variables.flash`, `companion.{export_version,build,db}`, `auto_fix.{forget_seconds,keep_months}`, `dante_max_shared`, `atem.{compare,source_folder}`, `spotify.{playlist,name,open_wait,play_wait}`, `fixes.*.{never_auto,template,params}`, `checks.*.{template,params}`, `remote.shell`, `hosts.*.{name,os,role,notes,pingable,self,runs,networks,manual_done}`, `gear.*.{location,notes}`); overnight windows in `in_windows` (end <= start runs past midnight); the "HOLD 2s" and "WAIT 1 MIN" texts built from `page_hold_ms` / `fix_cooldown_seconds` (one `self.cooldown`); a fix-thread registry (started in `run_plan`, `run_force` and the test verb) that writes `fixes-running.json`; a SIGTERM handler (stop event, refuse new fixes with RESTARTING SOON, join the registry up to ExitTimeOut-5); the HyperDeck off/on fix re-enabling in a `finally`; `live-now.json` written on change / every 5 s with a timestamp from the hyperdeck checks that follow the recording (`status == record`); one atomic writer for `state.json` with acknowledgements, auto-fix tries and `fix_last` keyed by (key, check identity) per `IDENTITY`, dropped for a key whose checks changed, with `ServiceMode` merging into it (migrating `service-mode.json` once); the command variable cleared once at start (logged, a queued hold dropped); a `test <key> <fix> <identity> <unix>` verb in `handle_commands` that resolves the check from the key as `run_fixes` does (deck ip from the snapshot, MIDI address from the last-seen peer address, recording guard, cooldown) and refuses netgear_poe / dante-auto / atem / a fix not attached to that key / an identity not on the key; a `resurvey` verb; the Windows (`cmd`) variant of remote-check batching as `(cmd) >nul 2>&1 && echo SPn=0 || echo SPn=1` pairs joined with `&` (never a bare `%ERRORLEVEL%`), with cmd-batched checks refused for a PC whose DefaultShell is not cmd; wording from `site.*` and `schema.phrases` (log header, Dante log line, webhook text); fix kind `spotify` (playlist and name from `spotify.*`); `FIX_KINDS` exported and validated in `load_config`; the host validated for every fix, remote check and MIDI link (exists, has ssh, not self); `never_auto`; `EXPECTED_MODULE` per check type; `hyperdeck_probe` removed in P0 together with `TestHyperDeckProbe`/`FakeDeck` (dead since the HyperDeck hold toggles the connection); a python 3.9-3.12 self-check at start (telnetlib). No SIGHUP reload, no `then` chain, no stop flag, no commands folder poll.
`make_page.py` reads `page_hold_ms`, `page_grid`, `page_stale_seconds`, `variables.flash`, `companion.export_version/build`, the site name and `remember_seconds` for the notes, writes the page address into the ALL key's note ("Settings: http://na-ingest.local:8020/"), gives the two static jump keys the markers "STATUS jump key" / "RESTART back key", exports `MARKERS` as one constant, reads the per-version nav template from `companion-objects/nav_templates/`, builds no Restart page and no jump key when the section is absent, and takes an output dir. `push_page.py` imports `MARKERS`, extends `safe_target` to the marker-only rule (every non-nav key carries a marker, whatever the page name; foreign keys reported by position) and fetches with `includeSecrets=false`. `nav_keys.py` walks by `page_number_N_name` up to 200 with the 5-miss end and writes `nav_locations.json` in the data dir. `push_page.py` / `nav_keys.py` default `--host/--port` from `companion.*`. `netgear.py`'s CLI reads `dante_switches` from the config.
## F. Tests
The source's test 10 (`test_apply_class`) went with the apply classes and is not listed; `python3 -m unittest` runs everything below beside the existing `tests/test_status_panel.py`.
1. `test_code_keys_in_schema`: a **receiver-aware** AST walk over `status_panel.py`, `make_page.py`, `push_page.py`, `nav_keys.py`, `settings/*.py`: it tracks names bound from `cfg`, `cfg[...]` paths, `chk`/`c`, `fix`/`f`, `t`/`tile`, `k`/`key`, `rp`, `names`, `a` (auto), `self.cfg`/`self.net`/`self.auto`/`self.restart_keys`, `w` (windows), host entries, and collects every `.get("literal"[, default])` and `X["literal"]` whose receiver is one of those; each (receiver kind, key) must be declared in `schema.KEY_RECORD`; subscripts of other receivers (dante-map entries, ATEM reports, tRPC results, nav templates, state.json, Companion status rows) are covered by an explicit per-receiver allowlist. The test asserts "every config read is declared", not "every literal subscript".
2. `test_defaults_match_schema`: for every config read whose default node is an `ast.Constant`, the literal equals `schema.DEFAULTS` for that path; expression defaults (`fix.get("label", fix_id)`) are listed in the schema as "derived" and skipped.
3. `test_schema_keys_used_in_code`: the reverse, with an explicit allowlist for UI-only keys (`name, os, role, notes, runs, networks, manual_done, pingable, self, template, params, contacts, settings_port, gear, source_folder, ...`) so dead config keys are caught but UI data is not flagged.
4. `test_check_types_and_fix_kinds`: `set(schema.CHECKS) == set(status_panel.CHECKS)`; `set(schema.FIX_KINDS) == set(status_panel.FIX_KINDS)`; every per-type sub list in `SUBS` matches the sub names the check returns; every `IDENTITY` field is a declared field of that check type; every `EXPECTED_MODULE` id is one the fixture's live connection list uses for that check type.
5. `test_glossary_covers_fault_words`: AST-collect only the text arguments of `Result(...)` constructors, `self.note(...)` calls and `return "..."` statements inside `check_*`, `render`, `run_fixes`, `run_force`, `reconnect_and_verify` (for `%`-formatted strings the template with placeholders stripped), with an allowlist for log-only strings; each distinct upper-case word sequence of two characters or more must appear in `GLOSSARY`.
6. `test_page_roundtrip`: today's config + schema defaults builds `status-page`/`restart-page` exports identical to the committed ones **after stripping `options.notes`**, and asserts the notes differ only where intended (the ALL note gains the page address, the static jump keys gain their markers, site wording); a config without `restart_page` builds no Restart page and no jump key. Run in Phase 0 and kept.
7. `test_sites_validate`: every `sites/*/config.json` (root `config.json` until Phase 6) passes `schema.validate`, `load_config`, builds both pages, the key guide renders with no "?" placeholders, builders regenerate, wording passes; `sites/_template` has one `self` record with `networks: []`.
8. `test_builders_regenerate`: for every fix **and check** with `template`, regenerating `run` / `cmd` / `match` from `params` equals the stored string, for all 27 of today's fixes (vista, arena, obs, bibleapp, rtpmidi_start, propresenter, ableton_restart, translate open-only, translate_restart with "also stop everything it started", ableton, dvs_prop, dvs_tracks, dvs_translation, dvs_ingest with prodcom rendered inline, bridge, caption_screen, captions with two helpers, proddash, prodcom, atem_midi, solo_afl, spotify_play, tascam_reconnect, hyperdeck, midi_reconnect, atem_audio, atem_camera); includes a metacharacter case (`Ableton Live 11 Suite`, a name with `'`, a Windows path with `&` or `!` refused); the "only if not already running" guard of every start builder; the batched Windows line contains no bare `%ERRORLEVEL%` and uses one `&& echo SPn=0 || echo SPn=1` pair per check; a builder given the `self` record emits a local kind; the `site.solo_dir` agreement check.
9. `test_apply_dry`: the pipeline against a temp data dir with a fake `launchctl` and the existing fake Companion: history written masked with the switcher hash and code version, secrets re-attached on restore, atomic write of config and xml, base_sha refusal, a second Apply refused by `apply.lock` with "an Apply is already running", fix-in-progress wait, a fix that starts between the empty-registry check and the kickstart is joined by the SIGTERM handler before the old process exits, success judged from `started.json` (an old PID's file must not count; a log rotation during the wait does not fail it; an unversioned or old `started.json` never counts as success), auto-rollback once on a bad config restoring both files, no second rollback, pages rolled back and `pages-out-of-sync.json` written when the page step fails, page / nav-key detection from built-page hashes (incl. the Phase 2 refusal case "available after the layout step"), page-number change resets the old page or lists the by-hand step, name-only change instructs "rename in Companion first", a missing variable is a hard error only when the fake Companion answers, the guard **refuses** and leaves `config.json` and the draft untouched inside a live window (incl. an overnight window) and while a rec_sync deck reports `status == record`, a stale `live-now.json` (> 30 s) falls back to the schedule, the Emergency override applies and is logged, Start-the-watchdog is allowed while live when the agent is down, persisted acks dropped for a key whose checks changed and kept for the other five MIDI peers when one is acknowledged, page-only data (a note) applies without a restart, the install job survives the settings agent's restart, a hold queued before the restart is dropped and logged at start.
10. `test_sigterm_mid_fix`: a SIGTERM during a stubbed 10 s fix lets it finish and the registry empties before exit; a kill between a HyperDeck's off and on still leaves the connection enabled (the `finally`); `install.sh`'s plist contains an `ExitTimeOut` equal to the value derived from the fixture config and the settings plist contains `AbandonProcessGroup`.
11. `test_probe_parsers`: parsers fed captured outputs of `launchctl list`, `ls LaunchAgents`, login items, `schtasks /Query /FO CSV` (full list), `sc query`, `Get-CimInstance Win32_Service`, `lsof LISTEN`, `Get-NetAdapterPowerManagement`, `Get-NetFirewallRule`, `ifconfig`, `dns-sd`, the nav_keys page walk with the page-96 gap, the `%COMPUTERNAME%` answer from cmd and from PowerShell (the DefaultShell light); every listed run gets the expected light and the raw inventory renders in full on the card; a run named `x; rm -rf ~` is refused by `schema.validate` and, forced in, produces a quoted, inert ssh line.
12. `tests/test_wording.py`: scans `settings/ui/strings.json`, every schema help and label, and the Help pages (not the rendered DOM) for the forbidden-token list (underscored words, `--flags`, `.py/.json/.plist`, `sp_state`-style names, launchd labels, `r2c6`-style positions, bare connection labels outside a pill, "tile", "machine", "host", "builder", "edit key", "nav key", "Sunday", "North Attleboro", "always on", "detect", "Refresh") outside Advanced sections and generated text; the four chosen words (key, computer, fix, connection) and "editing code", "login key", "page-number buttons" are allowed; every button label starts with a listed verb and read-only ones with "Check"; the alarms sentence appears verbatim wherever alarms are mentioned.
13. `test_push_page_markers`: `TestPage.test_push_page_looks_at_what_is_on_the_page` extended with a page holding only our marked tiles plus the two jump keys (accepted), a page with only nav keys (accepted), a page with the expected name and one foreign key (refused, with the key's position in the reason), and one with a foreign key and another name (refused).
14. `test_test_verb`: in `TestFixes` with the stubbed executor, `test HD2 hyperdeck <identity> <now>` resolves HyperDeck_2's conn and ip, `test MIDI midi_reconnect {"host":"tracks","session":"NA Tracks","peer":"Studio B"} <now>` resolves exactly that peer's address and leaves the other five alone, a recording deck is left alone, a bare `test <fix>` is refused, an identity not on the key is refused.
15. `test_settings`: routes against the fake Companion: every read answers without the code (logs, history bodies, masked config, inventories, generated scripts) and every write is refused without it; `/edit-key` 404 on a LAN socket and 200 on loopback (the only listener that serves it); the bind set resolved to IPv4 addresses from a canned `ifconfig` with a full / empty / stale `networks` list, 127.0.0.1 always present, and a changed address rebound on the minute re-check; lockout after 5 wrong codes; 409 field-wise merge and the same-field message; a Custom command edit needs the editing code typed again; secrets masked in every response; every file the settings agent writes in the data dir (`draft.json`, `history/*`, `machines-live.json`, `settings.log`, the `edit-key` fallback) is created 0600; the Origin check refuses a write from another host.
16. Existing tests: the behaviour suite loads `tests/fixture-config.json`; `TestPage`/`TestFixes` use `find_tile(cfg, id)`; `TestHyperDeckProbe`/`FakeDeck` deleted with `hyperdeck_probe` in P0 (dead since the HyperDeck hold toggles the connection); the valid-fix-kinds assertion reads `status_panel.FIX_KINDS`; the committed-pages test points at `sites/na` after Phase 6.
17. Phase 5: `test_generate_from_data_dir`: install.sh's copy step into a temp data dir, then generate.py finds every template and fills it.
No new dependencies: python3 stdlib (http.server, json, ast, subprocess, sqlite3, gzip, hashlib, threading, difflib, zipfile, urllib, signal, fcntl, secrets, shlex), Companion's node for ATEM as today, browser-side vanilla JS.
## G. Coverage checklist
Every inventory item, and where the plan handles it or why it is deliberately not editable. DNE = deliberately not editable because it is how the panel works, not what it watches (guideline 1: identical keys everywhere); documented in Help > Glossary / Where things are.
**Lens 1: config keys, code reads, hardcoded constants**
| Item | Handled where |
|---|---|
| companion.host, port | Site setup > Companion (Advanced); push/nav read them (Phase 0) |
| companion.db | Site setup > Companion (Advanced); the reader returns label / module id / config.host only (module version only if the Phase 1 spike finds it trivially; no surfaces) |
| page, page_name | Keys tab header ("Page 19"; number under Advanced; the Companion section links); name derived; marker-only safe target incl. the jump keys; foreign keys listed by position; old page reset or by-hand list / rename-first (§C, test 9); pages + nav keys |
| nav_keys, nav_hold_ms, nav_tap_page | Site setup > Companion (Advanced) as "page-number buttons" with the "use last page" note; Apply runs nav_keys whenever pages changed (export walk, 5-miss end); Undo button (asks) |
| variables.state/heartbeat/command; sp_state/sp_hb/sp_cmd by hand | Site setup > Companion lights; existence is a hard error when Companion answers, a warning when not; §C.7 bootstrap (tRPC spike Phase 1); the command variable is cleared once at every watchdog start |
| flash_tick custom variable + trigger flash_tick_500 | `variables.flash` NEW with the rename-the-trigger note; trigger light; `companion-objects/` export deployed to the data dir; by hand in Companion |
| service_windows, window shape days/start/end | not on the page (DNE in Help; schema default all day; a narrowed value warned in Review); validation (days, HH:MM, overnight = end <= start allowed) |
| debounce_seconds, per-check debounce, flap.count, flap.window_seconds | Site setup > Schedules Advanced Timing (restart, like every Apply); check row More settings |
| network.interface (live list), gateway, burst_connections/groups/window, remember_seconds | Site setup > Network & Dante (port and gateway main; blip rule Advanced with defaults) (restart: pollers built at start); NET note from remember_seconds |
| alert_webhook, "NA Ingest" text, 30 s threshold | Site setup > Site masked URL + prefix; `timing.alert_after_seconds` Schedules Advanced (restart: captured in run()); also receives a line for every command edit applied |
| ssh_key, key pair, public key in scripts | Site setup > Site (Advanced) "login key" light + public half; Make the login key (asks, the panel's own pair) on the Setup tab; templates (§B) |
| dante_switches, dante_map_seconds, netgear.py SWITCHES | Site setup > Network & Dante (switches main, survey Advanced); the CLI reads the config (Phase 0) |
| hosts map, .ssh, .mac, .wake_broadcast | Computers; validated for every reference (exists, has ssh, not self); an address change restarts the watchdog (MIDI streams and batched checks restart) |
| fixes map, .kind (validated), .host, .run, .timeout, .short, .label (required), .reconnect, .conn, .restart_ok, .part, .verify_seconds, .off_seconds, .hung_note, .switch/.port/.mac, .template/.params NEW | builders inline in the key editor and the Fixes catalogue (all 27 mapped exactly in §F test 8, quoted, `zsh -n`, the dvs_ingest -> prodcom step rendered inline, helper lists, self -> local); §A rows; the largest timeout sizes ExitTimeOut |
| fix kinds ssh/local/atem/reconnect/midi_connect/netgear_poe/hyperdeck | builders; `spotify` added; Run-now allowed-kinds rule; `test <key> <fix> <identity>` resolves the check; hyperdeck `finally` |
| "restart-connection", "dante-auto", per-sub fix map | key editor picker / Advanced table (§A) |
| implicit "enable" plan | DNE; Help explains; shown in the HOLDING 2 s WILL line |
| tiles list, .id (regex, `all`), .label, .pos, .kind, .members, .checks, .hold_always; hidden tiles; groups | Keys grid (8x4 default, fixed special chips, labelled tray) + key editor (progressive, When held with the inline fix builder, two previews); page detection from Phase 2 |
| checks common: type, conn (live, module-filtered), fix, tag, service_only (warned), debounce_seconds | Keys check rows via the question-first menu; persisted acks keyed by IDENTITY |
| conn.ignore_warning; conn fault texts; short_reason map | field; Glossary (DNE) |
| hyperdeck conn/optional/disk_service_only/full_minutes/low_minutes/rec_sync/ip; decks 2 and 4 only; module variable names | Keys HyperDeck card (expects bmd-hyperdeck, mismatch warned); Add or remove a HyperDeck job end to end incl. the per-deck "may have no disk" question (the emptied cells stay empty; one click moves HD4 next to HD2; D15); variable names DNE; rec_sync decks feed the live-service guard |
| dante conn/devices (live)/count_all; module variable names | Site setup > Network & Dante (only home; the check row links there; expects audinate-dantecontroller); names DNE |
| shure conn/channels/battery bars; psm conn/channels/muted_is_error; tascam conn (+ expected modules) | Keys rows with EXPECTED_MODULE filtering and the Review warning |
| ping host/conn/name/msg/failures (missing-host crash) | Keys; host or conn required |
| launchd label/name (picker); process match/name (builder); http url/msg/name/interval (>=5); disk path/warn/err | Keys via the question-first menu; http is never a live signal |
| midi_peer session/peer/host/address (+1 s debounce); coremidi identity | Keys MIDI link card; Add a MIDI link job end to end incl. a typed session/peer, a new peer, Bonjour and connection watches, and the question which key it goes on (MIDI when one exists, otherwise a new key; D14); host validated (Mac, has ssh, not self); identity (host, session, peer) for acks and Run now |
| pref domain/key/want/msg | Keys with the "A macOS preference equals" builder |
| spotify playlist (twice) / name (unread) / hardcoded behaviour / spotify_play run string | Site setup > Spotify one field; `spotify.name` is the single owner read by the Play builder's label/short, the key guide and the notes; `spotify` fix kind; Change the playlist job on Overview end to end; behaviour DNE; the settings agent never touches Spotify itself (no preview run, no Automation permission of its own) |
| mdns instance/name (picker); Bonjour type fixed | Keys; type DNE |
| midi_rx source/name/max_silence_seconds | Keys |
| remote host/cmd/msg/shell NEW (Macs and Windows PCs); artnet source/max_silence/msg (port DNE); atem check part/msg | Keys; remote built from "A program is running on a Mac" / "on a Windows PC" / "A helper file is fresh" builders, Custom under Advanced with the editing code typed again, `sh -n` on the batched Mac line, Windows pairs with no `%ERRORLEVEL%` (verified on the Lighting PC), DefaultShell light, never run before Apply; host not self |
| atem section/host/node/library (resolved)/reference (upload, dated history)/source_folder NEW; MIX_OPTIONS/CCU_PARAMS; parser; atem_watch.js constants; atem_state.js | Site setup > ATEM incl. `atem.compare`, pick-newest from the folder, upload cap/DOCTYPE/diff/tick, live refusal, the saved file as part of the applied set (atomic, hashed in History, restored by rollback and Go back, never overwritten by install.sh), guarded "Check the switcher now" with the client-limit note; "this campus has no ATEM" switch; js constants DNE |
| auto_fix section/windows/after/tries/retry/pause/restart_connections/fixes/forget_seconds/log_dir/retention/header text | Site setup > Schedules (tick list and restart_connections main; numbers, folder and `keep_months` Advanced with defaults); windows derived from live services; header and "may run by itself" phrase built from site + windows; restart (AutoLog in __init__) |
| site.live_services NEW (and the former site.live_signals idea) | Site setup > Schedules (one schedule incl. overnight windows); defaults to `auto_fix.windows`, the migration writes it; "Also counts as live" shows only the built-in recording-deck signal (no conn/var/value triple; "OBS is streaming" only as a built-in kind if the Phase 1 spike finds the module variable); stale handling (30 s); no quiet time and no queue: Apply is refused while live and the draft is kept |
| restart_page section/page/page_name/jump_pos/back_pos/keys id/label/pos/fix/watch; section absent | Keys Restart tab (jump/back chips, "this campus has no Restart page" switch with its cascade; make_page builds nothing when absent) |
| make_page options/steps/style, nav templates, stale texts, PURPLE, button style, action ids, notes markers | DNE; notes wording from the site (+ the page address on ALL, markers on the jump keys, one `MARKERS` constant); stale seconds editable; nav templates per version in `companion-objects/nav_templates/` (root file dropped) with the recapture recipe |
| HOLD_MS, GRID, STALE_SECONDS, NAV_LOCATIONS, EXPORT_VERSION/COMPANION_BUILD | `page_hold_ms` (read by the watchdog for "HOLD 2s"), `page_grid`, `page_stale_seconds`, `companion.export_version/build` (picks the nav template); NAV_LOCATIONS DNE (locked cells) |
| push_page/nav_keys --host defaults; "on NA Ingest" wording | read the config; site block (Phase 0) |
| poller intervals, fix cooldown, HOLD_MAX_AGE, freshness limits, main loop, HTTP timeout, remote MIDI 12 s, ATEM 8/5/30 s, Dante-wide 600 s | cooldown editable (one `self.cooldown`; restart) and read for "WAIT 1 MIN"; the rest DNE, listed in the Glossary |
| COLORS/ORDER, RESTARTABLE, ssh options, ports (WoL 9, 6454, 9993, 23), text fitting, CLI flags, OS tool paths, Companion API paths, db read, command grammar | DNE; hyperdeck_probe removed with its test; the `test <key> <fix> <identity>` and `resurvey` verbs added to the grammar; Where things are shows the data dir / CLI |
| runtime data: service-mode.json (-> state.json, one writer, identity-keyed acks/tries/cooldowns), dante-map.json, atem-reference.json, config.json.bak-*, fixes-running.json, live-now.json (timestamped, on change / 5 s), started.json NEW, pages-out-of-sync.json | Overview (ALARMS sentence, live now, the Needs Chris strip: pages out of sync, failed apply restored, page-number buttons missing), Network & Dante map, ATEM, History import (the .bak files once); one writer per file (the watchdog); the page sends `test` / `resurvey` on the command variable and writes no watchdog file; no pending files, no commands folder |
| Companion connection labels referenced by checks | Site setup > Companion live list with module id, mismatch rename-to (fictional example), module-mismatch warning, Add a key for it |
| Netgear keychain secret | Site setup > Network & Dante presence + account only; typed in Terminal by hand |
| external setup referenced from run strings (DVS helper path, LaunchAgent labels, schtasks names, solo-mode paths, Pro Tools bundle id and menu path, Automation) | `site.dvs_helper`, `site.solo_dir`, `runs` incl. `pt_bundle_id`/`solo_menu_path` (§B, PROBE_SAFE whitelist); fix and check builders derive names; swift template with four placeholders; lights |
| install.sh workflow and running copy in App Support | the page edits the App Support copy; install.sh validates with the new code, keeps prev/ with VERSION, deploys code + templates + companion-objects, never overwrites site-owned files unless told, derives ExitTimeOut, bootstraps the watchdog first and waits for started.json, restarts the settings agent last, runs as a one-shot launchd job from the page, writes History (§C.7); Install the staged folder under Site setup > Site |
**Lens 2: hardcoded site/layout things**
| Item | Handled where |
|---|---|
| site names in docstrings, AutoLog header, Dante log line, webhook texts, day names in UI phrases | `site.*` + `schema.phrases()` (Phase 0); "Sunday"/"North Attleboro" forbidden in typed strings (test_wording over strings, schema help and Help pages) |
| log path/rotation, --config/--data-dir defaults | DNE; Logs (open on the LAN), Where things are; rotation no longer affects the Apply success test (started.json) |
| auto-fix log folder/pattern/retention | log_dir + keep_months (Schedules Advanced) |
| COLORS, ORDER/RANK, text fitting, stale size/purple | DNE (Glossary > Colours) |
| hold-feedback, rendered and fault texts, short_reason | Glossary grouped by action; "HOLD 2s"/"WAIT 1 MIN" follow hold_ms/cooldown in the watchdog; RESTARTING SOON added |
| poller intervals, loop timing, Companion HTTP timeout, remote MIDI 12 s, ping method/misses, link method, Bonjour browse, HTTP method, hold freshness, cooldown, auto_fix/network/flap defaults | editable where listed in §A (numbers under Advanced with defaults beside them); methods DNE |
| HyperDeck/Shure/PSM/Tascam/Disk/Dante/Art-Net/ATEM/midi_rx defaults and module variable names | per-check fields (§A) with schema defaults; variable names DNE; the expected module per type in `EXPECTED_MODULE`; the built-in recording signal reads the HyperDeck `status` variable of rec_sync decks |
| CHECKS registry, per-check keys, RESTARTABLE, reserved fix ids, fix kinds, shells, exit-255/WoL, reconnect strings, hyperdeck_probe, load_config rules | the schema mirrors them with sync tests (§F, receiver-aware walk, IDENTITY); probe removed; validation reused (§C) |
| tiles/layout logic, restart_page schema (optional), special kinds, DAYS/HH:MM (overnight) | Keys (fixed chips for ALL/ALARMS/NET, no-Restart-page switch), Schedules |
| OS tool paths; launchd/pgrep/defaults methods | DNE (Ingest is a Mac); labels/domains editable through builders |
| Companion API paths, DB read, custom variables by hand, sp_cmd grammar | DNE; Site setup > Companion |
| Spotify AppleScript, playlist reader, URI twice, spotify_play string | DNE / single source `spotify.*` incl. name / builder (Site setup > Spotify, Change the playlist job); no settings-side permission; one light from the watchdog log |
| ATEM MIX_OPTIONS/CCU_PARAMS, parser, globs, restore file, source folder, atem_watch.js, atem_state.js | Site setup > ATEM |
| ssh_cmd options, hosts schema, remote MIDI over ssh, remote check batching (sh and cmd), known_hosts, DefaultShell | DNE; Computers; Windows batching with conditional echo pairs; DefaultShell probe + the generated script line; Forget the old login key; the self record refused as a host |
| netgear.py SWITCHES, KEYCHAIN_SERVICE, CLI driver, MAX_SHARED, ARP/VLAN interfaces, dante_map_seconds | config; Network & Dante; driver DNE; `dante_max_shared`; `networks` on the Ingest card (also the page's bind list) |
| coremidi NETWORK_DRIVERS, client names, identity rules | DNE; "no network MIDI driver found" light on the Ingest card and in the MIDI pickers; typed fallback when a Mac cannot be read |
| make_page EXPORT_VERSION/GRID/STALE/HOLD_MS/stale texts/style/jump keys/NAV_LOCATIONS/nav_templates/hold decision/notes/output names/action ids | editable: stamps, grid, stale, hold, jump chips; hold decision = built-page hash detection with verified import (§C, active from Phase 2 as a refusal); jump keys carry markers; nav template per version from companion-objects; outputs to the data dir; the rest DNE |
| push_page defaults/import mechanics/safety rule; nav_keys defaults/walk/nav_locations.json | defaults from the config; marker-only `MARKERS` rule in the Apply path (name shortcut removed), foreign keys listed by position, refusal surfaced, never --force; export walk with the 5-miss end; "coloured on N of M pages" counted when the Companion section is opened (no periodic probe), [Colour them now] and [Undo]; per site, data dir |
| install.sh label/DEST/file list/plist/deploy path/page workflow | updated (§E): derived ExitTimeOut, AbandonProcessGroup, pre-validation, prev/ + --rollback with VERSION, watchdog first + started.json wait, settings agent last, one-shot launchd install job; Apply replaces the page workflow; `--site/--config/--keep-config/--force-site-files`; site-owned files never overwritten; the editing code made by the settings agent; phase rule (§C.7) |
| NA Ingest key, Automation permission, keychain, DVS helper on Ingest, Companion app checklist, connection labels | Ingest card (§B) + Site setup > Companion; one Automation grant (watchdog); Make the login key |
| windows-setup.ps1 $Key/$IngestIPs/app list/task names/elevation/sshd/firewall/rtpMIDI/WoL/DefaultShell/manual steps/RESULT | template from `site.ingest_ips` + `runs` (§B), regex-checked (no `!`/`^`), deployed to the data dir; RESULT paste; elevation logic unchanged; DefaultShell set to cmd.exe (the user OKs); Get-CimInstance probes; by hand on the PC |
| mac-authorize.sh key, mac-dvs-helper.sh, install-solo-mode.sh, solo-mode-agent.swift paths/menu titles/bundle id/Accessibility | templates in the data dir incl. the swift template with `{{SOLO_DIR}}`, `{{PT_BUNDLE_ID}}`, `{{SOLO_MENU_PATH}}`; Accessibility light and re-grant note; test 17; by hand on that Mac |
| tascam-quiet | built module in `companion-objects/` (deployed to the data dir); module light on Site setup > Companion; imported by hand in Companion |
| .gitignore / one repo one site; data model gaps (site name, per-host inventory, duplicated values) | `sites/<id>/` (Phase 6) with the self record in the template; `site` block; `runs/networks/manual_done/gear`; single owners with generated checks and fixes (§A, builders, templates); site bundle export instead of a terminal step |
**Lens 3: what runs where**
| Item | Handled where |
|---|---|
| MacBook project folder, Gitea remote, checklist-ingest alias, memory files, gitea keychain item | DNE (dev side); scp staging (code only) + Export site bundle; `checklist-<id>` alias in the bootstrap list; no git credential on Ingest |
| NA Ingest LaunchAgent (exit timeout), staging folder, data dir, python3 3.9 (3.13 telnetlib risk), logs, auto-fix logs, state, dante-map, atem-reference.json, config backups, ssh sockets, MIDI streams, ATEM watcher, Art-Net listener, Bonjour, local CoreMIDI sessions, local probes, Spotify osascript, Companion db read | Ingest card lights incl. the python-version self-check, launchd agent state with the derived exit timeout and started.json (§B); Logs; History; Where things are; nothing copied to the staging folder by Apply |
| Companion app/version, custom variables, flash trigger, page 19, page 20, nav keys on 97 pages (gap at 96), quiet Tascam module, module folders, Companion Satellite, Stream Deck surfaces | Site setup > Companion (version, variables, pages exist / ours, page-number buttons "coloured on N of M" counted when the section is opened with the gap listed, module id per connection, version only if trivial); surfaces not shown; Satellite on the Ingest card (`watched:false`); Companion set-up by hand; `companion-objects/` for a new campus |
| every Companion connection (HyperDeck_1-6, tascam, x32, ATEM, dante, Axient_1/2, ULXD-4, PSM_1-3, ableton, Propresenter-API, BibleTV, lighting-midi, OBS-Youtube, arena, RESI_Decoder, resi-studio-cues, VX1000, UHDJr) and unwatched ones (Govee x7, DiGiCo_OSC, m32, AbleSet, tcp-HyperDeck-1..6) | Site setup > Companion live list with module id, watched-by, module-mismatch warning, "Add a key for it"; the HyperDeck job offers only bmd-hyperdeck connections; Open question Q4 |
| Resi bridge, lobby captions, caption screen, ProdCom (start method unknown), ProdDash, Spotify.app, ATEM Software Control + midiEnabled, DVS on Ingest, Church Translate disabled, Google Drive, free disk | Ingest `runs` (`disabled` kind for Translate; Drive folder probe via `atem.source_folder`) + INGEST key checks; bridge "never by itself"; captions fix with two helpers; dvs_ingest with prodcom rendered inline; prodcom/proddash without restart_ok unless ticked |
| other Ingest agents (checklist runner, backups, Itsycal, Alfred, Signiant) **and anything installed later** | Ingest `runs` with `watched:false`; the inventory probe lists every agent, login item and listening port under the collapsed "Everything found on this computer" list on the card, raw and unfiltered; no Add/Ignore, no counts on Overview |
| en0 / 10.6.3.222 (interface undetermined -> Check this Mac's network ports shows it), en17 Dante, en20 Netgear | Ingest `networks` (the page binds loopback + the LAN ones, every private port when empty); `site.ingest_ips`; port picker (Network & Dante) |
| ProPresenter Mac: host, key, DVS helper+sudoers, DVS engine, ProPresenter app + API :1025, session NA ProP | Computers card + Setup (expected session beside live); builders read the app path; check builder "A program is running on a Mac" |
| Tracks Mac: host (down today), key/helper/sudoers, Ableton Live 11 Suite + last set, NA Tracks session + Bonjour, DVS | same; "Problem: Ableton stopped since 09:12" on the Map with keys APPS MIDI DANTE; app name typed once in `runs` and read by "Reopen Ableton with its last set" and the running check; the MIDI link job works with the typed fallback while it is down |
| NA Translation: host, key/helper/sudoers, Church Translate, DVS | same; `translate` = "Start a Mac program" (gentle), `translate_restart` = "Restart a Mac program" with "also stop everything it started" |
| Studio B: host, solo agent, solo-mode.json/.fix, Accessibility, Pro Tools (bundle id, menu path), MIDI peer | same; `site.solo_dir` shared by the check builder, the fix and the swift template (schema check on generated strings; probe-safe); Pro Tools parameters on the helper_solo record |
| Lighting PC: host (blocks ping), sshd, DefaultShell, firewall, keys, tasks Vista/Arena/rtpMIDI, rtpMIDIService, Public profile, Vista autoload, Art-Net source, rtpMIDI peer | card with `pingable:false`, remote-shell light, `manual_done` ticks, `artnet` run; "Start a Windows service (+ task)" builder for rtpMIDI; "A program is running on a Windows PC" check with conditional echo pairs, verified here in Phase 0; the full `\StatusPanel\` task list in the raw inventory |
| LiveStream PC: host, WoL MAC/broadcast, sshd/firewall/keys, Start OBS elevated, OBS streaming variable | card; WoL light; BIOS tick; the OBS streaming variable is not a live signal in v1 (only the built-in recording-deck signal); "OBS is streaming" is added as a built-in kind only if the Phase 1 spike finds the module variable and its on/off values |
| Bible TV PC: host (Wi-Fi, blocks ping), sshd/firewall/keys, Start BibleApp at logon, Wi-Fi power settings | card; `pingable:false`; Wi-Fi tick + note; the page is not served on that Wi-Fi network once the LAN port is named |
| devices: ATEM, HyperDecks, LED, Tascam, X32, Shure, SuperRack, Resi decoder/encoder/cloud, gateway, Netgear switches, Studio B PoE boxes | the device list under each Map row and the **Gear** filter (one row each: address, kind, where it is, keys that watch it, light) linked from key editors, the Dante map, the ATEM section and the Help card; recording decks feed the live-service guard; the Resi encoder's HTTP answer is deliberately not a live signal (a Resi live kind is offered only if the spike finds a module variable) |
| secrets & permissions: ssh key, known_hosts (stale host key: Forget button), Netgear login, sudoers x4, Accessibility, Automation (watchdog only), Remote Login (incl. Ingest for the MacBook alias), ARD, Windows admin/execution policy, MacBook keychain, alert_webhook, editing code | lights (§B); §D never stored, masked everywhere incl. every data-dir copy (0600), re-attached on restore; by hand on each computer; web hook masked on Site; the one code made in the GUI session and shown only on loopback; lockout; command edits ask for the code again |
| workflow: deploy, rebuild/import pages, nav keys, offline tests, --once, new ATEM file, Companion one-time setup | §C Apply; no preview run: the key editor's WHAT THIS KEY WILL SAY line shows what a key will say and validate builds both pages in memory; ATEM upload / pick-newest with diff; Companion lights; Install the staged folder (one-shot launchd job, validated, rollback-able, survives the agent restart) |
**Lens 4: operator workflow and Companion-side state**
| Item | Handled where |
|---|---|
| edit config.json in the repo, validate, tests, --once | replaced by Draft -> Review & Apply (§C); behaviour tests on the frozen fixture, site tests per site (§F); the key editor's WHAT THIS KEY WILL SAY line shows the effect without saving |
| scp to staging, checklist-ingest alias, install.sh, --remove | the config no longer travels by scp; install.sh for code with pre-validation, prev/, the site-owned-files rule + History; Start/Stop/Restart the watchdog on Overview only, with the launchd state (guarded, STOP typed; Start allowed while live when the agent is down); Install the staged folder under Site setup > Site |
| data dir, LaunchAgent, logs, runtime state | §B, Logs, Where things are; state persisted across restarts in one `state.json` (identity-keyed) |
| make_page, push_page status/restart, import by hand, safe-target rule, tRPC sequence, export fetch, pages 22/97 | Apply does them in the data dir after the success test; marker-only rule incl. the jump keys; foreign keys refused by position; verified import; build hashes seeded from the live exports; refusal with by-hand steps; mechanics DNE |
| nav_keys and nav_locations.json | Site setup > Companion count + Colour them now + Undo; export walk; data dir, never overwritten by install.sh; per site |
| git commit/push, Gitea credential, README, "the artifact", verify live | Export site bundle + commit on the MacBook; no Pull button and no git credential on Ingest; README trimmed, the key guide is the living document; Apply verify (started.json) + Overview live deck |
| replace ATEM restore file; quiet Tascam module | Site setup > ATEM upload or pick-newest with cap, diff, tick, dated history, live refusal, part of the applied set, never overwritten by install.sh; module light, `companion-objects/`, by hand in Companion |
| custom variables by hand; sp_state/sp_hb/sp_cmd mechanics; flash_tick; stale rule | Site setup > Companion, by hand; mechanics DNE; `test`/`resurvey` verbs added; page requests go through the command variable like holds do; the command variable cleared at start |
| page numbers/names, nav positions/settings, nav_templates, export pins, HOLD_MS, expressions/actions/notes markers | Keys tab header and chips (only home) and the Lens 1 rows; page change consequences handled (§C); nav templates per version; markers kept (help warns not to edit notes in Companion) |
| companion host/port, HTTP endpoints, connection status fields, DB addresses, fast/slow variable lists | Site setup > Companion (Advanced); DNE; the HyperDeck `status` variable of rec_sync decks is on the fast list for the live signal |
| each connection's role (HyperDecks, tascam, Dante list, Shure/PSM, x32, lighting-midi, Propresenter-API, ableton/BibleTV, ATEM + library, OBS-Youtube, RESI, NovaStar, arena) | check cards with live, module-filtered pickers (Keys); the Companion list; the Gear filter |
| fix kinds acting on Companion, surfaces/grid, tablet URL, API press, single-page import rule | DNE; grid editable; one-way tablet link in the header; Run-now buttons through the watchdog with identities, only outside a live service |
| service_windows + ALARMS, auto_fix window/list, poller periods, SSH identity/options, hosts table, Mac/Windows external setup, Public profile, Vista autoload, keychain, Spotify Automation, MIDI sessions local/remote, Ingest services, ATEM Software Control pref, hidden tiles, groups, restart keys, fix catalogue, network, alert_webhook, colours, cooldown/hold age, project folder, variables not persisted, full change sequence | covered by §A-§D; one schedule on the page; the one alarms sentence and the Silence alarms until midnight button (not guarded); the full change sequence collapses to: change -> Review & Apply -> done (refused while live with the draft kept; Emergency override), with the repo snapshot optional |
| History / Undo reachable from a clean state; pending work visible; vendor noise; two edit homes; four meanings of "key"; two meanings of "Settings" | History from Overview and its own menu entry, list and bodies open on the LAN; no pending work exists (a change refused while live stays a saved draft, said on Overview and in Review); raw inventory list on the card and no Overview count; fixes edited from the key, the catalogue under the "Set up once" divider; one owner screen per value with read-only links; editing code / login key / page-number buttons / connection; the page is "Status Panel", the section is "Site setup" |
| too many places; admin actions hidden under Help; three status vocabularies; contradictions about alarms; thresholds on main screens; hidden jobs; confusing tray; two watchdog homes; mixed confirmations; walkthrough behind the code | eight entries, the two set-up-once screens under a divider and readable by anyone; Help read-only, actions under Site setup > Site (Install the staged folder, Export site bundle); the one status legend printed once under the top bar's `?`; one alarms sentence; numbers under Advanced with defaults; three job buttons on Overview + More jobs; the labelled tray; watchdog buttons on Overview only; one confirmation style; the walkthrough needs no code because reads are open (the Training copy is deferred) |
| the user's four named asks: a new MIDI link (incl. a brand-new peer, or a Mac that is off), a different Spotify playlist (name follows everywhere), HyperDecks 2 and 4 only (both set to "must have a disk", generic TCP connections not offered), a new campus (own windows incl. overnight, own wording, reachable page before the LAN port is named, optional Restart page, optional ATEM) | the Overview jobs (first three, one click each) and the "New campus from this one" job (started from a site bundle you pick, exported from the source campus's page and copied over with the folder, or from any sites/<x>/ folder in the copied repo; `install.sh --site <id>` uses sites/<id>/ when present, else sites/_template/; D12) incl. bootstrap, the template's self record with `networks: []` and `companion-objects/` in the data dir; every copied value flagged "copied from <site> - check it" until touched |