christian/status-indicators
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restart page (23), force restarts, Tascam reconnect, ProdCom, Resi encoder

Browse Source 
- Page 23 "Restart": hold a key 2 s to restart DVS (4 Macs), ProPresenter,
  Ableton, Church Translate, ProdCom, ProdDash, captions, the bridge or the
  Tascam connection even when the status page says it is fine ("force",
  shared 60 s guard). RESTART PAGE key on page 19, STATUS PAGE back.
- Holds carry Companion's clock; ones queued while the watchdog was down
  are dropped instead of firing together later.
- TASCAM hold always reconnects and checks again after 20 s: HUNG - POWER
  CYCLE IT (pings, control port hung), NO NETWORK, or turns a disabled
  connection on. Tascam pinged at its Companion address.
- INGEST: ProdCom (:24480), ProdDash's page (:24500), the caption screen;
  DVS on Ingest restarts ProdCom afterwards. RESI: the encoder (10.1.10.202).
- Windows starts check the app really started (FIX FAILED otherwise); OBS's
  start task runs elevated (OBS is "Run as administrator"); setup script
  does that for any such app and gives sshd restart-on-failure.
- push_page.py checks what is on the target page, not its name (pages 22
  and 97 are named "PAGE" but hold buttons); --which restart.
- Companion custom variables are written in the request body (no URL limit).
Reviewed by a 3-lens review with per-finding verification; 7 confirmed
issues fixed. Tested live: OBS hold opens OBS; Restart page TASCAM key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
christian
2026-10-08 18:08:29 -04:00
co-authored by Claude Opus 5.5
parent 17347a5d9a
commit a0c72183bd
7 changed files with 544 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+26 -5
View File
@@ -13,7 +13,7 @@ There are two parts:
|--------|-------|-------|-------|-------|-------|-------|-------|-------|
| row 0 | UP | **ALL** | REC | AUDIO | RF | SHOW | VIDEO | STREAM |
| row 1 | page | HD1 | HD2 | HD3 | HD4 | HD5 | HD6 | TASCAM |
| row 2 | DOWN | X32 | DANTE | MIDI | APPS | VISTA | | |
| row 2 | DOWN | X32 | DANTE | MIDI | APPS | VISTA | RESTART PAGE | |
| row 3 | ALARMS | ATEM | OBS | RESI | NET | INGEST | LED | ARENA |
Row 0 holds the group buttons. Each one shows the first problem in its group, plus "+N" if there are more:
@@ -55,7 +55,8 @@ Row 0 holds the group buttons. Each one shows the first problem in its group, pl
| DANTE: PROP MAC / TRACKS / TRANSL / INGEST GONE | starts Dante Virtual Soundcard's engine on that Mac (`remote-access/mac-dvs-helper.sh`) |
| DANTE: any device powered by its switch port (today the four Studio B boxes) | power-cycles the port it is on **now**, found by its MAC on the Netgear Dante switches |
| DANTE: any other device GONE | restarts Companion's Dante module (hardware: if still gone, it is off or unplugged) |
| INGEST: BRIDGE / CAPTIONS / PRODDASH | restarts that service on NA Ingest |
| INGEST: BRIDGE / CAPTIONS / CAPTION SCREEN / PRODCOM / PRODDASH | restarts that one on NA Ingest |
| TASCAM (always, even when it looks fine) | reconnects Companion to the recorder and looks again after 20 s. **HUNG - POWER CYCLE IT**: the recorder answers ping but its control port has hung (only a power-cycle at the rack fixes that; it has no network reboot). **NO NETWORK / CHECK POWER**: it does not answer at all. |
| any button whose Companion connection is **disabled** | turns the connection back on (nothing else) |
| HD1–6 lost by Companion | asks the deck itself first: if it answers, Companion's connection is restarted; only a deck that is reachable but hung is rebooted. Never one that is recording, and never for NO DISK or NOT REC. The deck's IP comes from its Companion connection, so changing an IP in Companion needs no config change. |
| anything else with a Companion connection | restarts that Companion connection |
@@ -65,6 +66,22 @@ Row 0 holds the group buttons. Each one shows the first problem in its group, pl
It checks over SSH every 10 s that ProPresenter, Ableton and Church Translate are running, using NA Ingest's key `~/.ssh/id_status_panel`. That key was authorized on the Macs with `remote-access/mac-authorize.sh`.
- **ALARMS:** alarms are armed all the time (`service_windows` in `config.json`: every day, 00:00–23:59). Press ALARMS to mute the blinking, for example during maintenance. It re-arms itself at midnight.
## Restart page (page 23)
Tap **RESTART PAGE** on the status page (row 2). Each key restarts one thing **even when the status page says it is fine**; hold it 2 s. It shows RESTARTING, then DONE (or FAILED / what to do). The same thing will not run twice within a minute. **STATUS PAGE** (bottom left) goes back.
| Key | Restarts |
|---|---|
| DVS PROP / TRACKS / TRANSL / INGEST | Dante Virtual Soundcard's engine on that Mac (audio to and from it drops 5–10 s). DVS INGEST also restarts ProdCom afterwards, which otherwise keeps a dead input. |
| PROPRES | ProPresenter (quits and reopens the newest version) |
| ABLETON | Ableton on the Tracks Mac (quits it: unsaved set changes are lost, and Live may ask to recover - click it on the Tracks Mac) |
| TRANSLATE | Church Translate on NA Translation |
| PRODCOM | ProdCom on NA Ingest (lobby captions blank for 5–15 s) |
| PRODDASH / CAPTIONS / BRIDGE | that service on NA Ingest |
| TASCAM | reconnects Companion to the recorder (see TASCAM above) |
Add or move keys in `"restart_page"` in `config.json`, then `python3 make_page.py` and `python3 push_page.py --which restart`. `push_page.py` refuses any page that holds real buttons unless it already is that page.
## Page keys on every page
The **page number key** on every page follows the ALL button. It's green when everything is fine, flashes red when something is wrong, and turns purple if the watchdog stops. UP and DOWN stay normal. Which keys follow ALL is `"nav_keys"` in `config.json` (`["pagenum"]`; add `"pageup"` / `"pagedown"` to colour those too). The coloured key is Companion's own key converted to a regular button (`nav_templates.json`).
@@ -81,16 +98,16 @@ The **page number key** on every page follows the ALL button. It's green when ev
|---|---|
| HD1–HD6 | Companion connection; **NO DISK**; record time **LOW** (<30 min, amber) / **FULL** (<5 min); **NOT REC** when another deck is recording and this one isn't |
| | HD2 and HD4 are `"optional": true` in `config.json`: with no disk they sit grey with no alarm, and once a disk is in they are watched like the others. Add or remove `"optional": true` on any deck to change that. |
| TASCAM | connection; the recorder's own Error / Caution. Use the **1.3.1-quiet** Tascam module (see below) so Companion's log isn't flooded. |
| TASCAM | connection; the recorder's own Error / Caution. Use the **1.3.1-quiet** Tascam module (see below) so Companion's log isn't flooded.; pinged at the address on its Companion connection. |
| X32, ATEM, ARENA, OBS | Companion connection |
| LED | VX1000 and UHD Jr: Companion connection **and** each one answers ping (the NovaStar module keeps saying "Connected" after a processor is switched off). "UHDJR NO PING" = processor off or unreachable; "UHDJR OFFLINE" = Companion's connection failed; "UHDJR DISABLED" (amber) = connection turned off in Companion. |
| DANTE | All 19 Dante devices ("19/19"), named when one drops: "WING GONE", "TRACKS GONE", "PROP MAC GONE". |
| MIDI | All six network MIDI links on the FOH / Studio B checklist, each named when it drops: **LIGHT PC** (Ingest → Lighting), **SUPERRACK** (Ingest → Livebox), **FX LIGHTS** (Tracks → Lighting, the FX lighting cues), **TRK LIVEBOX** (Tracks → Livebox), **TRK STUDIO B** (Tracks → Studio B), **PROP LIGHTS** (ProPresenter → Lighting). NA Ingest's own session is read locally; NA Tracks' and NA ProP's over SSH every 5 s. Also: the Lighting PC and NA Tracks MIDI sessions still announced on the network. A clean disconnect shows within about 10 s; a crash or pulled cable takes about a minute (macOS's own timeout, measured). Hold 2 s reconnects whichever links dropped. |
| APPS | ProPresenter API (offline or error only: its harmless yellow warning, every 15–20 min, is ignored with `"ignore_warning": true`); Ableton OSC; tracks Mac pings; Bible TV app answering on :8080 (every 5 s); Bible TV OSC. Messages name the app ("PROPRES OFFLINE", "BIBLE APP DOWN"). Hold 2 s restarts the ProPresenter, Ableton and Bible TV Companion connections. |
| VISTA | Vista on the Lighting PC: its Art-Net poll (sent every 0.5 s) must be heard within 5 s, otherwise **SILENT** (Vista closed or crashed, or the Lighting PC is off the network). Hold 2 s starts Vista if it is not running; Vista then opens **Prop Takeover** from its show folder (`E:\media\Documents\Vista 3\Shows`) by itself, through its own autoload preference (`<autoloadpath>` in `E:\media\Documents\Vista 3\.preferences`, set 2026-10-08). Resolume is the ARENA button. |
| RESI | decoder (RossTalk connection and ping); resi-studio-cues; Resi cloud reachable |
| RESI | decoder (RossTalk connection and ping, 10.1.10.203); **encoder** (ping and its web page, 10.1.10.202, LAOEncoder); resi-studio-cues; Resi cloud reachable. Only shows they are on the network, not that the encoder is streaming. |
| NET | NA Ingest's link; gateway 10.1.0.1 pings; **4+ connections from 3+ groups dropping within 10 s** (what happened Sunday 10/5 at 10:21) |
| INGEST | Resi bridge and lobby captions services running; ProdDash running; free disk |
| INGEST | the bridge (makes ProPresenter's stage screen count down to the next Resi cue), lobby captions server and caption screen, **ProdCom** (transcription, answering on :24480), ProdDash (running and its page on :24500); free disk |
| RF (group) | Shure connections; battery ≤1 bar red, 2 bars amber; RF interference (only while that transmitter is on); PSM RF muted; each receiver pinged every 3 s ("AXIENT 1 NO PING" in about 6–9 s; Companion's Shure modules take 18–27 s to notice). The address is the one set on its Companion connection, read from Companion's settings every minute, so changing it there is enough; `"host"` in `config.json` is only the fallback. |
A problem has to last 3 seconds before it shows, so one bad reading doesn't trigger an alarm. Recovery shows immediately.
@@ -163,6 +180,10 @@ The stock Tascam DA-6400 module (1.3.0) polls the recorder 5 times a second, the
To go back, pick 1.3.0 on the connection again.
## Windows PCs
`remote-access/windows-setup.ps1` sets each PC up once (SSH for NA Ingest, start tasks, Wake-on-LAN). An app ticked "Run this program as an administrator" (OBS on the LiveStream PC) gets a start task that runs with highest privileges; otherwise Windows refuses to start it (error 0x800702E4). Windows restarts the SSH service and rtpMIDI by itself if they stop.
## Files
| File | Purpose |