Status Indicators — Companion STATUS page for NA Ingest

One Companion page (page 19, "19 Status") that is solid green when everything is fine and blinks red with the actual problem ("HD3 NO DISK", "LIGHT PC MIDI LOST") when something needs attention. The UP / page number / DOWN keys on every other page follow the same colour, so you see a problem from any page.

There are two parts:

• status_panel.py — a watchdog that runs on NA Ingest as the LaunchAgent org.waterschurch.status-panel. Every second it reads Companion's connection statuses and module variables. It also checks what Companion can't see: network MIDI peers, Bonjour MIDI sessions, pings, launchd services, disk space, the Bible TV app, and whether the Resi cloud is reachable. It writes the result into one Companion custom variable.
• status-page.companionconfig — the page. Every button just draws what the watchdog wrote and blinks on your existing flash_tick trigger.

Layout (Stream Deck XL; MK.2 and Mini show the top-left corner)

	col 0	col 1	col 2	col 3	col 4	col 5	col 6	col 7
row 0	UP	ALL	REC	AUDIO	RF	SHOW	VIDEO	STREAM
row 1	page	HD1	HD2	HD3	HD4	HD5	HD6	TASCAM
row 2	DOWN	X32	DANTE	MIDI	APPS
row 3	ALARMS	ATEM	OBS	RESI	NET	INGEST	LED	ARENA

Row 0 holds the group buttons. Each one shows the first problem in its group, plus "+N" if there are more:

• REC: HD1–6 and the Tascam.
• AUDIO: X32 and Dante.
• RF: all six Shure units (AD4Q ×2, ULXD4, PSM1000 ×3).
• SHOW: MIDI and APPS.
• VIDEO: ATEM, LED and Arena.
• STREAM: OBS and Resi.

Colours

Colour	Meaning
Green	OK. The second line shows something useful, like 28h of record time or 19/19 Dante devices.
Blinking red	Something is wrong.
Solid dark red	Acknowledged. It stays red until the problem clears, then goes green. A new failure blinks again.
Amber	Warning: low disk, battery at 2 bars, RF interference, a device that keeps dropping out ("WING GONE x3"), a network blip in the last 3 minutes ("BLIP 10:21").
Dim red, not blinking	A fault while alarms are muted.
Grey	Not in use: an optional HyperDeck with no disk, or a disabled connection.
Purple	The watchdog has stopped. The master shows WATCHDOG DOWN and Companion's own error count.

Buttons

• Press any button: acknowledge its alarms (stops the blinking). Press ALL to acknowledge everything.
• Hold a device button for 2 seconds: restart that button's Companion connection. This is the fix for a connection stuck after a network blip. It only restarts Companion's side and never reboots the device.
• ALARMS: alarms are armed all the time (service_windows in config.json: every day, 00:00–23:59). Press ALARMS to mute the blinking, for example during maintenance. It re-arms itself at midnight.

Page keys on every page

The UP / page number / DOWN keys on every page follow the ALL button. They're green when everything is fine, flash red when something is wrong, and turn purple if the watchdog stops. They are Companion's own nav keys converted to regular buttons (nav_templates.json), so presses work exactly as before.

• python3 nav_keys.py applies them to every page. Run it again after adding pages.
• python3 nav_keys.py --revert puts Companion's built-in nav keys back.

What each button checks

Button	Checks
HD1–HD6	Companion connection; NO DISK; record time LOW (<30 min, amber) / FULL (<5 min); NOT REC when another deck is recording and this one isn't
	HD2 and HD4 are "optional": true in config.json: with no disk they sit grey with no alarm, and once a disk is in they are watched like the others. Add or remove "optional": true on any deck to change that.
TASCAM	connection; the recorder's own Error / Caution
X32, ATEM, ARENA, LED, OBS	Companion connection
DANTE	All 19 Dante devices ("19/19"), named when one drops: "WING GONE", "TRACKS GONE", "PROP MAC GONE".
MIDI	Lighting PC and SuperRack still joined to NA Ingest's network MIDI session; the Lighting PC and NA Tracks MIDI sessions still on the network. A clean disconnect shows within seconds; a crash or pulled cable takes about a minute (macOS's own timeout, measured).
APPS	ProPresenter API; Ableton OSC; tracks Mac pings; Bible TV app answering on :8080 (every 5 s); Bible TV OSC. Messages name the app ("PROPRES OFFLINE", "BIBLE APP DOWN"). Hold 2 s restarts the ProPresenter, Ableton and Bible TV Companion connections.
RESI	decoder (RossTalk); resi-studio-cues; Resi cloud reachable
NET	NA Ingest's link; gateway 10.1.0.1 pings; 4+ connections from 3+ groups dropping within 10 s (what happened Sunday 10/5 at 10:21)
INGEST	Resi bridge and lobby captions services running; ProdDash running; free disk
RF (group)	Shure connections; battery ≤1 bar red, 2 bars amber; RF interference (only while that transmitter is on); PSM RF muted

A problem has to last 3 seconds before it shows, so one bad reading doesn't trigger an alarm. Recovery shows immediately.

One-time Companion setup

1. Variables → Custom Variables: add three variables. Leave the start values empty and "persist" off:
   • sp_state
   • sp_hb
   • sp_cmd
2. Import / Export → Import: choose status-page.companionconfig (it's on NA Ingest's Desktop) and import it into page 19. Page 19 is empty today.
3. Make sure the flash_tick_500 trigger stays enabled. It's the blink clock.

Changing things

This folder is the source of truth. Edit config.json here, then copy the folder to NA Ingest:

scp status_panel.py coremidi.py config.json install.sh checklist-ingest:status-panel-install/

Then run the installer there. It backs up the old config and restarts the watchdog:

ssh checklist-ingest 'zsh ~/status-panel-install/install.sh'

If you add or move tiles, rebuild the page:

python3 make_page.py

Then import it into page 19, either by hand (Import/Export → Import) or with:

python3 push_page.py

push_page.py does the same import over Companion's own API. It refuses to overwrite page 19 unless that page is empty or already the status page.

Other commands:

python3 status_panel.py --once          # poll once and print every tile; writes nothing
python3 -m unittest discover -s tests   # offline tests, no Companion or gear needed
zsh install.sh --remove                 # stop the watchdog

The log is ~/Library/Logs/status-panel.log on NA Ingest. Every fault and clear is written there with a timestamp, which is handy for a post-service review.

Optional: see what the tracks send to the lights

The TRACKS tile can show when the tracks rig last sent MIDI ("TRK MIDI 3s"). For that, NA Ingest has to listen to the "NA Tracks" session. Use a separate session so nothing Companion sends can reach Ableton:

1. On NA Ingest, open Audio MIDI Setup → Window → Show MIDI Studio → Network (the globe icon).
2. Under My Sessions, press +, rename the new session to NA Ingest Listen, and tick it to enable it.
3. With NA Ingest Listen selected, choose NA Tracks in the Directory and press Connect.

The watchdog picks it up within about 10 seconds. Until then that line just stays hidden.

The LIGHT PC tile can do the same in the other direction. If the lighting software can send a heartbeat note back to NA Ingest every few seconds, set "max_silence_seconds": 10 on the LIGHT PC midi_rx check. The tile then goes amber if the lighting PC stops answering during service.

Files

File	Purpose
status_panel.py	the watchdog
coremidi.py	reads network MIDI sessions and listens to MIDI sources (ctypes, no extra installs)
config.json	tiles, positions, checks, service windows
make_page.py	builds status-page.companionconfig from config.json
install.sh	installs or updates the LaunchAgent on NA Ingest
push_page.py	imports the status page into page 19 over Companion's API
nav_keys.py	colours (or restores) the page nav keys on every page
nav_templates.json	Companion's own nav key designs, coloured by make_page.py
tests/	offline tests, including a fake Companion and a MIDI loopback