The deploy was the one pipeline with no failure notification. It either worked or
you found out by noticing the site was stale — the same shape as the off-site
backup before today.
alert.js now takes CHECKLIST_ENV_FILE and CHECKLIST_NODE_MODULES, because the
deploy job runs in a container that has the repo but no server/.env and no
node_modules; both live on the NAS at the mounted deploy directory.
What this still does not catch, and the reason it is written down in the workflow:
a runner that wedges. Today the NAS runner claimed run 7, executed none of its six
steps, and Gitea timed it out fifteen minutes later — every step was marked failed
without one having run, so no in-job step could have reported it. The container
said "Up 2 hours" throughout and admitted on restart that it had been holding the
job. A stuck runner looks exactly like a slow one, and catching it needs something
watching from outside the run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A push fires the web deploy and the app build at the same instant, and on commit
b03fb25 the second SSH session to Gitea came back with "Failed to update public
key" 32 seconds in. The pull succeeded by hand immediately afterwards, so it was
contention between the two runs rather than anything about the code — but the job
had already failed and emailed "App build FAILED".
The pull is now attempted three times with a pause between, and only gives up
after the third. It is the one step here that depends on another machine being
responsive at a moment we do not control, so it is the one worth retrying.
The point is not the build: a false failure alert is what teaches you to ignore
the alert, and an ignored alert is how the off-site backup sat dead for eight days
without anyone noticing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Run 4 built both apps successfully and then emailed to say the build had FAILED.
Two separate faults.
The notification step died on bash 3.2, which macOS still ships as /bin/bash. It
mis-parses a heredoc inside command substitution when the text contains an
apostrophe: the word "anyone's" was enough to produce "unexpected EOF while
looking for matching '". Reproduced on the ingest Mac before and after the fix.
The body is written to a file first, which avoids the construct entirely.
The failure alert was then sent because it keyed off failure() for the whole job,
so a bug in the success notification presented as a broken build. It is now scoped
to steps.build.outcome, so it reports only what it claims to report.
An alert that fires when nothing is wrong trains you to ignore it, which costs more
than the missing alert it replaced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The alert steps passed their message body as a bare multi-line argument starting
at column 0. That ends the YAML block scalar, which makes the whole file invalid:
"a multiline key may not be an implicit key". Gitea still listed the workflow as
active and simply never created a run for it — no error anywhere, which is why two
pushes appeared to do nothing.
The bodies are built with indented heredocs now, so every line stays inside the
block scalar. Validated by parsing the file rather than by eye; a workflow that is
registered but silently never triggers looks exactly like one that works.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The web app has deployed on push since earlier today, but the phone apps carry
their own frozen copy of the UI, so a change to server/public silently left them
behind until someone remembered to cut a release. Remembering was the weak part.
A second Actions runner now runs on the ingest Mac — in host mode, because an app
build needs Xcode, the Android SDK and the signing keys, none of which exist in
the NAS runner's Linux container. A push touching server/public, mobile/src or
the Capacitor config rebuilds and signs both apps, then emails to say the shipped
apps are behind, or that the build broke.
It deliberately stops before uploading. An upload needs a build number that has
never been used, Play refuses to reuse or lower a versionCode, and bumping it
means committing the new number back from the build machine on every run. That
needs write access from that machine and permanently changes the numbering
scheme — a one-way door, so shipping stays an explicit ./release.sh.
release.config.json is excluded from the trigger paths so that bumping a build
number by hand cannot set off a rebuild of what was just built.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
act_runner already mounts /var/run/docker.sock into every job container, so
declaring it again in the workflow made the daemon refuse to create the
container at all: "Duplicate mount point: /var/run/docker.sock". The deploy
directory still has to be declared, and still at its own host path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Until now a server change reached the NAS only when someone ran ./release.sh --web
from a machine on the church network. That made the deploy depend on which laptop
was awake and where it was, which is the same fragility that let the nightly Drive
backup go stale unnoticed.
An act_runner now runs in Docker on the NAS itself and picks up this workflow, so
a push to main rebuilds the container regardless of where the push came from.
The workflow keeps the protections the old remote deploy script had, because they
are the ones that matter: server/data and server/.env are excluded from the copy
and then explicitly checked for afterwards, and the container is recreated with
'up -d --build' rather than 'down' first, so an interrupted deploy leaves the old
container serving instead of tearing the stack down. rsync runs without --delete,
matching the tarball deploy it replaces — a stale file is survivable, a wiped
data/ is not.
This covers the web app only. The iOS and Android apps bundle a copy of the UI at
build time, so they still need ./release.sh and a store release; server-side
changes reach them immediately, since they call this same server live.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
