Files
Waters ChurchandClaude Opus 5 39c65cf9de
Build apps / build (push) Successful in 1m30s
Deploy web to NAS / deploy (push) Successful in 1m14s
Make the website deploy report its own failures
The deploy was the one pipeline with no failure notification. It either worked or
you found out by noticing the site was stale — the same shape as the off-site
backup before today.

alert.js now takes CHECKLIST_ENV_FILE and CHECKLIST_NODE_MODULES, because the
deploy job runs in a container that has the repo but no server/.env and no
node_modules; both live on the NAS at the mounted deploy directory.

What this still does not catch, and the reason it is written down in the workflow:
a runner that wedges. Today the NAS runner claimed run 7, executed none of its six
steps, and Gitea timed it out fifteen minutes later — every step was marked failed
without one having run, so no in-job step could have reported it. The container
said "Up 2 hours" throughout and admitted on restart that it had been holding the
job. A stuck runner looks exactly like a slow one, and catching it needs something
watching from outside the run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-24 16:11:22 -04:00

116 lines
5.1 KiB
YAML

# Push to main -> the NAS rebuilds itself.
#
# This replaces having to run ./release.sh --web by hand for a server change. It does the
# same two things that script's remote half did, in the same order and with the same
# protections: copy server/ over the live directory, then recreate the container in place.
#
# It runs on the "nas" runner (act_runner, in Docker on the NAS itself), so it does not
# care whether the laptop is awake, on the church network, or in the building.
#
# NOT covered here on purpose: the iOS and Android apps. They bundle a copy of the web UI
# at build time (see mobile/scripts/sync-web.js), so a UI change only reaches phones via a
# new store release. Server-side changes reach them immediately, because they call this
# same server live.
name: Deploy web to NAS
on:
push:
branches: [main]
paths:
- 'server/**'
- '.gitea/workflows/deploy.yml'
workflow_dispatch:
jobs:
deploy:
runs-on: nas
container:
image: node:20-alpine
volumes:
# The docker socket is deliberately NOT listed here: act_runner already mounts it
# into every job container, and declaring it again makes the daemon reject the
# container with "Duplicate mount point: /var/run/docker.sock".
#
# Mounted at the SAME path it has on the host. docker-compose.yml binds ./data
# relative to the compose file, and the daemon resolves that as a HOST path — so if
# this were mounted anywhere else the app would come up pointed at a directory that
# does not exist, with an empty data volume.
- /volume2/docker/recurring-checklist:/volume2/docker/recurring-checklist
steps:
- name: Install tools
run: apk add --no-cache docker-cli docker-cli-compose rsync
- name: Checkout
uses: actions/checkout@v4
- name: Copy server files over the live directory
run: |
set -eu
DEST=/volume2/docker/recurring-checklist
test -f "$DEST/docker-compose.yml" || { echo "REFUSING: $DEST is not the deploy directory"; exit 1; }
# data/ holds the live checklists, uploads and backups; .env holds the admin and
# Gmail credentials. Neither is in git, and neither may ever be written here.
# No --delete: this mirrors the old tarball deploy, which only ever added and
# overwrote. A stale file left behind is survivable; a wiped data/ is not.
rsync -a \
--exclude 'data/' \
--exclude 'data-smoke/' \
--exclude '.env' \
--exclude 'node_modules/' \
server/ "$DEST/"
- name: Verify the live data and credentials survived
run: |
set -eu
DEST=/volume2/docker/recurring-checklist
test -f "$DEST/.env" || { echo "ABORT: .env is gone"; exit 1; }
test -d "$DEST/data" || { echo "ABORT: data/ is gone"; exit 1; }
echo ".env and data/ intact"
- name: Rebuild the container in place
# No 'down' first: that removes the container AND the network, so an interrupted
# deploy would leave the whole stack torn down. 'up -d --build' recreates only the
# app container, and the worst case on failure is the old one still serving.
run: docker compose -f /volume2/docker/recurring-checklist/docker-compose.yml up -d --build --remove-orphans
- name: Healthcheck
run: |
set -eu
for i in $(seq 1 20); do
S=$(docker inspect --format '{{.State.Health.Status}}' recurring-checklist 2>/dev/null || echo starting)
[ "$S" = "healthy" ] && { echo "container healthy"; exit 0; }
sleep 3
done
echo "container did not report healthy in time:"
docker ps --filter name=recurring-checklist --format '{{.Names}} {{.Status}}'
docker logs --tail 30 recurring-checklist
exit 1
- name: Tell Christian the deploy broke
# The web deploy had no failure notification at all until now, which is the same
# silent-failure shape as the backup: it either worked or you found out by looking.
#
# NOT covered by this: a runner that wedges. On 2026-09-24 the NAS runner claimed
# this job, ran none of its six steps, and Gitea timed it out — no step ran, so no
# step could report. Catching that needs something watching from outside the run.
if: failure()
run: |
D=/volume2/docker/recurring-checklist
BODY_FILE="$(mktemp)"
cat > "$BODY_FILE" <<MSG
A push to main failed to deploy the website to the NAS.
The running container was left alone — the deploy recreates it in place and only
after the files are copied, so the previous version should still be serving. Worth
confirming: https://check.watersmedia.org
Full log:
http://chronicles-ab.waterschurch.org:3000/christian/recurring-checklist/actions
MSG
CHECKLIST_ENV_FILE="$D/.env" CHECKLIST_NODE_MODULES="$D/node_modules" \
node scripts/alert.js "Website deploy FAILED" "$(cat "$BODY_FILE")" || true
rm -f "$BODY_FILE"