The deploy was the one pipeline with no failure notification. It either worked or you found out by noticing the site was stale — the same shape as the off-site backup before today. alert.js now takes CHECKLIST_ENV_FILE and CHECKLIST_NODE_MODULES, because the deploy job runs in a container that has the repo but no server/.env and no node_modules; both live on the NAS at the mounted deploy directory. What this still does not catch, and the reason it is written down in the workflow: a runner that wedges. Today the NAS runner claimed run 7, executed none of its six steps, and Gitea timed it out fifteen minutes later — every step was marked failed without one having run, so no in-job step could have reported it. The container said "Up 2 hours" throughout and admitted on restart that it had been holding the job. A stuck runner looks exactly like a slow one, and catching it needs something watching from outside the run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
116 lines
5.1 KiB
YAML
116 lines
5.1 KiB
YAML
# Push to main -> the NAS rebuilds itself.
|
|
#
|
|
# This replaces having to run ./release.sh --web by hand for a server change. It does the
|
|
# same two things that script's remote half did, in the same order and with the same
|
|
# protections: copy server/ over the live directory, then recreate the container in place.
|
|
#
|
|
# It runs on the "nas" runner (act_runner, in Docker on the NAS itself), so it does not
|
|
# care whether the laptop is awake, on the church network, or in the building.
|
|
#
|
|
# NOT covered here on purpose: the iOS and Android apps. They bundle a copy of the web UI
|
|
# at build time (see mobile/scripts/sync-web.js), so a UI change only reaches phones via a
|
|
# new store release. Server-side changes reach them immediately, because they call this
|
|
# same server live.
|
|
|
|
name: Deploy web to NAS
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
paths:
|
|
- 'server/**'
|
|
- '.gitea/workflows/deploy.yml'
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
deploy:
|
|
runs-on: nas
|
|
container:
|
|
image: node:20-alpine
|
|
volumes:
|
|
# The docker socket is deliberately NOT listed here: act_runner already mounts it
|
|
# into every job container, and declaring it again makes the daemon reject the
|
|
# container with "Duplicate mount point: /var/run/docker.sock".
|
|
#
|
|
# Mounted at the SAME path it has on the host. docker-compose.yml binds ./data
|
|
# relative to the compose file, and the daemon resolves that as a HOST path — so if
|
|
# this were mounted anywhere else the app would come up pointed at a directory that
|
|
# does not exist, with an empty data volume.
|
|
- /volume2/docker/recurring-checklist:/volume2/docker/recurring-checklist
|
|
|
|
steps:
|
|
- name: Install tools
|
|
run: apk add --no-cache docker-cli docker-cli-compose rsync
|
|
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Copy server files over the live directory
|
|
run: |
|
|
set -eu
|
|
DEST=/volume2/docker/recurring-checklist
|
|
test -f "$DEST/docker-compose.yml" || { echo "REFUSING: $DEST is not the deploy directory"; exit 1; }
|
|
|
|
# data/ holds the live checklists, uploads and backups; .env holds the admin and
|
|
# Gmail credentials. Neither is in git, and neither may ever be written here.
|
|
# No --delete: this mirrors the old tarball deploy, which only ever added and
|
|
# overwrote. A stale file left behind is survivable; a wiped data/ is not.
|
|
rsync -a \
|
|
--exclude 'data/' \
|
|
--exclude 'data-smoke/' \
|
|
--exclude '.env' \
|
|
--exclude 'node_modules/' \
|
|
server/ "$DEST/"
|
|
|
|
- name: Verify the live data and credentials survived
|
|
run: |
|
|
set -eu
|
|
DEST=/volume2/docker/recurring-checklist
|
|
test -f "$DEST/.env" || { echo "ABORT: .env is gone"; exit 1; }
|
|
test -d "$DEST/data" || { echo "ABORT: data/ is gone"; exit 1; }
|
|
echo ".env and data/ intact"
|
|
|
|
- name: Rebuild the container in place
|
|
# No 'down' first: that removes the container AND the network, so an interrupted
|
|
# deploy would leave the whole stack torn down. 'up -d --build' recreates only the
|
|
# app container, and the worst case on failure is the old one still serving.
|
|
run: docker compose -f /volume2/docker/recurring-checklist/docker-compose.yml up -d --build --remove-orphans
|
|
|
|
- name: Healthcheck
|
|
run: |
|
|
set -eu
|
|
for i in $(seq 1 20); do
|
|
S=$(docker inspect --format '{{.State.Health.Status}}' recurring-checklist 2>/dev/null || echo starting)
|
|
[ "$S" = "healthy" ] && { echo "container healthy"; exit 0; }
|
|
sleep 3
|
|
done
|
|
echo "container did not report healthy in time:"
|
|
docker ps --filter name=recurring-checklist --format '{{.Names}} {{.Status}}'
|
|
docker logs --tail 30 recurring-checklist
|
|
exit 1
|
|
|
|
- name: Tell Christian the deploy broke
|
|
# The web deploy had no failure notification at all until now, which is the same
|
|
# silent-failure shape as the backup: it either worked or you found out by looking.
|
|
#
|
|
# NOT covered by this: a runner that wedges. On 2026-09-24 the NAS runner claimed
|
|
# this job, ran none of its six steps, and Gitea timed it out — no step ran, so no
|
|
# step could report. Catching that needs something watching from outside the run.
|
|
if: failure()
|
|
run: |
|
|
D=/volume2/docker/recurring-checklist
|
|
BODY_FILE="$(mktemp)"
|
|
cat > "$BODY_FILE" <<MSG
|
|
A push to main failed to deploy the website to the NAS.
|
|
|
|
The running container was left alone — the deploy recreates it in place and only
|
|
after the files are copied, so the previous version should still be serving. Worth
|
|
confirming: https://check.watersmedia.org
|
|
|
|
Full log:
|
|
http://chronicles-ab.waterschurch.org:3000/christian/recurring-checklist/actions
|
|
MSG
|
|
CHECKLIST_ENV_FILE="$D/.env" CHECKLIST_NODE_MODULES="$D/node_modules" \
|
|
node scripts/alert.js "Website deploy FAILED" "$(cat "$BODY_FILE")" || true
|
|
rm -f "$BODY_FILE"
|