The previous commit moved the fetch to scp, which was a regression: modern scp
speaks SFTP, this Synology does not enable that subsystem, and every file failed
with "subsystem request failed on channel 0". It failed safe — nothing was
copied and nothing was emailed — but nothing would have copied again either.
Caught by running it rather than by reading it.
With key auth there is no pty in the way, so streaming the file through a plain
ssh session is byte-exact. Verified against the NAS's own byte count: 390163 in,
390163 out, parses with 43 lists. scp -O would also work, but the legacy protocol
is being retired from OpenSSH.
Worth recording what this proves about the original fault: today's file was never
corrupt. cat was never the problem. The pty expect wrapped around it was.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each nightly backup is ~275 KB and was pulled with `cat` through an expect
pseudo-terminal, then recovered by slicing from the first { to the last } of the
transcript. A pty does line discipline and has finite buffers — it is not a file
transfer — so captures arrived truncated often enough to fail the JSON check most
nights between 29 September and 8 October, emailing each time. No backup was ever
lost; every file copied on a later attempt.
The ingest Mac now has a key authorised on the NAS, so there is no password prompt
to automate and the file comes over scp, byte for byte. expect is gone from this
script, and so is the NAS password: nothing here reads the keychain any more,
which also removes the reason it could not be run by hand over SSH.
The JSON check stays, but it only verifies now — there is no transcript to repair.
A backup that is present but unreadable is still worse than one that is obviously
missing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The nightly backup has emailed "1 backup file(s) could not be copied" most nights
since 29 September. Nothing was wrong with the backups: every file that failed
that way copied cleanly on a later night, and the only one still outstanding is
today's. The alert was reporting a transport hiccup as if it were data loss.
The fetch is the fragile part. Each file is ~275 KB and arrives through an expect
pseudo-terminal, which does line discipline and has finite buffers, so a capture
occasionally comes back truncated and fails to parse. It is now retried three
times within the same run, which should turn most of these into a non-event.
Alert severity now depends on the file's AGE rather than on this run. Today's
backup failing is a retry tomorrow and is logged, not mailed. A file still missing
after STALE_DAYS means the retries are not recovering and that does raise mail,
naming the files and their ages.
A nightly email that never needed action is how an alert stops being read, which
is the exact failure this script was written to prevent — it was training its
reader to ignore it.
The real cure is to stop pulling files through a pty at all: with key-based SSH to
the NAS this could be a plain scp and be byte-exact. That needs a key installed on
the NAS and is left for a deliberate change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Under launchd, macOS allows a named path on ~/Library/CloudStorage to be stat'd
but blocks enumerating the directory. A glob therefore expands to nothing, and
the reporting block read that as zero: the first real LaunchAgent run after the
TCC fix logged "0 copies off-site" and emailed "no off-site copies at all"
while 57 files sat in the folder. The copy loop was unaffected, because it uses
filenames taken from the NAS rather than a glob.
Freshness is now derived by testing explicit filenames for the last 60 days,
which works in both contexts. Enumeration is still used for the total count, but
a count of zero is reported as unknown rather than alerted on — an unreadable
folder and an empty one are different things, and only one of them is an
emergency.
Preferred over granting Full Disk Access: a privacy grant is exactly what an OS
update silently reset here in the first place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The nightly copy to Drive had been dead since 15 September. DRIVE_DIR pointed at
Apps/Backups/Checklist Backups; the folder is Apps/Checklist Backups, with no
Backups level. The guard for a missing folder did its job and exited — into a
logfile nobody reads, so from the outside a broken backup and a healthy no-op
looked identical. Eight days were missing; this run copied them.
That is the second time the same failure has hidden in the same place. The
header comment already records the first: ten weeks stale in July, found in
September. Correcting the path alone would leave the mechanism that hid it
untouched, so:
- The Drive folder is probed rather than hard-coded, across every mounted
GoogleDrive-* account and both known folder layouts. The laptop mounts
christian@ where the Studio mounts media@, and the folder has moved once
already, so a literal path is the wrong thing to depend on.
- Every failure now emails as well as logs, via scripts/alert.js, which reuses
the server's own SMTP settings. A failed alert never masks the failure it was
reporting.
- A run that does not error but leaves the newest off-site copy more than
STALE_DAYS old now alerts too. That is the check that would have caught both
incidents; the error paths only fire when the script actually runs.
What this still cannot catch is the script never running at all — a sleeping
Mac or an unloaded LaunchAgent is silent by construction.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The project has lived only as a folder on the Mac Studio and, briefly, as a
transfer tarball. This is the first commit: the Express server and its web
front end, the Capacitor iOS and Android wrappers, and the release and backup
scripts.
The repo is the source, not a backup of the machine. Left out on purpose, by
the .gitignore files that were already in the tree: node_modules, build output,
mobile/www (regenerated from server/public on every build), and server/data,
which holds the live checklists and belongs on the NAS rather than in git.
Every credential is excluded too — server/.env, the Play upload keystore and
its properties file, the App Store Connect key and ios-signing.config. The one
secret that was not in an ignored file was the NAS SSH password, hard-coded in
build-deploy.sh and grepped back out of it by backup-to-drive.sh. Both now read
it from the login keychain (service checklist-nas, account Christian), falling
back to NAS_PASS in the environment, so rotating it is one keychain entry
rather than a history rewrite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
