An update that goes wrong is no longer the end of it #11

2026-09-17T16:19:53-04:00

michael commented

2026-09-17 16:19:53 -04:00

Findings 1 and 2 from the update audit. Both live on the archive path — the one every shipped .app uses, since the bundle carries no .git.

1. Settings could be wiped, and admin silently unlocked

keep() protected config/modules.json and config/layouts/ by name, but not config/proddash.json — which holds adminPin, theme, forge, repo, branch and port.

On an install whose data directory is the app's own config/, an update wrote the repo's shipped defaults straight over it. Reproduced before the fix: set a PIN and a theme, update, and the file came back with adminPin gone, theme gone, pinSet: false, authRequired: false — admin open, no message.

That configuration happens when ~/Library/Application Support/ProdDash is not writable (resolveDataDir falls back silently) or PRODDASH_DATA_DIR points inside the app folder. Not the normal path, but nothing warns you when you are on it.

Fixed by refusing to write anything inside the data directory, wherever it sits — computed from DATA_DIR rather than a name list, so it keeps holding as files are added. The two legacy names stay for an install whose state never moved out.

2. A bad update was unrecoverable

No rollback. Worse, a fresh .app install would not help: seedWorkingCopy only replaces an older copy, and a broken one carries a higher version — so it declines. The real fix was deleting <dataDir>/app by hand, which nothing tells you.

Three changes, in the order they catch things:

The download has to parse first. node --check on the new server.js before anything is swapped in. A failure reports the syntax error and leaves the running copy untouched.
The replaced copy is set aside as app.previous, then dropped once the new version has stayed up ninety seconds — so that folder existing means "the last update has not proven itself yet", and never anything staler.
The launcher puts it back. It already gives up after four stops in two minutes; now it looks for app.previous first and rolls back, with a notification. Once only — if the old copy will not run either, the update was not the problem.

Verified

Against a stand-in for a shipped .app (no .git, exactly what build.sh embeds), with a local forge serving releases I controlled:


Broken release (`server.js` replaced with nonsense)	refused — `SyntaxError: Unexpected identifier 'is'`; `server.js` still 81700 bytes, staging cleaned, still serving 200
Good release, data dir inside the app	1.6.0 → 9.9.9 with `pin`, `theme` and `forge` all intact, admin still locked
Backup	`app.previous` held 1.6.0; after ~85s: "9.9.9 has stayed up — dropped the copy it replaced"
Rollback	compiled the launcher's `restorePrevious` into a harness against a real 9.9.9/1.6.0 pair: rolled back to 1.6.0, `.previous` consumed, no `.broken` left, settings survived, and a second call correctly found nothing to do

swiftc -typecheck passes on the launcher; node -c on the server. A git-checkout install is untouched by all of this — it still takes git pull --ff-only.

Worth knowing

The launcher half only reaches a machine when a new .app is installed there. The server half travels by the normal update, so AB and NA get finding 1 and the parse check as soon as they update; the automatic rollback waits for the next .app.

Findings 3–7 from the audit (stale native helpers, files never deleted, plain-HTTP unverified downloads, Node/shell drift, no "update all") are untouched.

🤖 Generated with Claude Code

Findings 1 and 2 from the update audit. Both live on the archive path — the one every shipped `.app` uses, since the bundle carries no `.git`. ## 1. Settings could be wiped, and admin silently unlocked `keep()` protected `config/modules.json` and `config/layouts/` by name, but not `config/proddash.json` — which holds `adminPin`, `theme`, `forge`, `repo`, `branch` and `port`. On an install whose data directory **is** the app's own `config/`, an update wrote the repo's shipped defaults straight over it. Reproduced before the fix: set a PIN and a theme, update, and the file came back with `adminPin` gone, `theme` gone, `pinSet: false`, `authRequired: false` — **admin open, no message**. That configuration happens when `~/Library/Application Support/ProdDash` is not writable (`resolveDataDir` falls back silently) or `PRODDASH_DATA_DIR` points inside the app folder. Not the normal path, but nothing warns you when you are on it. **Fixed** by refusing to write anything inside the data directory, wherever it sits — computed from `DATA_DIR` rather than a name list, so it keeps holding as files are added. The two legacy names stay for an install whose state never moved out. ## 2. A bad update was unrecoverable No rollback. Worse, a fresh `.app` install would not help: `seedWorkingCopy` only replaces an **older** copy, and a broken one carries a **higher** version — so it declines. The real fix was deleting `<dataDir>/app` by hand, which nothing tells you. Three changes, in the order they catch things: 1. **The download has to parse first.** `node --check` on the new `server.js` before anything is swapped in. A failure reports the syntax error and leaves the running copy untouched. 2. **The replaced copy is set aside** as `app.previous`, then dropped once the new version has stayed up ninety seconds — so that folder existing means *"the last update has not proven itself yet"*, and never anything staler. 3. **The launcher puts it back.** It already gives up after four stops in two minutes; now it looks for `app.previous` first and rolls back, with a notification. Once only — if the old copy will not run either, the update was not the problem. ## Verified Against a stand-in for a shipped `.app` (no `.git`, exactly what `build.sh` embeds), with a local forge serving releases I controlled: | | | |---|---| | Broken release (`server.js` replaced with nonsense) | refused — `SyntaxError: Unexpected identifier 'is'`; `server.js` still 81700 bytes, staging cleaned, still serving 200 | | Good release, data dir inside the app | 1.6.0 → 9.9.9 with `pin`, `theme` and `forge` **all intact**, admin still locked | | Backup | `app.previous` held 1.6.0; after ~85s: *"9.9.9 has stayed up — dropped the copy it replaced"* | | Rollback | compiled the launcher's `restorePrevious` into a harness against a real 9.9.9/1.6.0 pair: rolled back to 1.6.0, `.previous` consumed, no `.broken` left, settings survived, and a second call correctly found nothing to do | `swiftc -typecheck` passes on the launcher; `node -c` on the server. A git-checkout install is untouched by all of this — it still takes `git pull --ff-only`. ## Worth knowing The launcher half only reaches a machine when a new `.app` is installed there. The server half travels by the normal update, so AB and NA get finding 1 and the parse check as soon as they update; the automatic rollback waits for the next `.app`. Findings 3–7 from the audit (stale native helpers, files never deleted, plain-HTTP unverified downloads, Node/shell drift, no "update all") are untouched. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

michael added 1 commit 2026-09-17 16:19:53 -04:00

An update that goes wrong is no longer the end of it 12f6e22187

Two things the audit found, both about the archive path every shipped .app
uses.

**Settings could be wiped.** The update kept config/modules.json and
config/layouts/ by name, but not config/proddash.json — which holds the admin
PIN, the theme and the forge pointer. On an install whose data directory IS
the app's own config/ (an unwritable Application Support, or PRODDASH_DATA_DIR
aimed inside the app), an update wrote the repo's shipped defaults over it:
the theme went, and admin silently unlocked. Nothing inside the data
directory is written now, wherever it happens to sit, and the two legacy
names stay listed for an install whose state never moved out.

**A bad update was unrecoverable.** No rollback, and a fresh .app would not
help: seeding only replaces an *older* copy, and a broken one carries a
higher version — so the fix was deleting a folder nothing told you about.
Three changes, in the order they catch things:

- The download must parse before anything is swapped in. `node --check`
  against the new server.js, and a failure is reported with the syntax error
  and leaves the running copy untouched.
- Past that, the copy being replaced is set aside as `app.previous`, and
  dropped once the new version has stayed up ninety seconds — so that folder
  existing means "the last update has not proven itself yet", and never
  anything older.
- The launcher, which already gives up after four stops in two minutes, now
  looks for that folder before it does and puts the old copy back. Once: if
  the old one will not run either, the update was not the problem.

The launcher half only reaches a machine with a new .app; the server half
travels by the normal update.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

michael merged commit c075655817 into main

2026-09-17 16:30:44 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: michael/ProdDash#11