christianandClaude Opus 5
0db5e9b07d
First commit on the local Gitea server, made after moving the tree off the Windows machine. Build outputs, node_modules and the AI models stay out (see .gitignore); everything the installer build needs is tracked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
77 lines
4.1 KiB
Markdown
77 lines
4.1 KiB
Markdown
# Interlinear re-conversion (STEPBible TAHOT → `interlinear/*.json`)
|
||
|
||
These scripts exist because the ORIGINAL converter did not. It was a one-off, was never kept, and
|
||
its reference regex silently dropped 2,083 OT verses — 96 whole chapters, including 62 of the 150
|
||
Psalms — leaving the word-popup card blank on all of them for months. Keep these files so the data
|
||
can be regenerated and audited instead of re-derived from scratch.
|
||
|
||
`tools/` is excluded from `build.files`, so nothing here ships in the installer.
|
||
|
||
## Rebuilding the data
|
||
|
||
```bash
|
||
# 1. Fetch the four TAHOT OT files (~67 MB) into ./tahot/
|
||
# https://github.com/STEPBible/STEPBible-Data -> "Translators Amalgamated OT+NT"
|
||
# TAHOT Gen-Deu / Jos-Est / Job-Sng / Isa-Mal (CC BY 4.0, Tyndale House Cambridge)
|
||
|
||
# 2. Sanity-check the converter against the data already shipping.
|
||
# It must reproduce existing verses almost exactly before you trust it on missing ones.
|
||
node tools/interlinear/tahot-convert.cjs tools/interlinear/tahot/TAHOT_Gen-Deu.txt . Genesis
|
||
|
||
# 3. Dry-run, then merge (additive: never changes a verse that already exists).
|
||
node tools/interlinear/merge.cjs . "" --dry
|
||
node tools/interlinear/merge.cjs . ../bible-app-interlinear-backup-<date>
|
||
|
||
# 4. Rebuild the usage stats from the merged corpus.
|
||
node tools/interlinear/gen-stats.cjs interlinear compare # measure against the current file
|
||
node tools/interlinear/gen-stats.cjs interlinear write
|
||
```
|
||
|
||
## The traps, in the order they bite
|
||
|
||
1. **The reference parenthetical — this was the bug.** TAHOT writes `Gen.32.1(32.2)`: the English
|
||
reference (NRSV numbering) with the Hebrew reference in brackets, wherever the two traditions
|
||
number verses differently. A regex anchored on `Book.ch.v#word` rejects it. Every verse in such
|
||
a chapter carries the bracket, so whole chapters disappear — and because scattered single verses
|
||
carry it too (`Gen.31.55(32.1)`), a whole-chapter audit alone will not find them all. Key on the
|
||
ENGLISH reference; the app's `chapter|verse` keys are English.
|
||
|
||
2. **Punctuation.** `/` separates prefixes/suffixes, `\` separates punctuation. Drop a **trailing**
|
||
`\׃` (sof pasuq) and nothing else: the maqqef `־` and paseq `׀` stay, and a sof pasuq *followed
|
||
by* a paragraph marker also stays (`אֶחָֽד\׃\ \פ` → `אֶחָֽד׃ פ`, 90 such words in Genesis alone).
|
||
|
||
3. **Word numbers are two digits, plus two more for a sub-position.** `#0501` is "word 5, sub 1" —
|
||
the LXX-restored words that sit *between* words 5 and 6 in Gen 4:8. Parsing it as the number 501
|
||
sorts them to the end of the verse.
|
||
|
||
4. **Strong's numbers lose their disambiguation letter.** `H7225G` → `H7225`; the app's dictionaries
|
||
are keyed on the plain 4-digit form.
|
||
|
||
5. **Do not "fix" the 36 verses that disagree.** A fresh conversion differs from the shipped data on
|
||
36 verses: Qere/Ketiv variant picks, empty-`w` placeholders (14 rows across all of TAHOT, all
|
||
`Q(K)` with no Strong's), and the verse-boundary cases TAHOT's own header lists (Num.26.1,
|
||
1Sa.21.1, 1Ki.18.33, 20.3, 22.22, 22.43, 1Ch.12.4, Isa.64.1). `merge.cjs` refuses to write if any
|
||
existing verse would change — leave that guard in.
|
||
|
||
6. **Psalm titles arrive as verse 0** (116 keys). Harmless: the app never requests verse 0.
|
||
|
||
## Field mapping
|
||
|
||
| app | TAHOT column | transform |
|
||
|---|---|---|
|
||
| `w` | 2 Hebrew | drop trailing `\׃`, then remove `/` and `\` |
|
||
| `tr` | 3 transliteration | remove `/` |
|
||
| `g` | 4 English | `/` → space, collapse |
|
||
| `s` | 5 dStrongs | the `{...}` root, disambiguation letter dropped, padded to 4 digits |
|
||
| `m` | 6 grammar | verbatim |
|
||
| `l`, `lg` | 12 expanded tags | lemma + gloss from `{...}`; strip leading `: `, `»` → ` » `, `_` → space |
|
||
|
||
`kjv` is **NT-only** (from TAGNT) — no OT word carries it.
|
||
|
||
## Verifying
|
||
|
||
- `node src/coverage.js` in `../bible-lexicon-export` — expects zero missing chapters.
|
||
- Against the app's own `bible-structure.js`, 31,069 of 31,078 verses carry tagged words (99.97%).
|
||
The 9 short ones are all NT versification differences (John 7:53, Acts 19:41, Rom 16:25-27,
|
||
2Co 13:13-14, Php 1:29-30) — TAGNT vs the NLT-derived structure, not gaps.
|