bdeshiandClaude Opus 5
1979450c82
ScholaX read as too decorative (Sammay), and the font metrics agree: of the faces considered it sets the smallest lowercase relative to its capitals — x-height/cap 0.645 against XCharter's 0.717 — so its caps and ascenders dominate while the lowercase does the reading. Chosen on measured grounds, not taste alone. All 35 non-ASCII characters this text uses are covered; the dots-below transliteration set (ḍ ḥ ṃ ṅ ṇ Ṛ ṛ ṣ ṭ) disqualifies ETbb, Domitian, STEP and Tempora, which are otherwise reasonable book faces. Real small caps and oldstyle figures confirmed present, so the acronym setting is unaffected. XCharter ships with TeX Live, so no font is added. Erewhon was the runner-up until the metrics showed its x-height is smaller than the current face, which would have moved readability the wrong way. Bengali needed no manual adjustment: Scale=MatchLowercase re-derives Tiro Bangla from the main font's x-height, and the Bengali-to-Latin height ratio measured off the Scheme of Transliteration held at 2.12 -> 2.08 across the switch, within the ±1px noise of a 300 dpi read. 455 pages, down from 470. Underfull boxes 123 -> 84: XCharter's narrower set gives the line-breaker more room at the same measure. Zero overfull, zero font warnings, reprocheck 0 tokens short. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
150 lines
15 KiB
Markdown
150 lines
15 KiB
Markdown
# Ross 1988 re-typeset — working instructions for Claude Code
|
||
|
||
Goal: convert `source/10731406.pdf` (scan of Fiona Ross, *The Evolution of the Printed
|
||
Bengali Character from 1778 to 1978*, Ph.D., SOAS 1988; 431 PDF pages) into a searchable,
|
||
re-typeset PDF. Design decisions are final and recorded in `PLAN.md`; current state and the
|
||
next batch are in `CONTINUATION.md`; open readings for the owner (Sammay) in `QUESTIONS.md`.
|
||
Read those three files first, every session.
|
||
|
||
## Hard rules
|
||
1. **The scan's text layer is not a source.** Transcribe from the page image
|
||
(`scans/hi-NNN.jpg`, 130 dpi; re-rasterize at 300 dpi for anything small). The
|
||
tesseract text in `work/ocr/` is only a typing scaffold — proofread every word against
|
||
the image before it goes in a `.tex` file. Never `pdftotext` the source for content.
|
||
2. **Never guess.** A word, diacritic, number or glyph you cannot read with confidence is
|
||
written as `\unsure{best reading}` and added to `QUESTIONS.md` with PDF page, original
|
||
page and a description. Do not silently normalise or "correct" the author's text.
|
||
3. **Original spelling and typography stand**, including the author's inconsistencies.
|
||
Handwritten corrections in the scanned copy are adopted and logged in `QUESTIONS.md`.
|
||
4. **One file per original page**: `src/pages/pNNNN.tex`, NNNN = zero-padded **PDF** page.
|
||
Row in `manifest.tsv` updated in the same step (printed page, kind, status=done, src_file, notes).
|
||
5. **Bengali: modern script is text, historical typefaces are images.** Bengali that shows
|
||
the modern script — the Scheme of Transliteration on orig. p.13 — is typed as Unicode and
|
||
sets itself in Tiro Bangla. Bengali that shows a **historical typeface** (every specimen the
|
||
original pastes into its own sentences, the List-of-Plates glyphs, plate captions) is never
|
||
typed: cut it from the scan with `tools/crop_inline.py` and set it with `\ig`. The argument in
|
||
the text is about those shapes — retyping them destroys the evidence. (Sammay, 2026-09-13.)
|
||
6. **Plates keep native resolution**: crop with `make plate N=…`; never resample — the sole
|
||
exception is the automatic deskew, which rotates (and so resamples) a page that is measurably
|
||
out of square. The tool also sweeps scanner bars and the grey gutter shadow from the edges.
|
||
Tune per page
|
||
with `ARGS`: `--top/--bottom/--edge` (strip page number, caption, scanner edge), `--rotate 270`
|
||
for plates the original prints sideways (lossless), `--trim T,R,B,L` after rotation for skewed
|
||
scanner edges the heuristics cannot see; `--box L,T,R,B` gives the crop explicitly and skips the
|
||
bar/page-number heuristics altogether — needed for dark plates (photographed manuscripts,
|
||
photographic plates), where “dark means scanner bar” is false; `--raw` (with `--box`) takes the
|
||
box **exactly**, skipping the shadow sweep, deskew, bar sweep and autocrop — needed for a plate
|
||
that reproduces a whole **framed** page, whose own border rules the edge sweeps mistake for
|
||
scanner bars and strip along with content. `--box` values are fractions of the raw page, never
|
||
pixels. Record the recipe in the manifest `notes` column.
|
||
7. After each batch: `make check` must pass, then `make build` must succeed, and
|
||
`python3 tools/reprocheck.py` must print `0 tokens short` — it compares every token of
|
||
`src/pages/*.tex` against the built PDF and is the only check that catches content the
|
||
*typesetter* loses (see CONTINUATION.md for the `\hangafter` case it was written for). Do not leave
|
||
the tree in a state where either fails. `make verify` runs all three.
|
||
8. Do not edit `src/main.tex` (generated by `build.sh`). **Git is the version history**
|
||
(initialised 2026-09-14, Sammay): the build no longer archives the previous PDF, the PDF
|
||
is not tracked — `make` remakes it — and the 68 archived copies in `older-versions/` were
|
||
deleted. Superseded design drafts stay in `older-versions/` as text and are tracked.
|
||
9. Ask before changing anything in `PLAN.md`'s design table.
|
||
|
||
## Macro reference (`src/preamble.tex`)
|
||
| Macro | Use |
|
||
|---|---|
|
||
| `\chapstart[Short]{Title}` / `\chaphead[Short]{Title}` | Starts a chapter/section: new page, centred bold head, bookmark, note group "Notes to Title". **Must precede `\origpage` in the file.** Resets note numbering: next `\fn` is `\fn{1}`. The optional `[Short]` is the form used for the PDF outline, the running head and the note group, for a head the original sets on two lines — the Abstract repeats the thesis title above its own, and without it the bookmark reads as one glued string. |
|
||
| `\origpage{N}` | Original page N starts here (mid-sentence is normal). Exactly one per text page file, at the point of the break. |
|
||
| `\fn{n}{text}` | Footnote n of the original → endnote. n must be the original's number; `make check` enforces 1,2,3… per chapter. |
|
||
| `\plateop{N}{no}{plates/pNNNN.png}{caption}` | A plate that begins original page N (the usual case: one plate per page). |
|
||
| `\plate{no}{img}{caption}` | Plate not at a page start. |
|
||
| `\subhead{text}` | Italic run-in subheading (as the original's *Introduction:* etc.). |
|
||
| `\chapnum{Chapter 1}{Charles Wilkins}` | Chapter head as the original sets it: number over title, both centred bold; notes group as “Chapter 1: Charles Wilkins”. Precedes `\origpage`. |
|
||
| `\begin{extract}…\end{extract}` | A displayed quotation as the original sets them — indented both sides, set solid, no quotation marks. An environment, so a quotation broken by an original page break opens in one page file and closes in the next (`\origpage` then sits inside it). |
|
||
| `\ig{img}` | An inline type specimen cut from the scan by `tools/crop_inline.py`, set at native size (1 px = 1/300 in). Use wherever the original pastes typeforms into its own text — never retype those as Tiro Bangla, the argument is about their shapes. |
|
||
| `\pg{N}` | Hyperlink to original page N (used in Contents/List of Plates). |
|
||
| `\begin{biblist}…\end{biblist}`, `\bibgroup{…}`, `\bibhead{…}` | The Bibliography (orig. pp.419–429): the original sets it solid — a bold group heading (`\bibgroup`), then one entry per paragraph with the continuation lines indented. `biblist` gives that block (no paragraph skip, 1.6em hanging indent); `\bibhead` is a centred bold heading inside it. |
|
||
| `\fig{img}` | An **unnumbered** illustration the original sets in the run of the text (no plate number, no caption — the preceding sentence carries the reference). Centred, native size, shrunk only if wider than the measure. Crop it by explicit pixel box with PIL to `plates/fig-pNNNN.png` and record the box in the manifest `notes` column. |
|
||
| `\pl{no}{caption}{page}`, `\tocl{indent}{label}{title}{page}`, `\toclnp{indent}{label}{title}` | List-of-Plates / Contents lines. `\tocl` carries a page number and links the whole line to `page.N`; `\toclnp` is the no-page form for a front-matter entry whose number is set separately with `\pg`. |
|
||
| `\unsure{text}` | Doubtful reading (logged to `unsure.log`). |
|
||
| `\qslip{as printed}` | A suspected slip **inside quoted matter**. Never corrected — the slip may be the quoted source's, not Ross's. Sets the text unchanged, logs to `qslips.log`; list it in `QUESTIONS.md` with the source so it can be checked. |
|
||
| `\erratum{corrected}{as printed}` | A slip in the **original** that this edition corrects: sets the corrected reading, logs to `errata.log`. Arguments must be plain text (put `\emph` outside). Every use needs a matching `\erratumline{orig page}{as printed}{corrected}` in `src/errata.tex` — `make check` enforces it — and the pair prints in the Errata section. Handwritten corrections in the scan are *not* errata: adopt them silently and log in `QUESTIONS.md`. |
|
||
| `\partstart{Part I}{Title}` | Part-title page: label and title centred about two fifths down, bookmark, new note group. Precedes `\origpage` like `\chapstart`. |
|
||
| `\sectionstart{Section A}{Title}` | Section-title page — flush left, as the original sets them. |
|
||
| `\emph{}` for italics, `` ` '' `` for quotes, `\ldots\ ` for …, `\&`, `\%`. | Typewriter conventions of the original (no ligatures etc.) need no special handling. |
|
||
|
||
## Typography (set once in `src/preamble.tex` — do not re-litigate per page)
|
||
12pt **XCharter** (Carter's Charter, extended; loaded by path from the TeX tree, which fontconfig does
|
||
not index — note the upright face is `*-Roman`, not `*-Regular`) on a 5.95in measure, ~73 characters a line; Liberation Sans for margin marks and running
|
||
heads; Tiro Bangla for Bengali. **Leading 1.59 — a 22.9pt line, the original's own, measured off the scan
|
||
at 300 dpi; the inline specimens set it, not the text** (see CONTINUATION.md). Inline specimens at native
|
||
size (`\igscale` = 72.27/300), which is the 74% of the line the original gives them. First line indented
|
||
1.6em with `\parskip` 0.15em, `\frenchspacing`
|
||
(the original is a typescript with uniform spacing, and TeX's sentence spacing after `p.` is wrong here).
|
||
**Ragged right and unhyphenated** (`\RaggedRight`; `\lefthyphenmin=62` plus `\hyphenpenalty=10000` — the original typescript breaks only at spaces and at hyphens it actually typed, so `\exhyphenpenalty` stays low and `\tolerance`/`\emergencystretch` are raised to pay for the lost breaks). (Sammay, 2026-09-14.) **Hanging punctuation** via microtype protrusion,
|
||
dashes hanging far less. **Widows and orphans forbidden** with `\raggedbottom`. Running head: current
|
||
chapter (a `chap` mark class, set by every heading macro) on the left, original page range on the right.
|
||
Endnotes set a step smaller and tighter via `\notesection`; plate captions are `\footnotesize` with their own leading, a clear step below the text as in the original. Contents and List-of-Plates entries are
|
||
whole-line links to `page.N`.
|
||
|
||
## `tools/polish.py` — the build-time typographic pass
|
||
`build.sh` runs it first: it reads `src/pages/*.tex` and writes `work/pages/*.tex`, and the build inputs
|
||
the latter. **The transcription in `src/pages` always stays exactly as typed** — never hand-edit the
|
||
polished copies. The pass applies what the typewriter could not (Sammay, 2026-09-13):
|
||
* ties, so a reference or unit never breaks across a line (`p.~63`, `Part~II`, `240~lbs`)
|
||
* en-dashes for numeric ranges (1778--1978)
|
||
* small caps for institutional acronyms — an explicit whitelist in the script (IOL, BMS, OUP, SOAS,
|
||
MS, BL …). Fount codes (CW1, SB4, BM V, VF1) and Roman numerals are deliberately excluded
|
||
* `\mbox` round every inline specimen so it never breaks from adjacent punctuation
|
||
* **cross-reference links**: `pl. N` and `chapter N` always link (they can only be this thesis's own);
|
||
`p. N`/`pp. N` link **only where the thesis cites itself**. A page reference inside a citation
|
||
belongs to somebody else's book: the script tests the preceding clause for citation markers (a
|
||
title in `\emph`, `ibid.`, a shelfmark, publication data, a surname followed by a comma) before
|
||
accepting self-reference words (`above`, `below`, `see`, `mentioned`). Neither signal → no link; a
|
||
missing link is cheaper than one that lands on the wrong page. **Re-audit after changing this rule**:
|
||
print every link with its context and read them.
|
||
Protected from all of the above: comment lines, file paths, and the arguments of `\erratum`, `\qslip`,
|
||
`\ig`, `\origpage`, `\fn`, and the heading macros (their text becomes bookmarks and marks, and a link
|
||
inside them breaks the build).
|
||
|
||
## Inline type specimens — the workflow
|
||
For any page where the original pastes typeforms into its own text:
|
||
1. `docker compose run --rm -T tex python3 work/lines.py N` — lists the page's text lines with their
|
||
y-fractions. **Its line N is usually the line *after* the one you counted by eye; if a sheet comes
|
||
back with the wrong line, try the band one earlier.**
|
||
2. `… work/glyphs.py N y0,y1 [y0,y1 …]` — numbered contact sheet of the ink runs on those bands,
|
||
written to `work/sheetN.png`. Read it and pick the runs that are specimens.
|
||
3. `… tools/crop_inline.py N name=SPEC …` — cuts them to `plates/inline/pNNNN-name.png`.
|
||
SPEC is a run index plus, optionally, `:left|right|first|last|mid|trimright|trimleft` to split a
|
||
run that merged with its neighbour, `+PX` to start PX into the run, `~PX` to keep only its first
|
||
PX. Use `+PX`/`~PX` when the typewriter's space is too noisy to register as a gap.
|
||
4. Where a run defeats all of that (a word set solid inside a footnote), crop by explicit pixel box
|
||
with PIL and **record the box in the manifest `notes` column** so the cut is reproducible.
|
||
|
||
## Per-batch procedure (≈12–15 pages per batch)
|
||
1. `make prep A=<pdf> B=<pdf>` → `scans/hi-*.jpg` + `work/ocr/p*.txt`.
|
||
2. For each page, **look at the image**. Decide kind: `text`, `plate`, `mixed`, `blank`.
|
||
3. Text page: `python3 tools/ocr2tex.py <pdf> <printed> [--head "Chapter title"]` gives a
|
||
draft; then proofread against the image: italics (titles, transliterated words), `\fn{}`
|
||
marks placed exactly where the superscript sits, footnote text moved into `\fn{}`, all
|
||
diacritics (ā ī ū ṛ ṃ ṅ ñ ṭ ḍ ṇ ś ṣ ḥ), hyphenation joined, en-dashes kept as `-` as the
|
||
original types them. **The typescript never hyphenates a word at a line end** — verified across
|
||
all 431 leaves: every one of its 59 line-ending hyphens is either a compound the author typed
|
||
(`offset-lithography`, `Hof-und`, `forty-eight`) or OCR noise off a reproduced plate. So a hyphen
|
||
in a page file is always intrinsic to the word; never invent one, and never keep one that splits a
|
||
word. Output hyphenation is off to match (Sammay, 2026-09-14). Where the author compounds
|
||
inconsistently (`type-casting`/`typecasting`, `punch-cutter`/`punchcutter`) rule 3 applies: each
|
||
instance stands as printed. Page break inside a paragraph: the previous file ends mid-sentence
|
||
and this file begins `\origpage{N}` followed by the continuation — no blank line between
|
||
files (the build concatenates them), so do **not** start the file with `\par`.
|
||
4. Plate page: `make plate N=<pdf>` (inspect `plates/pNNNN.png`; if the caption or page
|
||
number survived or content was clipped, rerun with `ARGS="--top 0.05 --bottom 0.11"`
|
||
etc.), then a one-line file with `\plateop{...}` and the caption transcribed exactly.
|
||
5. Update `manifest.tsv` rows (`python3 tools/setrow.py <pdf> <printed> <kind> "notes"`); `make check`; `make build`; spot-check 2–3 output pages
|
||
(`pdftoppm -jpeg -r 60 -f X -l Y ross-1988-retypeset.pdf work/o`).
|
||
6. Update `CONTINUATION.md` (state, next batch) and `QUESTIONS.md`. Stop at a chapter or
|
||
batch boundary, never mid-file.
|
||
|
||
## Environment
|
||
Tools run in Docker: `make image` once, then the `make` targets above (`NODOCKER=1 make …`
|
||
if xelatex, poppler-utils, tesseract, python3-pil/numpy/pdfplumber are installed locally).
|
||
Fonts: Liberation (system), FreeSerif (system), Tiro Bangla (`fonts/`). Do not add fonts.
|