tepbc-ross-retyped/CONTINUATION.md

# CONTINUATION — Ross 1988 re-typeset

## State (end of session 5, 2026-09-15)
Edition complete and verified: **455 output pages**, `make verify` clean — 431/431, 0 overfull,
0 underfull *font* warnings, `reprocheck` 0 tokens short. 84 underfull boxes remain, the documented
cost of ragged right without hyphenation. Git is the history; the PDF is not tracked (`make` remakes it).

Changed this session:
- **Text face is XCharter**, replacing TeX Gyre ScholaX (Sammay). Chosen on metrics: ScholaX had the
  smallest lowercase relative to its capitals of the faces tested, x-height/cap 0.645 vs 0.717, so it
  read as decorative. Bengali needed no manual scale — `Scale=MatchLowercase` re-derived Tiro Bangla
  and the measured Bengali-to-Latin ratio held (2.12 → 2.08). Page count 470 → 455; underfull boxes
  123 → 84. **The measure is now ~73 characters a line against the ~70 that 5.95in was chosen for —
  an open decision, see below.**
- **Small caps removed entirely.** A typescript cannot set them, so the source defines none.
- **Clickable regions marked**: 0.4pt hairline rule, RGB(95,125,165), on Contents/List-of-Plates page
  numbers and resolved cross-references. Endnote superscripts and whole-line entries stay unmarked.
  `polish.py` emits `\xref` for the marked kind and `\hyperlink` for the unmarked.
- **Nine diacritic corrections**, all verified against the page image at 500–900 dpi. Full record and
  the ~30 still unchecked are in QUESTIONS.md.
- The edition's one editorial `[sic]` was removed: **it now interpolates nothing into the author's prose.**
- Plates inside quotations are full-measure again (`\platefullmeasure`); plate 1's `\plx` entry is in
  `build_maps()`, so `pl. 1` links at last.

### Next, in the order I would take them
1. **Link audit.** CLAUDE.md requires re-auditing the cross-reference rule after any change to it, and
   `polish.py` changed twice this session (`\xref`, and `\plx` added to `build_maps`). Print every link
   with its context and read them. Never done; nothing else validates that a link lands on the right page.
2. **The measure** — hold ~70 characters (about 5.7in) or accept 73. One line in `geometry`.
3. **Diacritic sweep remainder**, ~30 occurrences listed by page in QUESTIONS.md. Last five batches
   found nothing, so the yield looks low; unchecked is not cleared.
4. **Quotation slips** — 10 checkable in a library, 5 in archives nobody outside them can reach.

### Earlier state
## State (end of session 4, 2026-09-14)
- **All 431 PDF pages transcribed.** No `\unsure` readings outstanding.
- Chapter 5 `Vincent Figgins and Richard Watts` closed at orig. p.211 (PDF 213, notes 1–93, plates 89–96).
- Chapter 6 `European Typefoundries` orig. pp.214–240 (PDF 216–242, notes 1–81, plates 97–105).
- Section C `Indigenous Ventures` title at orig. p.241 (PDF 243).
- Chapter 7 `The Girīśa Vidyāratna Press and later foundry types.` orig. pp.242–263 (PDF 244–265,
  notes 1–54, plates 106–117).
- **Part II** `Mechanical Typefounding and Composition of Bengali` title at orig. p.266 (PDF 268).
- Chapter 8 `Linotype Composition` orig. pp.267–308 (PDF 269–310, notes 1–124, plates 118–133).
- Chapter 9 `Monotype Composition` orig. pp.309–337 (PDF 311–339, notes 1–54, plates 134–148).
- **Part III** `Photocomposition of the Bengali Script` title at orig. p.338 (PDF 340).
- Chapter 10 `Filmsetting` orig. pp.339–363 (PDF 341–365, notes 1–74, plates 149–160).
- Chapter 11 `Digital Photocomposition` orig. pp.364–404 (PDF 366–406, notes 1–80, plates 161–174).
- `Conclusion` orig. pp.405–410 (PDF 407–412, notes 1–19).
- `Epilogue` orig. pp.411–418 (PDF 413–420, notes 1–14, plates 175–178).
- `Bibliography` orig. pp.419–429 (PDF 421–431), set with the new `biblist`/`bibgroup`/`bibhead`
  macros: a bold group heading, then one entry per paragraph with a 1.6em hanging indent, set solid.

### Earlier state (end of session 3, 2026-09-13)
- Building clean: **195 edition pages, 199/431 PDF pages done**. No `\unsure` readings outstanding.
- Done (PDF pages): 1 title, 2 colophon, 3 dropped, 4–16 front matter, 17–30 Introduction (orig. pp. 15–28,
  notes 1–26, plates 1–6), 31 Part I title, 32 Section A title, 33–74 chapter 1 `Charles Wilkins`
  (orig. pp. 31–72, notes 1–131, plates 7–25).
- **Errata section** at the end; seven entries. `\erratum` + `src/errata.tex` + `make check` keep text and list in step.
  PLAN.md's Editorial row records the policy; its Type row records the specimen rule.
- **Inline type specimens**: 214 cut so far into `plates/inline/`, set with `\ig` at native size. Workflow per page:
  **NB `work/lines.py`'s line N is usually the line *after* the one counted visually — if a sheet comes
  back with the wrong line, try the band one earlier.**
  `work/lines.py N` lists the text lines → `work/glyphs.py N y0,y1 …` makes a numbered contact sheet of the ink
  runs on chosen lines → `tools/crop_inline.py N name=INDEX[:left|right|first|last|mid|trimright|trimleft] …`
  cuts them. When the space between a word and the glyph is too noisy to register as a gap, measure the run's
  column profile and use the `INDEX+PX` form (e.g. `48+63`).
- **Watch for scanner bars in plate crops.** The `--rule` guard that stops engraved rules splitting a plate also
  let thin bars survive; `crop_plate.py` now sweeps any very dark thin run at an edge *after* autocropping
  (`strip_edge_bars`). Nine plates were re-cropped for this. If a plate looks like it has a black line down one
  side, that is the cause — re-run `make plate` with its recorded recipe.

- **Bengali policy (Sammay, 2026-09-13)**: modern script → Tiro Bangla text; a historical typeface →
  cut from the scan and set with `\ig`. Slips inside quoted matter are never corrected — mark them
  `\qslip{}` (logged to `qslips.log`) and list them in QUESTIONS.md for checking against the source.


## Typography and plates — reworked 2026-09-13/14 at Sammay's request
- **Type**: 11pt TeX Gyre ScholaX (Century Schoolbook, from `texlive-fonts-extra`, added to the Dockerfile
  for it) on a 5.2in measure ≈70 characters a line; leading 1.22; ragged right; hanging punctuation;
  `\frenchspacing`; widows/orphans forbidden; heads a step larger; endnotes smaller and tighter
  (`\notesection`); running head names the current chapter.
- **`tools/polish.py`** runs first in `build.sh`: `src/pages` (faithful transcription, never hand-edit the
  output) → `work/pages` (what is actually typeset). It applies ties, en-dashes for ranges, small caps for
  institutional acronyms, `\mbox` round specimens, and cross-reference links. See CLAUDE.md for the full
  contract, especially the internal-vs-external rule for `p. N` references and the instruction to re-audit
  the links after changing it.
- **Contents** and **List of Plates** entries are whole-line links to `page.N`.
- **Plates**: `crop_plate.py` deskews an out-of-square page (rule-centre measurement, profile-variance
  fallback) and sweeps the grey gutter shadow. Deskew is the only step that resamples. All 89 plates were
  re-cropped; a sample was checked and none lost content. If a plate looks tilted or has a dark band down
  one side, re-run `make plate N=… ARGS="<recipe from the manifest>"`.

## Known cosmetic issue
Three plate files (p0130, p0131, p0218) report an identical `Overfull \hbox (57.8pt)`; the rendered pages show
no visible damage and the plates sit inside the measure. Cause not yet identified — not the image width (the
images differ) and not the margin note (moving it outside the minipage changed nothing). The three share one
trait: their images are **wide** enough that `width=\textwidth` (not `height=0.74\textheight`) is the binding
constraint, so the minipage content is exactly the measure and anything else on the line tips it over.

## Open with Sammay — see QUESTIONS.md
Nothing outstanding. The quotation-slip table in QUESTIONS.md lists the 15 suspected slips inside quoted
matter, left as printed for checking against the quoted sources; it is a courtesy list, not a blocker.

## Next
1. **Transcription is complete.** What remains is finishing work, not conversion:
   read the built PDF end to end for damage the per-batch spot-checks could miss;
   settle the two open QUESTIONS items (p.295 *compliments*, p.348 a dropped *to*);
   move the QUESTIONS "adopted corrections" list into the colophon.
2. Expect more inline specimens — budget for the `lines.py` → `glyphs.py` → `crop_inline.py` loop per page.
3. Later polish: move the QUESTIONS "adopted corrections" list into the colophon; a plate-only page's running
   head shows the previous text page too ("orig. pp. 16–38") — acceptable, refine if wanted.

### Never kill a build mid-run without clearing `work/main.aux`
A full build is ~20 min at 420 pages, so the temptation is to stop one that is carrying a change you
are about to supersede. Don't, unless you then `rm -f work/main.aux work/main.toc work/main.out`.
`build.sh` does not clear them, so the next run reads back a `.aux` truncated mid-`\enotez@note` and
dies at `\begin{document}` with `File ended while scanning use of \enotez@note` / `Runaway argument?`
— an error that looks like an unbalanced brace in a page file and is not. Check
`tail -c 200 work/main.aux` first: a clean block-boundary size (e.g. exactly 172032 bytes) ending
mid-entry is the tell.

### Hyphenation — settled, do not re-open
The original never breaks a word at a line end. Checked over all 431 leaves: 59 line-ending hyphens,
every one a compound the author typed or OCR noise off a plate image (`work/lineend-hyphens.json`
holds the list). Output hyphenation is therefore off (`\lefthyphenmin=62`, `\hyphenpenalty=10000`),
with `\exhyphenpenalty` left low so a typed hyphen may still end a line as it does in the original.

### Unnumbered in-text figures — `\fig`
Chapters 7 and 8 set illustrations in the run of the text with **no plate number and no caption** (the
sentence before them carries the reference): orig. p.245's two cut/penned `ব` forms, orig. p.259's tier
diagram. `\fig{plates/fig-pNNNN.png}` (added 2026-09-14) centres one at native size, shrinking it only if
it is wider than the measure. Cut these by explicit pixel box with PIL — `crop_plate.py` is for whole-page
plates — and record the box in the manifest `notes` column.

### A printed page number inside the plate's own column range
Plate 119 (PDF 272) has the thesis's printed page number sitting in the same columns as the plate's lower
figure, so no `--box` can exclude it. Crop such a plate with PIL instead: paint the page number's rectangle
white, then take the ink bounding box. Record both steps in the manifest `notes` column.

### Framed plates need `--raw`
Chapter 6's plates are reproductions of whole framed pages (`plates 97–103`). The frame's own border rules sit
at the very edge of the crop, where `strip_edge_bars` reads them as scanner bars and sweeps them off — taking
content with them on p0222, where two lines were lost. `crop_plate.py --raw` (added 2026-09-14) takes an
explicit `--box` exactly: no shadow sweep, no deskew, no bar sweep, no autocrop. Measure the box first by
finding the frame's rules (columns/rows whose ink mean exceeds 0.5), then pass `--raw --box L,T,R,B` as
**fractions** of the raw page — `--box` is fractional, not pixels, and pixel values raise a PIL
DecompressionBombError. Every plate recipe is in the manifest `notes` column.

### Measuring the original's leading — autocorrelate, don't count bands
Counting gaps between ink bands gives nonsense: a line's ascender band and its x-height band are often
detected separately, halving the apparent pitch, and a page with a plate or a displayed extract poisons the
median. Pages came back at 40, 53, 70, 94 px for the same document. Autocorrelating the row-ink profile over
the text block and taking the peak in 35-140 px is robust: every ordinary page then returns 94-95 px at
300 dpi, and so does every specimen page. **The original is leaded 22.9 pt uniformly** — the thing to
reproduce. Its specimens reach 16.9 pt at the 90th percentile, 74% of the line, and only 0.7% crowd it.
Our build now measures 23.1 pt on a plain page and 23.1 pt on a specimen page (r=0.85 / 0.41; the low r is
a seven-line block, not unevenness).

### First-line indent and the ragged2e trap
The original never indents: it separates paragraphs by a blank line, and being justified, that blank line
sits under a flush line and reads clearly. Reflowed and ragged right the cue is much weaker, so this edition
indents 1.6em and keeps `\parskip` small (Sammay, 2026-09-14). Setting `\parindent` alone is not enough —
ragged2e's `\RaggedRight` assigns `\parindent` from `\RaggedRightParindent` (0pt by default) every time it
is invoked, so set both, and remember that any `\RaggedRight` inside a macro re-zeroes the indent locally.

### Swallowed digits: a TeX integer assignment eats the text after it
Reported 2026-09-14: the Halhed shelfmarks on orig. p.420 came out as `- 5593`, `and 5596`, `A and 5660B`,
`E`, `F`, `A`, `B`. The transcription in `src/pages/p0422.tex` was correct and so was `work/pages/p0422.tex`;
the loss was in the typesetting. `biblist` set `\everypar{\hangindent=1.6em\hangafter=1}`, and `\hangafter`
is an integer parameter: TeX goes on scanning digits after the `1`, so a paragraph beginning `5590 - 5593`
assigned `\hangafter=15590` and the number was gone. Fixed by terminating the assignment with `\relax`.
**Any numeric assignment in an `\everypar`, a hook, or a macro that can be followed by body text needs
`\relax`.**

### One build at a time in `work/` — now enforced by a lock
Every pass runs in `work/`, so two concurrent builds share `main.aux`: the second reads the first's
half-written copy and dies at `\begin{document}`. It surfaces as **"Undefined control sequence"** or
**"File ended while scanning use of `\enotez@note`"**, and the reported line belongs to `main.aux`, not to
the file the message appears to name — chasing it through `src/main.tex` or the page files wastes an hour.
The tell is the log's file stack: the last file opened before the error is `(work/main.aux`.

`build.sh` now takes an atomic `mkdir work/.buildlock` and refuses to start rather than corrupting a running
build. Diagnosed wrongly twice before that (2026-09-14/15): first as a truncated aux left by an interrupted
run, then as an orphaned container. Neither. The cause both times was simply starting `make` again while an
earlier build was still going — a build takes several minutes and prints nothing until it ends, so a running
one is easy to forget. Check `docker ps` first.

### A group that ends before its own `\par` loses its leading
Twice now: the plate captions, and the last paragraph of `src/colophon.tex`. `\setstretch` (like `\small`)
is a paragraph-shape parameter read at the `\par` that ends the paragraph, so `{\setstretch{1.2} text}` or
`\begingroup\setstretch{1.2} ... \endgroup` sets the final paragraph at the *outer* leading. End the
paragraph explicitly — `\par` before the closing brace or `\endgroup`.

### `tools/reprocheck.py` — the reproduction check that found it
Bag comparison, not sequence: every token in `src/pages/*.tex` must reach the built PDF at least as often as
it appears in the source. Order-insensitive, because endnotes and the Contents move text about. Run it after
any build that changes the preamble:

    python3 tools/reprocheck.py [pdf]

It de-TeXes the page files (macro table in the script says which argument of each macro carries copy) and
compares against `pdftotext -layout`. Two extraction artifacts are reported separately as "merged into a
neighbour", not as losses: poppler glues an endnote superscript onto the word before it (`em.`+`55`), and it
drops the thin space between the two Bengali forms in a Scheme-of-Transliteration cell. `-layout` is required
— without it poppler rejoins a compound broken at its own hyphen (`Staats-Druckerei`), which this text does
constantly, and ~80 false positives appear. A clean run prints `0 tokens short`.

## Environment
Runs from this directory with Claude Code; tools via `make` (Docker). `CLAUDE.md` has the rules and macro table.
The Makefile exports `UID`/`GID` (zsh does not). `make check` and `make build` must both pass at the end of
every batch.

## Layout
```
ross-thesis/
  PLAN.md  CONTINUATION.md  QUESTIONS.md  manifest.tsv  build.sh
  src/preamble.tex  src/colophon.tex  src/errata.tex  src/main.tex (generated)  src/pages/pNNNN.tex
  tools/prep.sh  tools/ocr2tex.py  tools/crop_plate.py  tools/crop_inline.py  tools/check.py  tools/setrow.py  tools/reprocheck.py
  work/lines.py  work/glyphs.py (scratch helpers for the specimen workflow)
  fonts/TiroBangla-*.ttf   plates/pNNNN.png  plates/inline/pNNNN-*.png  plates/logo-proquest.png
  scans/ work/ (scratch; excluded from tarball)   older-versions/   ross-1988-retypeset.pdf
```