tepbc-ross-retyped/CLAUDE.md

Ross 1988 re-typeset — working instructions for Claude Code

Goal: convert source/10731406.pdf (scan of Fiona Ross, The Evolution of the Printed Bengali Character from 1778 to 1978, Ph.D., SOAS 1988; 431 PDF pages) into a searchable, re-typeset PDF. Design decisions are final and recorded in PLAN.md; current state and the next batch are in CONTINUATION.md; open readings for the owner (Sammay) in QUESTIONS.md. Read those three files first, every session.

Hard rules

1. The scan's text layer is not a source. Transcribe from the page image (scans/hi-NNN.jpg, 130 dpi; re-rasterize at 300 dpi for anything small). The tesseract text in work/ocr/ is only a typing scaffold — proofread every word against the image before it goes in a .tex file. Never pdftotext the source for content.
2. Never guess. A word, diacritic, number or glyph you cannot read with confidence is written as \unsure{best reading} and added to QUESTIONS.md with PDF page, original page and a description. Do not silently normalise or "correct" the author's text.
3. Original spelling and typography stand, including the author's inconsistencies. Handwritten corrections in the scanned copy are adopted and logged in QUESTIONS.md.
4. One file per original page: src/pages/pNNNN.tex, NNNN = zero-padded PDF page. Row in manifest.tsv updated in the same step (printed page, kind, status=done, src_file, notes).
5. Bengali: modern script is text, historical typefaces are images. Bengali that shows the modern script — the Scheme of Transliteration on orig. p.13 — is typed as Unicode and sets itself in Tiro Bangla. Bengali that shows a historical typeface (every specimen the original pastes into its own sentences, the List-of-Plates glyphs, plate captions) is never typed: cut it from the scan with tools/crop_inline.py and set it with \ig. The argument in the text is about those shapes — retyping them destroys the evidence. (Sammay, 2026-09-13.)
6. Plates keep native resolution: crop with make plate N=…; never resample — the sole exception is the automatic deskew, which rotates (and so resamples) a page that is measurably out of square. The tool also sweeps scanner bars and the grey gutter shadow from the edges. Tune per page with ARGS: --top/--bottom/--edge (strip page number, caption, scanner edge), --rotate 270 for plates the original prints sideways (lossless), --trim T,R,B,L after rotation for skewed scanner edges the heuristics cannot see; --box L,T,R,B gives the crop explicitly and skips the bar/page-number heuristics altogether — needed for dark plates (photographed manuscripts, photographic plates), where “dark means scanner bar” is false; --raw (with --box) takes the box exactly, skipping the shadow sweep, deskew, bar sweep and autocrop — needed for a plate that reproduces a whole framed page, whose own border rules the edge sweeps mistake for scanner bars and strip along with content. --box values are fractions of the raw page, never pixels. Record the recipe in the manifest notes column.
7. After each batch: make check must pass, then make build must succeed, and python3 tools/reprocheck.py must print 0 tokens short — it compares every token of src/pages/*.tex against the built PDF and is the only check that catches content the typesetter loses (see CONTINUATION.md for the \hangafter case it was written for). Do not leave the tree in a state where either fails. make verify runs all three.
8. Do not edit src/main.tex (generated by build.sh). Git is the version history (initialised 2026-09-14, Sammay): the build no longer archives the previous PDF, the PDF is not tracked — make remakes it — and the 68 archived copies in older-versions/ were deleted. Superseded design drafts stay in older-versions/ as text and are tracked.
9. Ask before changing anything in PLAN.md's design table.

Macro reference (src/preamble.tex)

Macro	Use
\chapstart[Short]{Title} / \chaphead[Short]{Title}	Starts a chapter/section: new page, centred bold head, bookmark, note group "Notes to Title". Must precede \origpage in the file. Resets note numbering: next \fn is \fn{1}. The optional [Short] is the form used for the PDF outline, the running head and the note group, for a head the original sets on two lines — the Abstract repeats the thesis title above its own, and without it the bookmark reads as one glued string.
\origpage{N}	Original page N starts here (mid-sentence is normal). Exactly one per text page file, at the point of the break.
\fn{n}{text}	Footnote n of the original → endnote. n must be the original's number; make check enforces 1,2,3… per chapter.
\plateop{N}{no}{plates/pNNNN.png}{caption}	A plate that begins original page N (the usual case: one plate per page).
\plate{no}{img}{caption}	Plate not at a page start.
\subhead{text}	Italic run-in subheading (as the original's Introduction: etc.).
\chapnum{Chapter 1}{Charles Wilkins}	Chapter head as the original sets it: number over title, both centred bold; notes group as “Chapter 1: Charles Wilkins”. Precedes \origpage.
\begin{extract}…\end{extract}	A displayed quotation as the original sets them — indented both sides, set solid, no quotation marks. An environment, so a quotation broken by an original page break opens in one page file and closes in the next (\origpage then sits inside it).
\ig{img}	An inline type specimen cut from the scan by tools/crop_inline.py, set at native size (1 px = 1/300 in). Use wherever the original pastes typeforms into its own text — never retype those as Tiro Bangla, the argument is about their shapes.
\pg{N}	Hyperlink to original page N (used in Contents/List of Plates).
\begin{biblist}…\end{biblist}, \bibgroup{…}, \bibhead{…}	The Bibliography (orig. pp.419–429): the original sets it solid — a bold group heading (\bibgroup), then one entry per paragraph with the continuation lines indented. biblist gives that block (no paragraph skip, 1.6em hanging indent); \bibhead is a centred bold heading inside it.
\fig{img}	An unnumbered illustration the original sets in the run of the text (no plate number, no caption — the preceding sentence carries the reference). Centred, native size, shrunk only if wider than the measure. Crop it by explicit pixel box with PIL to plates/fig-pNNNN.png and record the box in the manifest notes column.
\pl{no}{caption}{page}, \tocl{indent}{label}{title}{page}, \toclnp{indent}{label}{title}	List-of-Plates / Contents lines. \tocl carries a page number and links the whole line to page.N; \toclnp is the no-page form for a front-matter entry whose number is set separately with \pg.
\unsure{text}	Doubtful reading (logged to unsure.log).
\qslip{as printed}	A suspected slip inside quoted matter. Never corrected — the slip may be the quoted source's, not Ross's. Sets the text unchanged, logs to qslips.log; list it in QUESTIONS.md with the source so it can be checked.
\erratum{corrected}{as printed}	A slip in the original that this edition corrects: sets the corrected reading, logs to errata.log. Arguments must be plain text (put \emph outside). Every use needs a matching \erratumline{orig page}{as printed}{corrected} in src/errata.tex — make check enforces it — and the pair prints in the Errata section. Handwritten corrections in the scan are not errata: adopt them silently and log in QUESTIONS.md.
\partstart{Part I}{Title}	Part-title page: label and title centred about two fifths down, bookmark, new note group. Precedes \origpage like \chapstart.
\sectionstart{Section A}{Title}	Section-title page — flush left, as the original sets them.
\emph{} for italics, ` '' for quotes, \ldots\ for …, \&, \%.	Typewriter conventions of the original (no ligatures etc.) need no special handling.

Typography (set once in src/preamble.tex — do not re-litigate per page)

13pt XCharter (Carter's Charter, extended; loaded by path from the TeX tree, which fontconfig does not index — note the upright face is *-Roman, not *-Regular; set via fontspec Scale=1.0833 over a 12pt class, which scales every size in the family but leaves \baselineskip class-derived so the leading stays put) on a 5.95in measure — 73 characters a line, measured off the built PDF; Liberation Sans for margin marks and running heads; Tiro Bangla for Bengali. Leading 1.59 — a 22.9pt line, the original's own, measured off the scan at 300 dpi; the inline specimens set it, not the text (see CONTINUATION.md). Inline specimens at native size (\igscale = 72.27/300), which is the 74% of the line the original gives them. First line indented 1.6em with \parskip 0.15em, \frenchspacing (the original is a typescript with uniform spacing, and TeX's sentence spacing after p. is wrong here). Ragged right and unhyphenated (\RaggedRight; \lefthyphenmin=62 plus \hyphenpenalty=10000 — the original typescript breaks only at spaces and at hyphens it actually typed, so \exhyphenpenalty stays low and \tolerance/\emergencystretch are raised to pay for the lost breaks). (Sammay, 2026-09-14.) Hanging punctuation via microtype protrusion, dashes hanging far less. Widows and orphans forbidden with \raggedbottom. Running head: current chapter (a chap mark class, set by every heading macro) on the left, original page range on the right. Endnotes set a step smaller and tighter via \notesection; plate captions are \footnotesize with their own leading, a clear step below the text as in the original. Contents and List-of-Plates entries are whole-line links to page.N.

tools/polish.py — the build-time typographic pass

build.sh runs it first: it reads src/pages/*.tex and writes work/pages/*.tex, and the build inputs the latter. The transcription in src/pages always stays exactly as typed — never hand-edit the polished copies. The pass applies what the typewriter could not (Sammay, 2026-09-13):

• ties, so a reference or unit never breaks across a line (p.~63, Part~II, 240~lbs)
• en-dashes for numeric ranges (1778--1978)
• no small caps. The pass small-capped a whitelist of institutional acronyms until 2026-09-15 (Sammay). Abbreviations are now plain uppercase throughout: the whitelist could only ever be partial, and on the Abbreviations page BFBS, LMS and MLCo stood unconverted beside small-capped BL, BMS, EIC, IOL, OUP, SOAS, SPG — it also split MS EUR 30 across two styles. Completing the list is the wrong fix: the original is a typescript, a typewriter cannot set small caps, so the source defines none and rule 3 leaves every acronym as the plain uppercase it prints
• \mbox round every inline specimen so it never breaks from adjacent punctuation
• cross-reference links: pl. N and chapter N always link (they can only be this thesis's own); p. N/pp. N link only where the thesis cites itself. A page reference inside a citation belongs to somebody else's book: the script tests the preceding clause for citation markers (a title in \emph, ibid., a shelfmark, publication data, a surname followed by a comma) before accepting self-reference words (above, below, see, mentioned). Neither signal → no link; a missing link is cheaper than one that lands on the wrong page. Re-audit after changing this rule: print every link with its context and read them. Protected from all of the above: comment lines, file paths, and the arguments of \erratum, \qslip, \ig, \origpage, \fn, and the heading macros (their text becomes bookmarks and marks, and a link inside them breaks the build).

Inline type specimens — the workflow

For any page where the original pastes typeforms into its own text:

1. docker compose run --rm -T tex python3 work/lines.py N — lists the page's text lines with their y-fractions. Its line N is usually the line after the one you counted by eye; if a sheet comes back with the wrong line, try the band one earlier.
2. … work/glyphs.py N y0,y1 [y0,y1 …] — numbered contact sheet of the ink runs on those bands, written to work/sheetN.png. Read it and pick the runs that are specimens.
3. … tools/crop_inline.py N name=SPEC … — cuts them to plates/inline/pNNNN-name.png. SPEC is a run index plus, optionally, :left|right|first|last|mid|trimright|trimleft to split a run that merged with its neighbour, +PX to start PX into the run, ~PX to keep only its first PX. Use +PX/~PX when the typewriter's space is too noisy to register as a gap.
4. Where a run defeats all of that (a word set solid inside a footnote), crop by explicit pixel box with PIL and record the box in the manifest notes column so the cut is reproducible.

Per-batch procedure (≈12–15 pages per batch)

1. make prep A=<pdf> B=<pdf> → scans/hi-*.jpg + work/ocr/p*.txt.
2. For each page, look at the image. Decide kind: text, plate, mixed, blank.
3. Text page: python3 tools/ocr2tex.py <pdf> <printed> [--head "Chapter title"] gives a draft; then proofread against the image: italics (titles, transliterated words), \fn{} marks placed exactly where the superscript sits, footnote text moved into \fn{}, all diacritics (ā ī ū ṛ ṃ ṅ ñ ṭ ḍ ṇ ś ṣ ḥ), hyphenation joined, en-dashes kept as - as the original types them. The typescript never hyphenates a word at a line end — verified across all 431 leaves: every one of its 59 line-ending hyphens is either a compound the author typed (offset-lithography, Hof-und, forty-eight) or OCR noise off a reproduced plate. So a hyphen in a page file is always intrinsic to the word; never invent one, and never keep one that splits a word. Output hyphenation is off to match (Sammay, 2026-09-14). Where the author compounds inconsistently (type-casting/typecasting, punch-cutter/punchcutter) rule 3 applies: each instance stands as printed. Page break inside a paragraph: the previous file ends mid-sentence and this file begins \origpage{N} followed by the continuation — no blank line between files (the build concatenates them), so do not start the file with \par.
4. Plate page: make plate N=<pdf> (inspect plates/pNNNN.png; if the caption or page number survived or content was clipped, rerun with ARGS="--top 0.05 --bottom 0.11" etc.), then a one-line file with \plateop{...} and the caption transcribed exactly.
5. Update manifest.tsv rows (python3 tools/setrow.py <pdf> <printed> <kind> "notes"); make check; make build; spot-check 2–3 output pages (pdftoppm -jpeg -r 60 -f X -l Y ross-1988-retypeset.pdf work/o).
6. Update CONTINUATION.md (state, next batch) and QUESTIONS.md. Stop at a chapter or batch boundary, never mid-file.

Environment

Tools run in Docker: make image once, then the make targets above (NODOCKER=1 make … if xelatex, poppler-utils, tesseract, python3-pil/numpy/pdfplumber are installed locally). Fonts: Liberation (system), FreeSerif (system), Tiro Bangla (fonts/). Do not add fonts.