diff --git a/docs/content-model.md b/docs/content-model.md index 8812cf8..a996c49 100644 --- a/docs/content-model.md +++ b/docs/content-model.md @@ -241,6 +241,26 @@ assets, defaults off, and is the only thing that changes the answer. For a future-dated bundle the 404 expires at its publish time, so it becomes visible exactly when the bundle becomes public. +## Typography and localisation + +The line is drawn by who wrote the words (ADR-0034). + +**Authored body text** gets one change and no others: the Markdown typographer smooths quotes, dashes and +ellipses. It works on the parsed tree, so a code span keeps its straight quotes and its `--`. Body text is +never localised — Bengali numerals inside a sentence are the author's decision, and a title stays exactly +as typed. + +**Chrome** — every word the engine puts on a page that the author did not write — is localised: labels, +page counts, positions, month names and digits, from an engine-owned table reached by the template +functions in `theme-contract.md`. A Bengali page reads `পৃষ্ঠা ২ / ২` and `৮ মার্চ ২০২৬`. + +**Machine-readable output never localises.** A `datetime` attribute, a URL, or anything a parser reads +stays ASCII in every locale. + +Widow prevention is not implemented: doing it safely needs a transform over the parsed tree rather than a +pass over rendered HTML, which cannot tell prose from an escaped code span. It waits for the Stage +pipeline. + ## Includes and shortcodes `[spec]` Shortcodes are a Stage running on trusted content only (ADR-0003), never on comments. File inclusion diff --git a/docs/state.md b/docs/state.md index b142b67..467c4d9 100644 --- a/docs/state.md +++ b/docs/state.md @@ -1,6 +1,6 @@ # State -**Verified against:** `919d6fc` on 2026-07-30 — update this line every change. +**Verified against:** `430a5ae` on 2026-07-30 — update this line every change. If this file disagrees with the code, the code is right and this file is a bug. ## Inventory @@ -11,16 +11,19 @@ If this file disagrees with the code, the code is right and this file is a bug. | `internal/content/doc.go` | package comment | 5 | | `internal/content/content.go` | bundles: `os.Root` open, walk, frontmatter split, key/lang derivation, NFC, tag slugs, permalink building | 352 | | `internal/content/site.go` | the indexed site: lookup with language fallback, aliases, `Query` and `Run`, sections, `Sequence` | 286 | -| `internal/render/render.go` | goldmark, per-kind template sets with site override, `Page`/`List`/`Sequence`/`head` | 297 | +| `internal/render/render.go` | goldmark with the typographer, per-kind template sets with site override, `Page`/`List`/`Sequence`/`head` | 305 | +| `internal/render/chrome.go` | the engine's own words: phrase table, month names, digits, and the `t`/`num`/`day` template funcs (ADR-0034) | 105 | | `internal/render/templates/` | reference theme: `base.html`, `page.html`, `list.html`, `theme.css` (ADR-0026) | — | | `internal/web/resolve.go` | URL → (key, lang, page, tag) or a canonical redirect: language prefix, `/en/…` fork guard, pagination, tags, trailing slash | 112 | | `internal/web/web.go` | handler: resolve, look up with fallback, section and tag listings, sequence, `/static/`, degrade on failure | 156 | | `cmd/khosra/main.go` | flags, wiring, startup — the only place things are assembled | 53 | -| `*_test.go` | table-driven; symlink escape, permalink, language fallback, aliases, pagination, tags, sequences, 404 | 1011 | +| `*_test.go` | table-driven; symlink escape, permalink, language fallback, aliases, pagination, tags, sequences, chrome, typography, 404 | 1119 | Serves a bundle at `/{section}/{slug}/`, a paginated listing per section, tag listings global and -section-narrowed, sequence navigation and a series archive on any nested bundle, and `static/` verbatim. This repo holds engine source only — the site root is external -and passed with `-site` (ADR-0011). +section-narrowed, sequence navigation and a series archive on any nested bundle, and `static/` verbatim. +Chrome text, dates and digits render in English or Bengali; authored text is untouched but for typographic +smoothing (ADR-0034). This repo holds engine source only — the site root is external and passed with +`-site` (ADR-0011). Dependencies: three, all allowlisted — `goldmark`, `golang.org/x/text`, `gopkg.in/yaml.v3`. @@ -31,7 +34,7 @@ this change*. | Counter | Now | Extraction due at | What it buys | |---|---|---|---| -| Render transforms | 0 | **3** | Stage pipeline (ordered `func(ctx,*Page) error`) | +| Render transforms | 0 | **3** | Stage pipeline (ordered `func(ctx,*Page) error`). Typography is *not* one: it is a goldmark parser option, not a function over a page, so it buys the feature without moving the counter. Shortcodes (queue 12) will be the first real one | | Routing cases | 5 | **2** — done | Resolver at `internal/web/resolve.go`: bundle, language prefix, pagination, tag, section-narrowed tag | | Collection pages | 4 | **1** — done | Query primitive: `content.Query{Section, Tag, Lang}` + `Site.Run`. The fourth — a series archive — resolves through `Site.Sequence` instead: membership is structural and the sort ascends, so it shares the index but not the Query | | Views / output formats | 2 | **2** — due | Two template sets exist (bundle, listing); the View layer is Arc 2's third item | diff --git a/docs/theme-contract.md b/docs/theme-contract.md index d442d10..7267839 100644 --- a/docs/theme-contract.md +++ b/docs/theme-contract.md @@ -50,6 +50,28 @@ A listing page receives `.Title`, `.Lang`, `.Canonical`, `.Style` as above, plus | `.PrevURL`, `.NextURL` | empty at the ends; *newer* is `prev`, because the order is newest first | | `.Groups` | set instead of `.Items` when entries are grouped — a tag listing groups by section, each `.Name` and `.Items` | +## Chrome text and formatting + +Three functions, available in every template including a site root's own blocks. They exist so no template +hardcodes English: the words the engine supplies are the engine's to localise (ADR-0034). + +| Call | Gives | +|---|---| +| `{{t .Lang "newer"}}` | one phrase in that language. Extra arguments fill `%s` placeholders in order | +| `{{num .Lang .Page}}` | an integer in that language's digits — `12`, `১২` | +| `{{day .Lang .Date}}` | a date as that language reads it — `8 March 2026`, `৮ মার্চ ২০২৬`; empty for a zero date | + +Phrase keys today: `newer`, `older`, `empty`, `page-of` (two arguments), `position` (two arguments). An +unknown language falls back to the default locale and an unknown key returns itself, so a missing +translation can never blank a page or fail a render. + +Two rules a theme must keep: put the machine-readable form in the attribute and the localised form in the +text — `` — and never localise +`.Title`, `.HTML` or anything else the author wrote. + +A site root cannot add or override a phrase yet. A theme needing its own words writes them in its own +block; site-supplied strings wait for the settings cascade (`ideas/deferred-decisions.md`). + ## The stability rule Fields and names are **added, never renamed or removed**. Absence is always legal: a template reading a @@ -69,7 +91,7 @@ Breaking the contract is not a feature — it is a new contract version, and it | named template lookup | per-type sets; a theme redefines a named block and inherits the rest (ADR-0019) | | URLs | every path the engine emits, so a theme never constructs one by hand | | per-page assets | the `styles` / `scripts` frontmatter lists, resolved relative to the bundle | -| chrome strings | looked up by key and language, never hardcoded English in a template | +| chrome strings | looked up by key and language, never hardcoded English in a template — live, see above | | validity windows | a template that renders time-dependent output declares one | ## Extras view diff --git a/internal/render/chrome.go b/internal/render/chrome.go new file mode 100644 index 0000000..7569fe5 --- /dev/null +++ b/internal/render/chrome.go @@ -0,0 +1,105 @@ +package render + +import ( + "fmt" + "html/template" + "strconv" + "strings" + "time" + + "khosra/internal/content" +) + +// Chrome is every word the engine puts on a page that the author did not write, in each language it +// knows (ADR-0034). Keyed by phrase then language, so both forms sit side by side and a missing one is +// visible while reading rather than at request time. +// +// A phrase may carry %s placeholders, filled in order by the caller. Templates reach these through the +// `t`, `num` and `day` functions; nothing here is content, and content never comes from here. +var chrome = map[string]map[string]string{ + "newer": {"en": "Newer", "bn": "নতুন"}, + "older": {"en": "Older", "bn": "পুরোনো"}, + "empty": {"en": "Nothing here yet.", "bn": "এখনও কিছু নেই।"}, + "page-of": {"en": "Page %s of %s", "bn": "পৃষ্ঠা %s / %s"}, + "position": {"en": "%s of %s", "bn": "%s / %s"}, +} + +// months are Gregorian month names per language, indexed by [time.Month]-1. +var months = map[string][]string{ + "en": {"January", "February", "March", "April", "May", "June", + "July", "August", "September", "October", "November", "December"}, + "bn": {"জানুয়ারি", "ফেব্রুয়ারি", "মার্চ", "এপ্রিল", "মে", "জুন", + "জুলাই", "আগস্ট", "সেপ্টেম্বর", "অক্টোবর", "নভেম্বর", "ডিসেম্বর"}, +} + +// digits are the decimal digits of a script, indexed 0-9. A language absent here uses ASCII. +var digits = map[string][]rune{ + "bn": []rune("০১২৩৪৫৬৭৮৯"), +} + +// funcs are the chrome helpers a template may call. Registered before parsing, so a site root's override +// blocks may use them too. +var funcs = template.FuncMap{"t": text, "num": numerals, "day": day} + +// text is one chrome phrase in a language, with any %s placeholders filled. +// +// It cannot fail: an unknown language falls back to the default locale and an unknown phrase returns its +// own key, because a missing translation must never blank a page or break a render (invariant 1). +func text(lang, key string, args ...string) string { + forms := chrome[key] + phrase, ok := forms[lang] + if !ok { + phrase = forms[content.DefaultLang] + } + if phrase == "" { + return key + } + if len(args) == 0 { + return phrase + } + filled := make([]any, len(args)) + for i, a := range args { + filled[i] = a + } + return fmt.Sprintf(phrase, filled...) +} + +// numerals is an integer in the script of a language: 12 in English, ১২ in Bengali. +func numerals(lang string, n int) string { + return localiseDigits(lang, strconv.Itoa(n)) +} + +// day is a date as a reader of that language would read it — "8 March 2026", "৮ মার্চ ২০২৬". +// +// One path for every language rather than [time.Time.Format] for the default locale and a hand-built +// string for the rest, so English cannot drift from the others. The machine-readable form belongs in a +// datetime attribute and stays ASCII (ADR-0034). +func day(lang string, t time.Time) string { + if t.IsZero() { + return "" + } + names, ok := months[lang] + if !ok { + names = months[content.DefaultLang] + } + return strings.Join([]string{ + numerals(lang, t.Day()), + names[t.Month()-1], + numerals(lang, t.Year()), + }, " ") +} + +// localiseDigits swaps ASCII digits for a script's own, leaving everything else alone. +func localiseDigits(lang, s string) string { + set, ok := digits[lang] + if !ok { + return s + } + out := []rune(s) + for i, r := range out { + if r >= '0' && r <= '9' { + out[i] = set[r-'0'] + } + } + return string(out) +} diff --git a/internal/render/chrome_test.go b/internal/render/chrome_test.go new file mode 100644 index 0000000..78f30d2 --- /dev/null +++ b/internal/render/chrome_test.go @@ -0,0 +1,108 @@ +package render + +import ( + "strings" + "testing" + "time" + + "khosra/internal/content" +) + +func TestChromeTextIsLocalisedAndNeverEmpty(t *testing.T) { + cases := []struct{ lang, key, want string }{ + {"en", "newer", "Newer"}, + {"bn", "newer", "নতুন"}, + {"fr", "newer", "Newer"}, // unknown language falls back to the default locale + {"bn", "nope", "nope"}, // unknown phrase returns its key rather than nothing + } + for _, c := range cases { + if got := text(c.lang, c.key); got != c.want { + t.Errorf("text(%q, %q) = %q, want %q", c.lang, c.key, got, c.want) + } + } + if got := text("bn", "page-of", numerals("bn", 2), numerals("bn", 11)); got != "পৃষ্ঠা ২ / ১১" { + t.Errorf("page-of in Bengali = %q", got) + } + if got := text("en", "page-of", "2", "11"); got != "Page 2 of 11" { + t.Errorf("page-of in English = %q", got) + } +} + +func TestEveryPhraseCarriesEveryLanguage(t *testing.T) { + // A phrase missing a language falls back silently, so nothing else would catch a half-translated table. + for key, forms := range chrome { + for lang := range months { + if forms[lang] == "" { + t.Errorf("phrase %q has no %s form", key, lang) + } + } + } + for lang, names := range months { + if len(names) != 12 { + t.Errorf("%s has %d month names, want 12", lang, len(names)) + } + } + if len(digits["bn"]) != 10 { + t.Errorf("Bengali digits = %d, want 10", len(digits["bn"])) + } +} + +func TestDatesReadInTheirOwnScript(t *testing.T) { + when := time.Date(2026, time.March, 8, 0, 0, 0, 0, time.UTC) + if got := day(content.DefaultLang, when); got != "8 March 2026" { + t.Errorf("English date = %q", got) + } + if got := day("bn", when); got != "৮ মার্চ ২০২৬" { + t.Errorf("Bengali date = %q, want Bengali digits and month", got) + } + if got := day("en", time.Time{}); got != "" { + t.Errorf("an undated bundle should render nothing, got %q", got) + } +} + +func TestTypographerSmoothsProseAndLeavesCodeAlone(t *testing.T) { + r, err := New(nil) + if err != nil { + t.Fatal(err) + } + b, err := content.Parse("posts/x.md", []byte( + "---\ntitle: X\n---\nShe said \"wait\" -- then left... `\"raw\" -- code`\n")) + if err != nil { + t.Fatal(err) + } + out, err := r.Bundle(b, "en", nil, nil) + if err != nil { + t.Fatal(err) + } + got := string(out) + for _, want := range []string{"“wait”", "–", "…"} { + if !strings.Contains(got, want) { + t.Errorf("prose missing %s:\n%s", want, got) + } + } + if !strings.Contains(got, `"raw" -- code`) { + t.Errorf("a code span must survive untouched:\n%s", got) + } +} + +func TestMachineReadableOutputStaysASCII(t *testing.T) { + r, err := New(nil) + if err != nil { + t.Fatal(err) + } + b, err := content.Parse("posts/x.bn.md", []byte("---\ntitle: এক\ndate: 2026-03-08\n---\n")) + if err != nil { + t.Fatal(err) + } + out, err := r.Listing("posts", "bn", []content.Bundle{b}, 1) + if err != nil { + t.Fatal(err) + } + got := string(out) + if !strings.Contains(got, `datetime="2026-03-08"`) { + t.Errorf("datetime must stay ASCII for parsers (ADR-0034):\n%s", got) + } + if !strings.Contains(got, "৮ মার্চ ২০২৬") { + t.Errorf("the visible date should read in Bengali:\n%s", got) + } +} diff --git a/internal/render/render.go b/internal/render/render.go index f102920..a8f580e 100644 --- a/internal/render/render.go +++ b/internal/render/render.go @@ -14,6 +14,7 @@ import ( "time" "github.com/yuin/goldmark" + "github.com/yuin/goldmark/extension" "khosra/internal/content" ) @@ -129,7 +130,12 @@ func New(siteFS fs.FS) (*Renderer, error) { if err != nil { return nil, err } - return &Renderer{page: page, list: list, md: goldmark.New(), style: css}, nil + // The typographer smooths quotes, dashes and ellipses in authored prose and leaves code spans alone, + // because it works on the parsed tree rather than the text. That is the only change the engine makes to + // an author's words (ADR-0034), and it is a parser option rather than a render transform, so it does + // not move the transforms counter. + md := goldmark.New(goldmark.WithExtensions(extension.Typographer)) + return &Renderer{page: page, list: list, md: md, style: css}, nil } // parseSet builds one kind of page: the embedded base and block, then the site's versions of exactly @@ -140,7 +146,9 @@ func New(siteFS fs.FS) (*Renderer, error) { // site template into every set would let a listing's "main" leak into bundle pages, which is the // collision per-kind sets exist to prevent. func parseSet(siteFS fs.FS, kind string) (*template.Template, error) { - set, err := template.ParseFS(themeFS, "templates/base.html", kind) + // Funcs are attached before anything is parsed, so the chrome helpers are available to a site + // override's blocks as well as the embedded ones (ADR-0034). + set, err := template.New("theme").Funcs(funcs).ParseFS(themeFS, "templates/base.html", kind) if err != nil { return nil, fmt.Errorf("parse embedded: %w", err) } diff --git a/internal/render/templates/list.html b/internal/render/templates/list.html index a014280..d7282ca 100644 --- a/internal/render/templates/list.html +++ b/internal/render/templates/list.html @@ -6,7 +6,7 @@ {{- end}} @@ -14,17 +14,17 @@ {{- else}} -

Nothing here yet.

+

{{t .Lang "empty"}}

{{- end}} {{- if or .PrevURL .NextURL}} {{- end}} {{- end}} diff --git a/internal/render/templates/page.html b/internal/render/templates/page.html index 3023c5c..68bcdc4 100644 --- a/internal/render/templates/page.html +++ b/internal/render/templates/page.html @@ -7,7 +7,7 @@ {{- if .Index}} {{- else if .Members}} @@ -15,7 +15,7 @@
    {{- range .Members}}
  1. {{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}} -{{- if not .Date.IsZero}} {{end}}
  2. +{{- if not .Date.IsZero}} {{end}} {{- end}}