localise chrome, smooth prose, in English and Bengali

The engine now owns the words it puts on a page that the author did not write
(ADR-0034). `internal/render/chrome.go` holds the phrase table, Gregorian month
names and decimal digits per language, keyed phrase-then-language so both forms
sit side by side and a half-translated row is visible while reading. Templates
reach it through `t`, `num` and `day`, registered before parsing so a site
override's blocks may call them too.

The reference theme stops hardcoding English: "Newer", "Page 2 of 2" and every
date now come from the table, while `datetime` attributes stay ASCII because a
parser reads them.

Authored text gets goldmark's typographer and nothing else — quotes, dashes and
ellipses smoothed, code spans untouched because it works on the parsed tree. It
is a parser option rather than a function over a page, so the transforms counter
does not move; state.md now says why, so the next reader does not miscount.

Deliberately absent: relative dates, which need a validity window that only the
cache entry will have, and body widow prevention, which cannot be done safely by
a pass over rendered HTML.
This commit is contained in:
2026-08-01 02:23:35 +06:00
parent 206974650d
commit c244855ebc
8 changed files with 283 additions and 17 deletions
+20
View File
@@ -241,6 +241,26 @@ assets, defaults off, and is the only thing that changes the answer.
For a future-dated bundle the 404 expires at its publish time, so it becomes visible
exactly when the bundle becomes public.
## Typography and localisation
The line is drawn by who wrote the words (ADR-0034).
**Authored body text** gets one change and no others: the Markdown typographer smooths quotes, dashes and
ellipses. It works on the parsed tree, so a code span keeps its straight quotes and its `--`. Body text is
never localised — Bengali numerals inside a sentence are the author's decision, and a title stays exactly
as typed.
**Chrome** — every word the engine puts on a page that the author did not write — is localised: labels,
page counts, positions, month names and digits, from an engine-owned table reached by the template
functions in `theme-contract.md`. A Bengali page reads `পৃষ্ঠা ২ / ২` and `৮ মার্চ ২০২৬`.
**Machine-readable output never localises.** A `datetime` attribute, a URL, or anything a parser reads
stays ASCII in every locale.
Widow prevention is not implemented: doing it safely needs a transform over the parsed tree rather than a
pass over rendered HTML, which cannot tell prose from an escaped code span. It waits for the Stage
pipeline.
## Includes and shortcodes `[spec]`
Shortcodes are a Stage running on trusted content only (ADR-0003), never on comments. File inclusion
+9 -6
View File
@@ -1,6 +1,6 @@
# State
**Verified against:** `919d6fc` on 2026-07-30 — update this line every change.
**Verified against:** `430a5ae` on 2026-07-30 — update this line every change.
If this file disagrees with the code, the code is right and this file is a bug.
## Inventory
@@ -11,16 +11,19 @@ If this file disagrees with the code, the code is right and this file is a bug.
| `internal/content/doc.go` | package comment | 5 |
| `internal/content/content.go` | bundles: `os.Root` open, walk, frontmatter split, key/lang derivation, NFC, tag slugs, permalink building | 352 |
| `internal/content/site.go` | the indexed site: lookup with language fallback, aliases, `Query` and `Run`, sections, `Sequence` | 286 |
| `internal/render/render.go` | goldmark, per-kind template sets with site override, `Page`/`List`/`Sequence`/`head` | 297 |
| `internal/render/render.go` | goldmark with the typographer, per-kind template sets with site override, `Page`/`List`/`Sequence`/`head` | 305 |
| `internal/render/chrome.go` | the engine's own words: phrase table, month names, digits, and the `t`/`num`/`day` template funcs (ADR-0034) | 105 |
| `internal/render/templates/` | reference theme: `base.html`, `page.html`, `list.html`, `theme.css` (ADR-0026) | — |
| `internal/web/resolve.go` | URL → (key, lang, page, tag) or a canonical redirect: language prefix, `/en/…` fork guard, pagination, tags, trailing slash | 112 |
| `internal/web/web.go` | handler: resolve, look up with fallback, section and tag listings, sequence, `/static/`, degrade on failure | 156 |
| `cmd/khosra/main.go` | flags, wiring, startup — the only place things are assembled | 53 |
| `*_test.go` | table-driven; symlink escape, permalink, language fallback, aliases, pagination, tags, sequences, 404 | 1011 |
| `*_test.go` | table-driven; symlink escape, permalink, language fallback, aliases, pagination, tags, sequences, chrome, typography, 404 | 1119 |
Serves a bundle at `/{section}/{slug}/`, a paginated listing per section, tag listings global and
section-narrowed, sequence navigation and a series archive on any nested bundle, and `static/` verbatim. This repo holds engine source only — the site root is external
and passed with `-site` (ADR-0011).
section-narrowed, sequence navigation and a series archive on any nested bundle, and `static/` verbatim.
Chrome text, dates and digits render in English or Bengali; authored text is untouched but for typographic
smoothing (ADR-0034). This repo holds engine source only — the site root is external and passed with
`-site` (ADR-0011).
Dependencies: three, all allowlisted — `goldmark`, `golang.org/x/text`, `gopkg.in/yaml.v3`.
@@ -31,7 +34,7 @@ this change*.
| Counter | Now | Extraction due at | What it buys |
|---|---|---|---|
| Render transforms | 0 | **3** | Stage pipeline (ordered `func(ctx,*Page) error`) |
| Render transforms | 0 | **3** | Stage pipeline (ordered `func(ctx,*Page) error`). Typography is *not* one: it is a goldmark parser option, not a function over a page, so it buys the feature without moving the counter. Shortcodes (queue 12) will be the first real one |
| Routing cases | 5 | **2** — done | Resolver at `internal/web/resolve.go`: bundle, language prefix, pagination, tag, section-narrowed tag |
| Collection pages | 4 | **1** — done | Query primitive: `content.Query{Section, Tag, Lang}` + `Site.Run`. The fourth — a series archive — resolves through `Site.Sequence` instead: membership is structural and the sort ascends, so it shares the index but not the Query |
| Views / output formats | 2 | **2** — due | Two template sets exist (bundle, listing); the View layer is Arc 2's third item |
+23 -1
View File
@@ -50,6 +50,28 @@ A listing page receives `.Title`, `.Lang`, `.Canonical`, `.Style` as above, plus
| `.PrevURL`, `.NextURL` | empty at the ends; *newer* is `prev`, because the order is newest first |
| `.Groups` | set instead of `.Items` when entries are grouped — a tag listing groups by section, each `.Name` and `.Items` |
## Chrome text and formatting
Three functions, available in every template including a site root's own blocks. They exist so no template
hardcodes English: the words the engine supplies are the engine's to localise (ADR-0034).
| Call | Gives |
|---|---|
| `{{t .Lang "newer"}}` | one phrase in that language. Extra arguments fill `%s` placeholders in order |
| `{{num .Lang .Page}}` | an integer in that language's digits — `12`, `১২` |
| `{{day .Lang .Date}}` | a date as that language reads it — `8 March 2026`, `৮ মার্চ ২০২৬`; empty for a zero date |
Phrase keys today: `newer`, `older`, `empty`, `page-of` (two arguments), `position` (two arguments). An
unknown language falls back to the default locale and an unknown key returns itself, so a missing
translation can never blank a page or fail a render.
Two rules a theme must keep: put the machine-readable form in the attribute and the localised form in the
text — `<time datetime="{{.Date.Format "2006-01-02"}}">{{day .Lang .Date}}</time>` — and never localise
`.Title`, `.HTML` or anything else the author wrote.
A site root cannot add or override a phrase yet. A theme needing its own words writes them in its own
block; site-supplied strings wait for the settings cascade (`ideas/deferred-decisions.md`).
## The stability rule
Fields and names are **added, never renamed or removed**. Absence is always legal: a template reading a
@@ -69,7 +91,7 @@ Breaking the contract is not a feature — it is a new contract version, and it
| named template lookup | per-type sets; a theme redefines a named block and inherits the rest (ADR-0019) |
| URLs | every path the engine emits, so a theme never constructs one by hand |
| per-page assets | the `styles` / `scripts` frontmatter lists, resolved relative to the bundle |
| chrome strings | looked up by key and language, never hardcoded English in a template |
| chrome strings | looked up by key and language, never hardcoded English in a template — live, see above |
| validity windows | a template that renders time-dependent output declares one |
## Extras view
+105
View File
@@ -0,0 +1,105 @@
package render
import (
"fmt"
"html/template"
"strconv"
"strings"
"time"
"khosra/internal/content"
)
// Chrome is every word the engine puts on a page that the author did not write, in each language it
// knows (ADR-0034). Keyed by phrase then language, so both forms sit side by side and a missing one is
// visible while reading rather than at request time.
//
// A phrase may carry %s placeholders, filled in order by the caller. Templates reach these through the
// `t`, `num` and `day` functions; nothing here is content, and content never comes from here.
var chrome = map[string]map[string]string{
"newer": {"en": "Newer", "bn": "নতুন"},
"older": {"en": "Older", "bn": "পুরোনো"},
"empty": {"en": "Nothing here yet.", "bn": "এখনও কিছু নেই।"},
"page-of": {"en": "Page %s of %s", "bn": "পৃষ্ঠা %s / %s"},
"position": {"en": "%s of %s", "bn": "%s / %s"},
}
// months are Gregorian month names per language, indexed by [time.Month]-1.
var months = map[string][]string{
"en": {"January", "February", "March", "April", "May", "June",
"July", "August", "September", "October", "November", "December"},
"bn": {"জানুয়ারি", "ফেব্রুয়ারি", "মার্চ", "এপ্রিল", "মে", "জুন",
"জুলাই", "আগস্ট", "সেপ্টেম্বর", "অক্টোবর", "নভেম্বর", "ডিসেম্বর"},
}
// digits are the decimal digits of a script, indexed 0-9. A language absent here uses ASCII.
var digits = map[string][]rune{
"bn": []rune("০১২৩৪৫৬৭৮৯"),
}
// funcs are the chrome helpers a template may call. Registered before parsing, so a site root's override
// blocks may use them too.
var funcs = template.FuncMap{"t": text, "num": numerals, "day": day}
// text is one chrome phrase in a language, with any %s placeholders filled.
//
// It cannot fail: an unknown language falls back to the default locale and an unknown phrase returns its
// own key, because a missing translation must never blank a page or break a render (invariant 1).
func text(lang, key string, args ...string) string {
forms := chrome[key]
phrase, ok := forms[lang]
if !ok {
phrase = forms[content.DefaultLang]
}
if phrase == "" {
return key
}
if len(args) == 0 {
return phrase
}
filled := make([]any, len(args))
for i, a := range args {
filled[i] = a
}
return fmt.Sprintf(phrase, filled...)
}
// numerals is an integer in the script of a language: 12 in English, ১২ in Bengali.
func numerals(lang string, n int) string {
return localiseDigits(lang, strconv.Itoa(n))
}
// day is a date as a reader of that language would read it — "8 March 2026", "৮ মার্চ ২০২৬".
//
// One path for every language rather than [time.Time.Format] for the default locale and a hand-built
// string for the rest, so English cannot drift from the others. The machine-readable form belongs in a
// datetime attribute and stays ASCII (ADR-0034).
func day(lang string, t time.Time) string {
if t.IsZero() {
return ""
}
names, ok := months[lang]
if !ok {
names = months[content.DefaultLang]
}
return strings.Join([]string{
numerals(lang, t.Day()),
names[t.Month()-1],
numerals(lang, t.Year()),
}, " ")
}
// localiseDigits swaps ASCII digits for a script's own, leaving everything else alone.
func localiseDigits(lang, s string) string {
set, ok := digits[lang]
if !ok {
return s
}
out := []rune(s)
for i, r := range out {
if r >= '0' && r <= '9' {
out[i] = set[r-'0']
}
}
return string(out)
}
+108
View File
@@ -0,0 +1,108 @@
package render
import (
"strings"
"testing"
"time"
"khosra/internal/content"
)
func TestChromeTextIsLocalisedAndNeverEmpty(t *testing.T) {
cases := []struct{ lang, key, want string }{
{"en", "newer", "Newer"},
{"bn", "newer", "নতুন"},
{"fr", "newer", "Newer"}, // unknown language falls back to the default locale
{"bn", "nope", "nope"}, // unknown phrase returns its key rather than nothing
}
for _, c := range cases {
if got := text(c.lang, c.key); got != c.want {
t.Errorf("text(%q, %q) = %q, want %q", c.lang, c.key, got, c.want)
}
}
if got := text("bn", "page-of", numerals("bn", 2), numerals("bn", 11)); got != "পৃষ্ঠা ২ / ১১" {
t.Errorf("page-of in Bengali = %q", got)
}
if got := text("en", "page-of", "2", "11"); got != "Page 2 of 11" {
t.Errorf("page-of in English = %q", got)
}
}
func TestEveryPhraseCarriesEveryLanguage(t *testing.T) {
// A phrase missing a language falls back silently, so nothing else would catch a half-translated table.
for key, forms := range chrome {
for lang := range months {
if forms[lang] == "" {
t.Errorf("phrase %q has no %s form", key, lang)
}
}
}
for lang, names := range months {
if len(names) != 12 {
t.Errorf("%s has %d month names, want 12", lang, len(names))
}
}
if len(digits["bn"]) != 10 {
t.Errorf("Bengali digits = %d, want 10", len(digits["bn"]))
}
}
func TestDatesReadInTheirOwnScript(t *testing.T) {
when := time.Date(2026, time.March, 8, 0, 0, 0, 0, time.UTC)
if got := day(content.DefaultLang, when); got != "8 March 2026" {
t.Errorf("English date = %q", got)
}
if got := day("bn", when); got != "৮ মার্চ ২০২৬" {
t.Errorf("Bengali date = %q, want Bengali digits and month", got)
}
if got := day("en", time.Time{}); got != "" {
t.Errorf("an undated bundle should render nothing, got %q", got)
}
}
func TestTypographerSmoothsProseAndLeavesCodeAlone(t *testing.T) {
r, err := New(nil)
if err != nil {
t.Fatal(err)
}
b, err := content.Parse("posts/x.md", []byte(
"---\ntitle: X\n---\nShe said \"wait\" -- then left... `\"raw\" -- code`\n"))
if err != nil {
t.Fatal(err)
}
out, err := r.Bundle(b, "en", nil, nil)
if err != nil {
t.Fatal(err)
}
got := string(out)
for _, want := range []string{"&ldquo;wait&rdquo;", "&ndash;", "&hellip;"} {
if !strings.Contains(got, want) {
t.Errorf("prose missing %s:\n%s", want, got)
}
}
if !strings.Contains(got, `<code>&quot;raw&quot; -- code</code>`) {
t.Errorf("a code span must survive untouched:\n%s", got)
}
}
func TestMachineReadableOutputStaysASCII(t *testing.T) {
r, err := New(nil)
if err != nil {
t.Fatal(err)
}
b, err := content.Parse("posts/x.bn.md", []byte("---\ntitle: এক\ndate: 2026-03-08\n---\n"))
if err != nil {
t.Fatal(err)
}
out, err := r.Listing("posts", "bn", []content.Bundle{b}, 1)
if err != nil {
t.Fatal(err)
}
got := string(out)
if !strings.Contains(got, `datetime="2026-03-08"`) {
t.Errorf("datetime must stay ASCII for parsers (ADR-0034):\n%s", got)
}
if !strings.Contains(got, "৮ মার্চ ২০২৬") {
t.Errorf("the visible date should read in Bengali:\n%s", got)
}
}
+10 -2
View File
@@ -14,6 +14,7 @@ import (
"time"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/extension"
"khosra/internal/content"
)
@@ -129,7 +130,12 @@ func New(siteFS fs.FS) (*Renderer, error) {
if err != nil {
return nil, err
}
return &Renderer{page: page, list: list, md: goldmark.New(), style: css}, nil
// The typographer smooths quotes, dashes and ellipses in authored prose and leaves code spans alone,
// because it works on the parsed tree rather than the text. That is the only change the engine makes to
// an author's words (ADR-0034), and it is a parser option rather than a render transform, so it does
// not move the transforms counter.
md := goldmark.New(goldmark.WithExtensions(extension.Typographer))
return &Renderer{page: page, list: list, md: md, style: css}, nil
}
// parseSet builds one kind of page: the embedded base and block, then the site's versions of exactly
@@ -140,7 +146,9 @@ func New(siteFS fs.FS) (*Renderer, error) {
// site template into every set would let a listing's "main" leak into bundle pages, which is the
// collision per-kind sets exist to prevent.
func parseSet(siteFS fs.FS, kind string) (*template.Template, error) {
set, err := template.ParseFS(themeFS, "templates/base.html", kind)
// Funcs are attached before anything is parsed, so the chrome helpers are available to a site
// override's blocks as well as the embedded ones (ADR-0034).
set, err := template.New("theme").Funcs(funcs).ParseFS(themeFS, "templates/base.html", kind)
if err != nil {
return nil, fmt.Errorf("parse embedded: %w", err)
}
+6 -6
View File
@@ -6,7 +6,7 @@
<ul class="listing">
{{- range .Items}}
<li><a href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}}</a>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{.Date.Format "2 January 2006"}}</time>{{end}}</li>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{day $.Lang .Date}}</time>{{end}}</li>
{{- end}}
</ul>
{{- end}}
@@ -14,17 +14,17 @@
<ul class="listing">
{{- range .Items}}
<li><a href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}}</a>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{.Date.Format "2 January 2006"}}</time>{{end}}</li>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{day $.Lang .Date}}</time>{{end}}</li>
{{- end}}
</ul>
{{- else}}
<p>Nothing here yet.</p>
<p>{{t .Lang "empty"}}</p>
{{- end}}
{{- if or .PrevURL .NextURL}}
<nav class="pagination">
{{- if .PrevURL}}<a rel="prev" href="{{.PrevURL}}">Newer</a>{{end}}
<span>Page {{.Page}} of {{.Pages}}</span>
{{- if .NextURL}}<a rel="next" href="{{.NextURL}}">Older</a>{{end}}
{{- if .PrevURL}}<a rel="prev" href="{{.PrevURL}}">{{t .Lang "newer"}}</a>{{end}}
<span>{{t .Lang "page-of" (num .Lang .Page) (num .Lang .Pages)}}</span>
{{- if .NextURL}}<a rel="next" href="{{.NextURL}}">{{t .Lang "older"}}</a>{{end}}
</nav>
{{- end}}
{{- end}}
+2 -2
View File
@@ -7,7 +7,7 @@
{{- if .Index}}
<nav class="sequence">
{{- with .Prev}}<a rel="prev" href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}}</a>{{end}}
<span><a href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.URL}}{{end}}</a> {{.Index}} of {{.Count}}</span>
<span><a href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.URL}}{{end}}</a> {{t $.Lang "position" (num $.Lang .Index) (num $.Lang .Count)}}</span>
{{- with .Next}}<a rel="next" href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}}</a>{{end}}
</nav>
{{- else if .Members}}
@@ -15,7 +15,7 @@
<ol class="archive">
{{- range .Members}}
<li><a href="{{.URL}}">{{if .Title}}{{.Title}}{{else}}{{.Key}}{{end}}</a>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{.Date.Format "2 January 2006"}}</time>{{end}}</li>
{{- if not .Date.IsZero}} <time datetime="{{.Date.Format "2006-01-02"}}">{{day $.Lang .Date}}</time>{{end}}</li>
{{- end}}
</ol>
</nav>