bdeshi/khosra
0
0
Fork 0
Code Issues Pull Requests Releases Activity

make merging the default include model

Browse Source 
Composing a page from several files is why includes exist, and one endnote list
at its end is what that page wants. ADR-0066 kept the old default only so
existing content would not re-render; both models have now been seen and the
human chose. `include: embed` still asks for the other thing.

One-level needed enforcing on the new path: a spliced fragment's own include
line would otherwise be parsed as a call and expanded a second time, which a
test caught. It is dropped during the splice, matching what embed already did.
This commit is contained in:
Claude Opus 5
2026-08-02 00:25:10 +06:00
committed by bdeshi
parent 67defae912
commit aa9bd2a489
8 changed files with 64 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/content-model.md
+5 -5
View File
@@ -94,7 +94,7 @@ readable by templates (ADR-0002). Never add a required field.
| `view` | string | Per-bundle View override (Arc 2) |
| `styles` / `scripts` | []string | Page-specific assets, relative to the bundle |
| `lang` | string | Explicit language when the filename cannot carry it |
| `include` | string | `merge` splices this bundle's `::include` files into it before parsing, so the page is one document: one footnote list at the end, abbreviations reaching every part, every heading in the contents list. Absent or anything else means each fragment stays its own document (ADR-0066) |
| `include` | string | `embed` makes each `::include` file its own document, with its footnote ids namespaced and its notes rendered where it sits. Absent — the default — splices the files in before parsing, so the page is one document: one footnote list at the end, abbreviations reaching every part, every heading in the contents list (ADR-0066, ADR-0076) |
## Post types
@@ -399,10 +399,10 @@ rather than opening another, the same limit an include carries. `note`, `warn` a
expandable panel, and tabs when siblings share a `group` — and `aside`, a margin note. A kind the theme does
not define renders its body unwrapped rather than losing it.
**Merging instead of embedding.** With `include: merge` in frontmatter the fragments are spliced into the
source before anything is parsed, which is what a page assembled from several files usually wants: one
endnote list at the end rather than one per part (ADR-0066). The cost is that a fragment is no longer
contained — an unclosed code fence in a part affects the whole page, as textual inclusion always does.
**Merging is the default.** Fragments are spliced into the source before anything is parsed, which is what a
page assembled from several files nearly always wants: one endnote list at the end rather than one per part
(ADR-0066, ADR-0076). The cost is that a fragment is not contained — an unclosed code fence in a part affects
the whole page, as textual inclusion always does. `include: embed` asks for containment instead.
**Migrating from the retired form.** `{{< name key="value" >}}` is no longer a call and renders as literal
text. `khosra check` reports every one as fatal and names the replacement, so a site root is migrated by
docs/decisions.md
+14
View File
@@ -1269,3 +1269,17 @@ is a taste rather than a demonstration, kept to eight classes for that reason.
Revisit if: the binary size becomes the thing people notice, in which case chroma can build with fewer
lexers, or the feature can be dropped for `class="language-x"` and nothing else — the fragment would not
change.
## ADR-0076 — Merging is the default include model
Date: 2026-08-02 · Status: accepted (reverses ADR-0066's default; both models remain)
Decision: `::include` splices before the parse unless the bundle says `include: embed`. A fragment's own
include line is dropped during the splice, which is the one-level rule the embedded model already keeps.
Why: composing a page from several files is the reason includes exist, and one endnote list at the end is
what that page wants. ADR-0066 kept the old default only so existing content would not re-render; the human
has now looked at both and chosen. Embedding stays for the case it is better at — containing a malformed
fragment.
Consequence: cheap — the common case needs no frontmatter, and the mid-article footnote block is now
something you opt into. Expensive — an unclosed fence in a fragment affects the whole page by default, and
existing content that relied on per-fragment footnote lists must say `include: embed`.
Revisit if: containment turns out to matter more often than composition, which would be evidence the
defaults are the wrong way round again.
docs/state.md
+2 -2
View File
@@ -60,7 +60,7 @@ polls four times a second (ADR-0056).
`-site` (ADR-0011). `site.yaml` declares `base` and `title`; with a base, canonical, hreflang and OpenGraph
URLs go absolute (ADR-0039).
Frontmatter the parser lifts today: `title`, `date`, `tags`, `aliases`, `order`, `slug`. `include: merge` is read from `Extra` by the renderer (ADR-0066). Every other key in
Frontmatter the parser lifts today: `title`, `date`, `tags`, `aliases`, `order`, `slug`. `include: embed` is read from `Extra` by the renderer; merging is the default (ADR-0066, ADR-0076). Every other key in
`content-model.md`'s table — including `draft` and `type` — lands in `Extra` unread, so that table
is the accepted format, not a list of what runs.
@@ -124,7 +124,7 @@ with a stated reason. A list nothing drains is a graveyard of known defects.
| Sequence resolution rescans the index on every bundle request — two passes over every key, each doing a `Lookup` | Measured at the same time as the pictures (ADR-0044): a whole page is ~63µs, so this is not what costs anything. Remembering it would be a cache with no measurement behind it | A page render exceeding a few milliseconds, which is also what would revive the parked cache model |
| The root listing's `<title>` repeats itself — "A Khosra Demo · A Khosra Demo" | Spotted 2026-08-01 by looking at the served page, not by any test: `base.html` joins page title and site title unconditionally, and at the root those are the same string. Cosmetic, and the fix is one `if` in a template — theme layer, not engine | The next time `base.html` is edited for any reason — its old trigger named queue entry G4, which has been dropped |
| The theme and the index are two separate `atomic.Pointer` stores, so a request landing between them sees a new theme with the previous index | Accepted 2026-08-01 with ADR-0056: both halves are internally coherent and the gap is microseconds, so no page is ever internally inconsistent — it is simply not a snapshot of the disk. Closing it means one pointer holding both, which changes `web.Handler`'s signature and 20 test construction sites | Anything that makes the gap observable — a request rate high enough to land in it, or a feature where content and theme must agree exactly (an export, where every page is generated in one pass) |
| Under the default include model, a fragment's footnotes render where the include sits, so a long one puts an `<hr>` and a numbered list mid-article | Spotted 2026-08-01 by looking at the served page, not by any test. It is ADR-0038's documented consequence, and the ids are correctly namespaced (ADR-0058); only the placement reads badly. **An author who minds now says `include: merge`** (ADR-0066), which makes the page one document and puts every note at its end, so this is a default rather than a limit | The default itself proving wrong — a site where every composed page sets the flag, at which point the flag is the wrong way round |
| Under `include: embed`, a fragment's footnotes render where the include sits, so a long one puts an `<hr>` and a numbered list mid-article | Spotted 2026-08-01 by looking at the served page, not by any test. It is ADR-0038's documented consequence, and the ids are correctly namespaced (ADR-0058); only the placement reads badly. Merging is the default since ADR-0076, so this is now something an author opts into by asking for containment | Nothing: it is the documented cost of the model you chose |
## Open questions
docs/surface.md
+12 -11
View File
@@ -175,9 +175,9 @@ doc.go 8 · scaffold.go 94
- scaffold.go:76 func titleFrom(key string) string
- scaffold.go:85 func mkdirAll(root *os.Root, dir string) error
## internal/ext/shortcodes — 1311 lines + 842 test
## internal/ext/shortcodes — 1330 lines + 842 test
code.go 211 · containers.go 173 · doc.go 7 · icons.go 129 · images.go 241 · memo.go 73 · shortcodes.go 382 · toc.go 95
code.go 211 · containers.go 192 · doc.go 7 · icons.go 129 · images.go 241 · memo.go 73 · shortcodes.go 382 · toc.go 95
- code.go:27 const codeFragment = "code"
- code.go:30 var codeKind = ast.NewNodeKind("ShortcodeCode")
@@ -205,7 +205,8 @@ code.go 211 · containers.go 173 · doc.go 7 · icons.go 129 · images.go 241 ·
- containers.go:84 func (b bodies) Transform(doc *ast.Document, reader text.Reader, pc parser.Context)
- containers.go:108 func (f fragments) renderContainer(w util.BufWriter, source []byte, n ast.Node, entering bool) (ast.WalkStatus, error)
- containers.go:136 func Merge(src []byte, origin render.Origin) []byte
- containers.go:165 func included(origin render.Origin, name string) ([]byte, error)
- containers.go:165 func withoutIncludes(body []byte) []byte
- containers.go:184 func included(origin render.Origin, name string) ([]byte, error)
- icons.go:19 const iconFragment = "icon"
- icons.go:27 type icons struct{}
- icons.go:29 func (icons) Trigger() []byte { return []byte{' '} }
@@ -284,9 +285,9 @@ doc.go 8 · watch.go 125
- watch.go:90 func record(sum hash.Hash, p string, d fs.DirEntry, err error) error
- watch.go:113 func dropping(name string) bool
## internal/render — 756 lines + 489 test
## internal/render — 757 lines + 489 test
chrome.go 115 · render.go 456 · view.go 185
chrome.go 115 · render.go 457 · view.go 185
- chrome.go:19 var chrome = map[string]map[string]string{
- chrome.go:38 var months = map[string][]string{
@@ -315,12 +316,12 @@ chrome.go 115 · render.go 456 · view.go 185
- render.go:267 func (r *Renderer) Extras(b content.Bundle, served string, entries []content.Entry, selected *Selected) ([]byte, error)
- render.go:288 func (r *Renderer) RenderText(kind string, data []byte) (template.HTML, error)
- render.go:306 func (r *Renderer) Bundle(b content.Bundle, served string, variants []string, seq *content.Sequence) ([]byte, error)
- render.go:350 func (r *Renderer) Listing(section, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:369 func (r *Renderer) Tag(section, slug, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:396 func (r *Renderer) sequence(seq *content.Sequence, lang string) *Sequence
- render.go:423 func (r *Renderer) item(b content.Bundle, lang string) Item
- render.go:428 func (r *Renderer) paginate(title, lang, canonical string, all []content.Bundle, page int, url func(int) string) (List, []content.Bundle)
- render.go:450 func (r *Renderer) execute(set *template.Template, data any, what string) ([]byte, error)
- render.go:351 func (r *Renderer) Listing(section, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:370 func (r *Renderer) Tag(section, slug, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:397 func (r *Renderer) sequence(seq *content.Sequence, lang string) *Sequence
- render.go:424 func (r *Renderer) item(b content.Bundle, lang string) Item
- render.go:429 func (r *Renderer) paginate(title, lang, canonical string, all []content.Bundle, page int, url func(int) string) (List, []content.Bundle)
- render.go:451 func (r *Renderer) execute(set *template.Template, data any, what string) ([]byte, error)
- view.go:17 type head struct
- view.go:37 type Page struct
- view.go:56 type Sequence struct
examples/demo-site/content/writing/notes-on-water/index.en.md
+1 -2
View File
@@ -2,7 +2,6 @@
title: Notes on Water
date: 2026-03-25
tags: [monsoon, journal]
include: merge
---
The finished piece. Below, a part of it lives in a separate file and is included here:
@@ -11,7 +10,7 @@ The finished piece. Below, a part of it lives in a separate file and is included
That fragment starts with an underscore, so the scanner never treats it as a bundle: it has no URL of its own
and appears in no listing. An included file cannot itself include — one level, deliberately.
This bundle declares `include: merge`, so the fragment is spliced in before anything is parsed: its footnote
Fragments are spliced in before anything is parsed: its footnote
is numbered with the page's and both appear in one list at the end, rather than one list per part.
This bundle also has an `extras/` directory, so the theme offers a link to it at the foot of the page.
internal/ext/shortcodes/containers.go
+20 -1
View File
@@ -154,12 +154,31 @@ func Merge(src []byte, origin render.Origin) []byte {
slog.Error("skipping include", "file", args["file"], "err", err)
continue
}
out.Write(body)
out.Write(withoutIncludes(body))
out.WriteByte('\n')
}
return out.Bytes()
}
// withoutIncludes drops a fragment's own include lines. One level, the same rule the embedded model keeps:
// without this the spliced line would be parsed as a call and expanded a second time (ADR-0038).
func withoutIncludes(body []byte) []byte {
var out bytes.Buffer
for rest := body; len(rest) > 0; {
line, remainder, found := bytes.Cut(rest, []byte("\n"))
rest = remainder
if name, _, ok := parse(string(line), opener); ok && name == "include" {
slog.Error("ignoring an include inside an included file", "line", string(line))
continue
}
out.Write(line)
if found {
out.WriteByte('\n')
}
}
return out.Bytes()
}
// included reads one fragment, refusing a name that would leave the bundle — the same rule the embedded path
// enforces, and for the same reason: an include must not publish a template or a dotfile.
func included(origin render.Origin, name string) ([]byte, error) {
internal/ext/shortcodes/containers_test.go
+6 -6
View File
@@ -47,7 +47,7 @@ func TestAContainerDoesNotSwallowTheRestOfThePage(t *testing.T) {
// The reason the flag exists: a page built from several files should have one endnote list, at the end
// (ADR-0066).
func TestMergeGivesThePageOneFootnoteList(t *testing.T) {
got := bundle(t, mergeFS("include: merge\n"), "posts/composed")
got := bundle(t, mergeFS(""), "posts/composed")
if n := strings.Count(got, `class="footnotes"`); n != 1 {
t.Errorf("want one endnote list, got %d:\n%s", n, got)
}
@@ -62,9 +62,9 @@ func TestMergeGivesThePageOneFootnoteList(t *testing.T) {
}
}
// The default is untouched, which is what makes the flag safe to add.
func TestWithoutTheFlagEachFragmentKeepsItsOwnNotes(t *testing.T) {
got := bundle(t, mergeFS(""), "posts/composed")
// `include: embed` is the other model, and still reachable.
func TestEmbedKeepsEachFragmentsOwnNotes(t *testing.T) {
got := bundle(t, mergeFS("include: embed\n"), "posts/composed")
if n := strings.Count(got, `class="footnotes"`); n != 3 {
t.Errorf("embed is still one list per document, got %d:\n%s", n, got)
}
@@ -75,7 +75,7 @@ func TestWithoutTheFlagEachFragmentKeepsItsOwnNotes(t *testing.T) {
func TestMergeRefusesToLeaveTheBundle(t *testing.T) {
fsys := fstest.MapFS{
"content/posts/p/index.md": {Data: []byte("---\ntitle: P\ninclude: merge\n---\n::include{file=../../../secret.md}\n")},
"content/posts/p/index.md": {Data: []byte("---\ntitle: P\n---\n::include{file=../../../secret.md}\n")},
"secret.md": {Data: []byte("SECRET\n")},
}
if got := bundle(t, fsys, "posts/p"); strings.Contains(got, "SECRET") {
@@ -94,7 +94,7 @@ func TestRepeatedHeadingsAreSuffixedNotDuplicated(t *testing.T) {
"content/posts/c/_two.md": {Data: []byte("## Description\n\nTwo.\n")},
}
}
for _, model := range []string{"", "include: merge\n"} {
for _, model := range []string{"", "include: embed\n"} {
got := bundle(t, fs(model), "posts/c")
for _, want := range []string{`id="description"`, `id="description-1"`, `id="description-2"`} {
if !strings.Contains(got, want) {
internal/render/render.go
+4 -3
View File
@@ -309,10 +309,11 @@ func (r *Renderer) Bundle(b content.Bundle, served string, variants []string, se
pc := parser.NewContext()
origin := Origin{Dir: path.Dir(b.Path), Files: r.files, Lang: served}
WithOrigin(pc, origin)
// `include: merge` asks for one document rather than a page of embedded ones, so the fragments are
// spliced in before the parse and their footnotes, abbreviations and headings become the page's (ADR-0066).
// Fragments are spliced in before the parse, so their footnotes, abbreviations and headings are the
// page's — one document, which is what composing a page from files nearly always wants. `include: embed`
// asks for the other thing: each fragment its own document (ADR-0066, ADR-0076).
source := b.Body
if kind, _ := b.Extra["include"].(string); kind == "merge" && r.compose != nil {
if kind, _ := b.Extra["include"].(string); kind != "embed" && r.compose != nil {
source = r.compose(source, origin)
}
var body bytes.Buffer