make context a budget the harness enforces

Adopts ideas/token-conservation.md, parked 2026-07-28, plus the disciplines
the human added: read the compressed form first, discover by mechanism,
shrink output at the source, never pay twice for the same bytes.

docs/context-economy.md owns all of it and leads with a floor, because
every cheap failure mode is also a token saving — skipping the owning doc,
guessing a signature, reporting from a diff, thinning a test — and each has
already cost this repo a defect. Frugality is for presentation and
discovery, never for the artifact or the evidence.

Mechanical, not remembered:

- scripts/surface.sh generates docs/surface.md — every top-level
  declaration with its line, 261 lines standing for 3757 of source. The
  pre-commit hook regenerates and stages it, so it cannot be stale, and
  verify.sh compares independently for a clone that never set
  core.hooksPath. The hook refuses a commit with unstaged .go changes,
  since what it generated describes the working tree, not the commit.
- verify.sh --quiet: 48 lines of gate output become 1. The hook uses it.
- CLAUDE_LOC_MAX=150, the only budget billed per turn rather than per read.

Both new gates were watched failing before being kept: a doctored
surface.md, and CLAUDE_LOC_MAX temporarily set to 5.

state.md's inventory loses its LOC column. It had already drifted on six
files (content.go 381→450, render.go 447→454, web.go 206→217, check
216→223, watch 129→137, chrome 105→110) which is what a number written in
two places does; the generated file owns sizes now, the table owns purpose.
The subagent question is recorded there as the one open decision, with the
case for and against written out in the idea file.
This commit is contained in:
Claude Opus 5
2026-08-01 02:16:11 +06:00
committed by bdeshi
parent 8256c72180
commit 7796b0d919
15 changed files with 597 additions and 43 deletions
+8 -1
View File
@@ -29,6 +29,8 @@ a layout or markup decision dressed as an engine feature. |
## 1. Clarify ## 1. Clarify
Read `docs/state.md`, then `docs/README.md` — the map, always, before deciding what else to open. Read `docs/state.md`, then `docs/README.md` — the map, always, before deciding what else to open.
For code, read the compressed form before the source: `docs/surface.md` locates every declaration,
`go doc` gives a package's surface, a test states its contract. Disciplines: `docs/context-economy.md`.
Use its topic-ownership table to list the docs owning anything this change asserts a rule about, Use its topic-ownership table to list the docs owning anything this change asserts a rule about,
and read those. That list is a floor: skipping it is how a rule gets written twice and the second and read those. That list is a floor: skipping it is how a rule gets written twice and the second
copy contradicts the first. copy contradicts the first.
@@ -96,7 +98,9 @@ If the plan reveals a trunk, say so instead of planning and offer the leaf.
## 4. Verify ## 4. Verify
Run `./scripts/verify.sh` — doc coupling, format, vet, build, test, dependency allowlist, budgets. Run `./scripts/verify.sh --quiet` — doc coupling, format, vet, build, test, dependency allowlist,
budgets. Quiet prints only failures, warnings and the verdict; use the loud form when a gate fails and
you want its neighbours, or when auditing the gates themselves.
Then **at least one piece of feature-specific evidence you actually executed**: the new test name Then **at least one piece of feature-specific evidence you actually executed**: the new test name
and its output, `curl -s localhost:PORT/path | head`, a golden-file diff, the before/after fragment and its output, `curl -s localhost:PORT/path | head`, a golden-file diff, the before/after fragment
@@ -207,5 +211,8 @@ Swept: the old form you grepped for after a rename, or "n/a"
| Treated a one-sentence request as a narrow change | Articulation size is not blast radius | Walk the propagation table | | Treated a one-sentence request as a narrow change | Articulation size is not blast radius | Walk the propagation table |
| Splitting the difference between request and policy | A compromise nobody chose | Surface it; the human picks | | Splitting the difference between request and policy | A compromise nobody chose | Surface it; the human picks |
| Ending a turn with the work only in the working tree | The one copy is the one that gets lost | Commit before reporting | | Ending a turn with the work only in the working tree | The one copy is the one that gets lost | Commit before reporting |
| Opening a file to find out what is in it | The generated surface already answers it | `docs/surface.md`, then read the range |
| Grepping for callers before changing a signature | The compiler enumerates them exactly | Change it, then `go build ./...` |
| A shorter report that dropped a caveat, case or number | Truncation wearing compression's clothes | Restore it; cut words, never findings |
| One commit per file, or per doc touched | Shredding a single revertible unit | Bundle what would be undone together | | One commit per file, or per doc touched | Shredding a single revertible unit | Bundle what would be undone together |
| Committing without ever saying you would | Consent assumed, not given | The notice, once, before the first one | | Committing without ever saying you would | Consent assumed, not given | The notice, once, before the first one |
+7 -1
View File
@@ -24,6 +24,12 @@ disagree, the code wins — say so, fix the doc in Document.
writing. Before stating a rule, contract, threshold, or gate, read its owning doc; if it already writing. Before stating a rule, contract, threshold, or gate, read its owning doc; if it already
says it, amend there instead of restating elsewhere. says it, amend there instead of restating elsewhere.
**Read the compressed form first.** `docs/surface.md` (generated, gated) says where every declaration
lives; `go doc ./internal/<pkg>` gives a package's surface; a test states a contract in a fifth of the
lines that implement it. Locate with `grep -n`, then read that range — a whole file is for the doc that
owns a rule you are asserting, or code you are about to rewrite. Every discipline, and the floor none of
them may cross: `docs/context-economy.md`. A saving that buys a guess is not a saving.
`ideas/` and `reference/` are out of context by default, indexes included. Open one only when the `ideas/` and `reference/` are out of context by default, indexes included. Open one only when the
human names it. Never sweep, never list, never cite unasked. Storage, not background. human names it. Never sweep, never list, never cite unasked. Storage, not background.
@@ -86,7 +92,7 @@ Procedure: `.claude/skills/khosra-feature-loop/SKILL.md`. The three gates people
batched, up front, each with a **bold** default so silence answers. Never about naming, formatting, batched, up front, each with a **bold** default so silence answers. Never about naming, formatting,
or anything `docs/conventions.md` decides. None to ask? State assumptions in one line and move on. or anything `docs/conventions.md` decides. None to ask? State assumptions in one line and move on.
**Verify.** `./scripts/verify.sh` green, plus one piece of feature-specific evidence you actually **Verify.** `./scripts/verify.sh --quiet` green, plus one piece of feature-specific evidence you actually
ran (golden file, `curl`, test name, benchmark). Never report success from reading your own diff. ran (golden file, `curl`, test name, benchmark). Never report success from reading your own diff.
"Should work" is not a result. "Should work" is not a result.
+17 -2
View File
@@ -70,6 +70,18 @@ change detection. Treat a *second* raise as evidence that something belongs in `
two ceilings exist so that "core stops growing, ext rises" is observable, and that stops being true the moment two ceilings exist so that "core stops growing, ext rises" is observable, and that stops being true the moment
leaves are allowed into core. leaves are allowed into core.
**Context is a budget, and three pieces of it are mechanical.** The limit on this project is how much
work fits in a session, so `docs/context-economy.md` holds the reading, searching and reporting
disciplines — read the compressed form first, batch calls, script anything repeatable, never pay twice
for the same bytes (ADR-0053). Three parts are enforced rather than trusted: `docs/surface.md` is
regenerated and staged by the pre-commit hook and independently compared by `verify.sh`, so the map of
the code cannot mislead (261 lines standing for 3,757 of source) and no one has to remember to run it;
`CLAUDE_LOC_MAX` bounds the one file re-sent on every turn;
and `./scripts/verify.sh --quiet` prints a verdict instead of forty-three lines, which is what the
pre-commit hook now runs. The doc leads with a floor rather than the savings, because every cheap
failure mode is also a token saving — not opening the owning doc, guessing a signature, reporting
from a diff — and each of those has already cost this repo a defect.
**The agent commits by itself, and tells you the first time.** Work is committed as part of the loop **The agent commits by itself, and tells you the first time.** Work is committed as part of the loop
rather than on request (ADR-0052) — one commit per revertible unit, never a `push`. In a session where rather than on request (ADR-0052) — one commit per revertible unit, never a `push`. In a session where
you have not asked for that, the first commit is preceded by a line saying so and naming the off you have not asked for that, the first commit is preceded by a line saying so and naming the off
@@ -97,7 +109,8 @@ In order, cheapest first:
1. `./scripts/verify.sh` — one command, tells you whether the thing is still coherent and whether 1. `./scripts/verify.sh` — one command, tells you whether the thing is still coherent and whether
`docs/state.md` has fallen behind the code. `docs/state.md` has fallen behind the code.
2. `docs/state.md` — what exists, the earn-it counters, the latent list, the commit it was last verified 2. `docs/state.md` — what exists, the earn-it counters, the latent list, the commit it was last verified
against. This is the only doc that describes the present. against. This is the only doc that describes the present. Its companion `docs/surface.md` is generated:
every declaration and its line, so you can find your way around without opening anything.
3. `git log --oneline` — one revertible unit per commit, each body saying *why* (`conventions.md`). This is the 3. `git log --oneline` — one revertible unit per commit, each body saying *why* (`conventions.md`). This is the
real map of how the code got here. real map of how the code got here.
4. `/refresh-docs` — reconciles every doc against the actual code and reports drift, which is exactly the 4. `/refresh-docs` — reconciles every doc against the actual code and reports drift, which is exactly the
@@ -193,7 +206,9 @@ that describe it, in the same change.**
`gofmt`-only diff is exempt: it ships no behaviour, and failing it would only teach you `--no-verify`. `gofmt`-only diff is exempt: it ships no behaviour, and failing it would only teach you `--no-verify`.
- The gate is not optional. `scripts/hooks/pre-commit` runs it on every commit; enable once per clone - The gate is not optional. `scripts/hooks/pre-commit` runs it on every commit; enable once per clone
with `git config core.hooksPath scripts/hooks`. `--no-verify` bypasses it, and the commit body should with `git config core.hooksPath scripts/hooks`. `--no-verify` bypasses it, and the commit body should
say why. say why. The hook also regenerates `docs/surface.md` and stages it, so a generated file is never
something you have to remember — and it refuses a commit with unstaged `.go` changes, because what it
generated describes the working tree rather than what you are committing.
- `docs/decisions.md` registers every ADR number ever used, entries and withdrawals alike, so a citation - `docs/decisions.md` registers every ADR number ever used, entries and withdrawals alike, so a citation
can resolve to a decision or to a deferral but never to nothing. can resolve to a decision or to a deferral but never to nothing.
- `./scripts/verify.sh --list` names every gate that exists. A doc claiming enforcement is checkable - `./scripts/verify.sh --list` names every gate that exists. A doc claiming enforcement is checkable
+6
View File
@@ -35,6 +35,12 @@ test: ## run tests with the race detector
verify: ## everything the harness enforces; must be green before a commit verify: ## everything the harness enforces; must be green before a commit
./scripts/verify.sh ./scripts/verify.sh
quiet: ## the same gates, printing only what needs acting on
./scripts/verify.sh --quiet
surface: ## regenerate docs/surface.md, the compressed map of the code
./scripts/surface.sh --write
fmt: ## format all Go source fmt: ## format all Go source
gofmt -w . gofmt -w .
+5 -1
View File
@@ -14,6 +14,8 @@ Read the one you need. Do not read them all.
| `exploration.md` | Catalog of possible future *engine features*, leaf/trunk verdicts | When a verdict is reached | | `exploration.md` | Catalog of possible future *engine features*, leaf/trunk verdicts | When a verdict is reached |
| `toolchain.md` | Versions and agent-tooling contracts this was built against | When you deliberately move a version | | `toolchain.md` | Versions and agent-tooling contracts this was built against | When you deliberately move a version |
| `theme-contract.md` | What the engine promises a theme; the only theme-facing obligation | When the contract is extended — additively only | | `theme-contract.md` | What the engine promises a theme; the only theme-facing obligation | When the contract is extended — additively only |
| `context-economy.md` | How the agent spends context: what to read, in what form, and the floor no saving may cross | When a technique or a gate for one changes |
| `surface.md` | **Generated.** Every top-level declaration with its line — read before opening source | Never by hand; `make surface` |
Two sibling folders sit **outside** `docs/` because they are storage, not working material: Two sibling folders sit **outside** `docs/` because they are storage, not working material:
`../ideas/` (parked ideas, any topic) and `../reference/` (durable facts from conversation). Write `../ideas/` (parked ideas, any topic) and `../reference/` (durable facts from conversation). Write
@@ -41,6 +43,7 @@ that is the finding to report.
| what to build next | `.scratch/build-queue.md` if present, otherwise `git log` and `roadmap.md` | | what to build next | `.scratch/build-queue.md` if present, otherwise `git log` and `roadmap.md` |
| how the harness works — a gate, counter, budget, the loop | `HARNESS.md` + `scripts/verify.sh` + `CLAUDE.md` | | how the harness works — a gate, counter, budget, the loop | `HARNESS.md` + `scripts/verify.sh` + `CLAUDE.md` |
| what one commit contains, and when one happens | `conventions.md` "Git" (shape) + `CLAUDE.md` §4 (cadence, ADR-0052) | | what one commit contains, and when one happens | `conventions.md` "Git" (shape) + `CLAUDE.md` §4 (cadence, ADR-0052) |
| how to read, search, or report without wasting context | `context-economy.md` (+ ADR-0053) |
| a tool version, or why agent tooling stopped working | `toolchain.md` | | a tool version, or why agent tooling stopped working | `toolchain.md` |
| templates, layout, presentation, what a theme can rely on | `theme-contract.md` (+ ADR-0019, ADR-0023) | | templates, layout, presentation, what a theme can rely on | `theme-contract.md` (+ ADR-0019, ADR-0023) |
@@ -55,7 +58,7 @@ here; nothing else may state the value:
| Fact | Sole home | | Fact | Sole home |
|---|---| |---|---|
| Earn-it thresholds (transforms, routes, views, extensions) | the counters table in `state.md` | | Earn-it thresholds (transforms, routes, views, extensions) | the counters table in `state.md` |
| LOC ceilings, size warnings, dependency cap | `scripts/budgets.env` | | LOC ceilings, size warnings, dependency cap, the `CLAUDE.md` ceiling | `scripts/budgets.env` |
| The permalink path shape | `content-model.md` (decision recorded in ADR-0008) | | The permalink path shape | `content-model.md` (decision recorded in ADR-0008) |
| Which language is at the root, and the prefix form | ADR-0009 | | Which language is at the root, and the prefix form | ADR-0009 |
| The leaf/trunk definition | `architecture.md` invariant 7 | | The leaf/trunk definition | `architecture.md` invariant 7 |
@@ -91,6 +94,7 @@ Update in the Document step of the change that caused them. Not later.
| The theme contract extended, or embedded templates changed | `theme-contract.md`, same change. Enforced by `verify.sh`. | | The theme contract extended, or embedded templates changed | `theme-contract.md`, same change. Enforced by `verify.sh`. |
| A request spans engine and theme | contract extension here + a written note of the theme's part; never the theme itself | | A request spans engine and theme | contract extension here + a written note of the theme's part; never the theme itself |
| A new gate, counter, or budget added | `HARNESS.md` "Why the pieces exist" + a row in one of these tables | | A new gate, counter, or budget added | `HARNESS.md` "Why the pieces exist" + a row in one of these tables |
| A top-level declaration added, renamed or moved | nothing by hand — `make surface`, which `verify.sh` then compares |
| A rule, value, name or path changed | grep the repo for the old form; every doc naming the concept, same change | | A rule, value, name or path changed | grep the repo for the old form; every doc naming the concept, same change |
| The disk contract changed | `content-model.md` + `testdata/` fixtures + a written migration step for the site root, which the engine repo cannot touch | | The disk contract changed | `content-model.md` + `testdata/` fixtures + a written migration step for the site root, which the engine repo cannot touch |
| A doc or ADR mandates what a change made optional | supersede the ADR; do not reword it in place | | A doc or ADR mandates what a change made optional | supersede the ADR; do not reword it in place |
+87
View File
@@ -0,0 +1,87 @@
# Context economy
How the agent spends context. The limit on this project is not typing speed, it is how much work fits
in a session before the window fills — so bytes that buy nothing are features not built.
**This doc has a floor and the floor wins.** Every technique below is forbidden from buying:
skipping the doc that owns a rule (`CLAUDE.md` §1), guessing an API instead of reading it, reporting
success from a diff instead of a run (`SKILL.md` §4), keeping a test that was never seen to fail, or
thinning generated code, content, tests, comments or commit messages. Those are all cheaper in tokens
and every one of them has cost this project a defect. Frugality applies to **presentation and
discovery**, never to the artifact or the evidence. When a saving and the floor disagree, pay.
## 1. Fewer turns beats fewer bytes
The whole context is re-sent on every tool call, so a six-call sequence costs roughly six times the
window. Batch independent calls into one message. Where a sequence is known in advance or will be run
again, write the script instead: one artifact that sets up, runs, probes and prints its conclusion —
the live-reload check that measured the watcher is the shape to copy, and `verify.sh` is the same
technique applied to the gates.
## 2. Read the compressed form first
In order of cost, stop at the first that answers the question:
| Question | Cheapest answer |
|---|---|
| where does X live, what is in this package | `docs/surface.md` (generated; `make surface`) |
| what does this package offer callers | `go doc ./internal/<pkg>` |
| what does this behave like | its `_test.go` — a table states in 20 lines what 200 implement |
| why is it like this | `decisions.md`, `state.md` — never the code |
| what changed and when | `git log --oneline -- <path>` |
| how big is this before I open it | `wc -l`, or the manifest line in `surface.md` |
Then read the range, not the file: `grep -n` to locate, `Read` with `offset`/`limit`. A whole-file read
is for the doc that owns a rule you are about to assert, and for a file you are about to rewrite.
## 3. Let the machine find it
Discovery by mechanism beats discovery by reading.
- **Break it and read the errors.** Changing a signature: `go build ./...` returns the exact caller
list, complete, in a form a grep can miss.
- `scripts/verify.sh --list` answers "does this gate exist" without running or reading anything.
- `grep -c` for "does this exist", `-l` for "which files", `-n` for "where". Context flags only when
the surrounding lines *are* the answer.
- `go test -run TestName` over a full suite while iterating; the full suite once, at the end.
## 4. Shrink output at the source
Filtering inside the call is free. Filtering after it lands is impossible.
- `./scripts/verify.sh --quiet` — two lines on green instead of forty-three. The pre-commit hook uses
it. Use the loud form when a gate fails or you are auditing the gates themselves.
- Quiet and fail-fast flags by default: `-q`, `--porcelain`, `--short`, `--oneline`, `-failfast`,
`--stat` before `-p`, and `2>/dev/null` for known noise.
- Pipe through `tail`/`grep -v`/`awk` in the same call. Ten relevant lines, not four hundred.
- Long or noisy runs write to the scratchpad and get queried; never `cat` a lock file, a binary, or
generated output.
- Sample rather than enumerate: two bundles out of thirty answer whether the shape holds.
## 5. Never pay for the same bytes twice
- No re-reading after `Edit` or `Write` — they fail loudly if they did not apply.
- A background command's output is read once, after it completes. Not while polling, and not quoted
back afterwards.
- No restating the plan, then doing it, then summarising it. State once, report deltas.
- Write a fact down where it belongs the first time it is established (`state.md`, an ADR, a resume
note in the queue file) so the next session reads a line instead of re-deriving it. A handoff note
is cheaper than a compaction summary and loses less.
- Report `file:line`, never the code — the human has the file, and the reference is clickable.
## 6. Prose is the cheapest thing to cut and the easiest to overcut
Target density, not brevity: the same findings, numbers and caveats in fewer words. Cut preamble,
recap, restatement, option surveys, and hedging. Do not cut a finding, a measurement, a stated
assumption, or a caveat — a report that omits the caveat is not shorter, it is wrong.
## What is enforced, and what is not
Three of these are mechanical. `scripts/hooks/pre-commit` regenerates `docs/surface.md` and stages it,
so no commit can carry a stale one, and `verify.sh` compares independently for the clone that never set
`core.hooksPath`. `CLAUDE_LOC_MAX` bounds the one file billed on every turn. `--quiet` exists to be
used, and the hook uses it.
Everything else here is discipline — no script can see a redundant read, an unnecessary whole-file
Read, or a report that padded instead of informing. That asymmetry is the reason this doc is short
enough to be re-read, and the reason its floor is stated first.
+25
View File
@@ -788,3 +788,28 @@ said. `conventions.md` "Git" owns what one commit contains; `CLAUDE.md` §4 owns
Revisit if: the log fills with commits nobody would revert separately — then the unit is wrong, not the Revisit if: the log fills with commits nobody would revert separately — then the unit is wrong, not the
automation; or the human wants a review gate before anything lands, which is a different default, not a automation; or the human wants a review gate before anything lands, which is a different default, not a
tweak to this one. tweak to this one.
## ADR-0053 — Context economy is a doc with a floor, plus three gates
Date: 2026-08-01 · Status: accepted (adopts `ideas/token-conservation.md`)
Decision: `docs/context-economy.md` owns how the agent spends context — read the compressed form first,
fewer turns before fewer bytes, script anything repeatable, shrink output at the source, never pay twice
for the same bytes. Three parts are mechanical rather than remembered: `scripts/surface.sh` generates
`docs/surface.md`, the pre-commit hook regenerates and stages it while `verify.sh` compares it
independently, `CLAUDE_LOC_MAX=150` bounds the file re-sent every turn, and `verify.sh --quiet` prints
only what needs acting on (the hook uses it). Generated artifacts are produced by the hook, never by
memory; the hook refuses a commit with unstaged `.go` changes, since what it generated describes the
working tree rather than the commit.
Why: the binding constraint is context per session, not typing speed, so wasted bytes are features not
built. The measurements that decided the shape: `go doc` on `internal/content` is 38 lines against 1,579
of source, and `surface.md` is 261 against 3,757. The doc leads with a floor because every cheap failure
mode is also a saving — skipping the owning doc, guessing a signature, reporting from a diff, thinning a
test — and each has already cost this repo a defect. Generated-and-gated rather than hand-written,
because an index nobody regenerates is worse than no index: it is confidently wrong.
Consequence: cheap — a green gate run is 2 lines instead of 43, "where does X live" is one grep of a
tracked file, and the disciplines are auditable in one place. Expensive — a fourth generated artifact to
keep honest, one more budget to raise deliberately, and 261 lines of committed noise in every diff that
adds a function. Deliberately *not* mechanical: nothing can detect a redundant read, so most of the doc
is discipline, and it says so instead of implying enforcement (rule 8).
Revisit if: the surface grows past a few hundred lines, in which case it is per-package files rather than
one; or `surface.md` churn starts drowning real diffs, which is the argument for generating it on demand
instead of tracking it.
+40 -29
View File
@@ -5,34 +5,39 @@ If this file disagrees with the code, the code is right and this file is a bug.
## Inventory ## Inventory
| File | Purpose | LOC | Line counts are **not** here: `docs/surface.md` is generated from the code and carries per-file sizes
|---|---|---| plus every declaration's location (ADR-0053). This column drifted on six files before it was removed,
| `go.mod` | module `khosra`; `goldmark`, `x/text`, `yaml.v3` direct | 10 | which is what a number written in two places always does. What each file is *for* is the fact this
| `internal/content/doc.go` | package comment | 5 | table owns.
| `internal/content/content.go` | bundles: `os.Root` open, walk, frontmatter split, key/lang derivation, NFC, tag slugs, partial files, permalink building | 381 |
| `internal/content/clock.go` | the one place the engine reads the wall clock, which `verify.sh` enforces by filename | 12 | | File | Purpose |
| `internal/content/extras.go` | a bundle's supporting files: enumeration, classification, and their URLs (ADR-0047) | 96 | |---|---|
| `internal/content/settings.go` | `site.yaml`: the site's own declarations (`base`, `title`) and absolute-URL building (ADR-0039) | 59 | | `go.mod` | module `khosra`; `goldmark`, `x/text`, `yaml.v3` direct |
| `internal/content/site.go` | the indexed site: lookup with language fallback, aliases, `Query` and `Run`, sections, `Sequence`, `Everything`, slug routes, publication visibility | 424 | | `internal/content/doc.go` | package comment |
| `internal/render/render.go` | goldmark with the typographer, per-kind template sets with site override, the render methods | 447 | | `internal/content/content.go` | bundles: `os.Root` open, walk, frontmatter split, key/lang derivation, NFC, tag slugs, partial files, permalink building |
| `internal/render/view.go` | the theme contract in Go: `Page`, `List`, `Sequence`, `Extras`, `Item`, `Fragment`, `Picture`, `Origin` | 130 | | `internal/content/clock.go` | the one place the engine reads the wall clock, which `verify.sh` enforces by filename |
| `internal/render/chrome.go` | the engine's own words: phrase table, month names, digits, and the `t`/`num`/`day` template funcs (ADR-0034) | 105 | | `internal/content/extras.go` | a bundle's supporting files: enumeration, classification, and their URLs (ADR-0047) |
| `internal/render/templates/` | reference theme, complete: `base.html` (shell, navigation, language links, feed and OpenGraph), `page.html` (bundle, sequence, tags, extras), `list.html`, `extras.html`, `shortcodes.html`, `theme.css` (ADR-0026, ADR-0049) | — | | `internal/content/settings.go` | `site.yaml`: the site's own declarations (`base`, `title`) and absolute-URL building (ADR-0039) |
| `internal/ext/shortcodes/` | first feature: `{{< name key="value" >}}` block parser and node renderer, rendering through a theme fragment (ADR-0036). `figure`, `gallery`, `include`, plus the derivative pass and remembered picture inspection (ADR-0042, ADR-0044) | 564 | | `internal/content/site.go` | the indexed site: lookup with language fallback, aliases, `Query` and `Run`, sections, `Sequence`, `Everything`, slug routes, publication visibility |
| `internal/ext/scaffold/` | writes one draft directory bundle into a site root through `os.Root`: never an overwrite | 102 | | `internal/render/render.go` | goldmark with the typographer, per-kind template sets with site override, the render methods |
| `internal/ext/watch/` | polls the site root, ignores editor droppings, and reports a settled change (ADR-0022, ADR-0048) | 129 | | `internal/render/view.go` | the theme contract in Go: `Page`, `List`, `Sequence`, `Extras`, `Item`, `Fragment`, `Picture`, `Origin` |
| `internal/ext/check/` | third feature: validates a site root — what the engine worked around, broken internal links, missing titles and alt text, mixed series ordering | 216 | | `internal/render/chrome.go` | the engine's own words: phrase table, month names, digits, and the `t`/`num`/`day` template funcs (ADR-0034) |
| `cmd/khosra/wire.go` | the only list of enabled features (`extensions.md`) | 20 | | `internal/render/templates/` | reference theme, complete: `base.html` (shell, navigation, language links, feed and OpenGraph), `page.html` (bundle, sequence, tags, extras), `list.html`, `extras.html`, `shortcodes.html`, `theme.css` (ADR-0026, ADR-0049) |
| `internal/web/resolve.go` | URL → (key, lang, page, tag, feed, extras) or a canonical redirect | 168 | | `internal/ext/shortcodes/` | first feature: `{{< name key="value" >}}` block parser and node renderer, rendering through a theme fragment (ADR-0036). `figure`, `gallery`, `include`, plus the derivative pass and remembered picture inspection (ADR-0042, ADR-0044) |
| `internal/web/extras.go` | the extras route: listing, one entry selected, or `?raw` bytes, all behind the bundle lookup | 96 | | `internal/ext/scaffold/` | writes one draft directory bundle into a site root through `os.Root`: never an overwrite |
| `internal/web/asset.go` | files inside a bundle's own directory, looked up through the owning bundle so visibility can only ever inherit (ADR-0024) | 58 | | `internal/ext/watch/` | polls the site root, ignores editor droppings, and reports a settled change (ADR-0022, ADR-0048) |
| `internal/web/feed.go` | Atom for the site, a section or a tag, from dated bundles via one Query (ADR-0043) | 125 | | `internal/ext/check/` | third feature: validates a site root — what the engine worked around, broken internal links, missing titles and alt text, mixed series ordering |
| `internal/web/discover.go` | `/robots.txt` and `/sitemap.xml`, absolute and only with a declared base (ADR-0039) | 74 | | `cmd/khosra/wire.go` | the only list of enabled features (`extensions.md`) |
| `internal/web/web.go` | handler: `serve` dispatches by kind, `serveBundle` answers the commonest one; listings, `/static/`, `/derived/`, degrade on failure | 206 | | `internal/web/resolve.go` | URL → (key, lang, page, tag, feed, extras) or a canonical redirect |
| `cmd/khosra/main.go` | flags, wiring, startup, the derivative pass, and the atomic swap a rebuild goes through. `main` dispatches subcommands, `runServe` assembles the server, `rebuilder` is used at startup and on every change alike | 150 | | `internal/web/extras.go` | the extras route: listing, one entry selected, or `?raw` bytes, all behind the bundle lookup |
| `cmd/khosra/check.go` | the `check` subcommand: parse, print, exit code. What counts as a finding lives in the feature | 45 | | `internal/web/asset.go` | files inside a bundle's own directory, looked up through the owning bundle so visibility can only ever inherit (ADR-0024) |
| `cmd/khosra/new.go` | the `new` subcommand: arguments in either order, then the feature does the writing | 42 | | `internal/web/feed.go` | Atom for the site, a section or a tag, from dated bundles via one Query (ADR-0043) |
| `*_test.go` | table-driven, one file per source file; symlink escape (content and static), canonical paths, language fallback, aliases, pagination, tags, sequences, chrome, typography, shortcode escaping, galleries, includes, partials, site settings, absolute URLs, robots, sitemap, slug routes, bundle assets, derivatives, feeds, 404, plus benchmarks for the render path and the checker, unpublished visibility, listing shapes, scaffolding, extras, change detection, what a page can reach, the root listing, and the example site end to end | 3233 | | `internal/web/discover.go` | `/robots.txt` and `/sitemap.xml`, absolute and only with a declared base (ADR-0039) |
| `internal/web/web.go` | handler: `serve` dispatches by kind, `serveBundle` answers the commonest one; listings, `/static/`, `/derived/`, degrade on failure |
| `cmd/khosra/main.go` | flags, wiring, startup, the derivative pass, and the atomic swap a rebuild goes through. `main` dispatches subcommands, `runServe` assembles the server, `rebuilder` is used at startup and on every change alike |
| `cmd/khosra/check.go` | the `check` subcommand: parse, print, exit code. What counts as a finding lives in the feature |
| `cmd/khosra/new.go` | the `new` subcommand: arguments in either order, then the feature does the writing |
| `*_test.go` | table-driven, one file per source file; symlink escape (content and static), canonical paths, language fallback, aliases, pagination, tags, sequences, chrome, typography, shortcode escaping, galleries, includes, partials, site settings, absolute URLs, robots, sitemap, slug routes, bundle assets, derivatives, feeds, 404, plus benchmarks for the render path and the checker, unpublished visibility, listing shapes, scaffolding, extras, change detection, what a page can reach, the root listing, and the example site end to end |
Serves a listing of everything at `/` (ADR-0050), a bundle at `/{section}/{slug}/` — the slug derived, or declared in frontmatter without moving the Serves a listing of everything at `/` (ADR-0050), a bundle at `/{section}/{slug}/` — the slug derived, or declared in frontmatter without moving the
key (ADR-0035) — a paginated listing per section, tag listings global and key (ADR-0035) — a paginated listing per section, tag listings global and
@@ -105,7 +110,13 @@ with a stated reason. A list nothing drains is a graveyard of known defects.
## Open questions ## Open questions
None. Nothing blocks Arc 1 or the first deploy. **Subagents for fan-out reads — policy, not judgment.** Delegating read-heavy sweeps (`/audit`,
`/refresh-docs`, `/invariants`, rename sweeps) keeps thousands of lines of file dumps out of the main
window and returns only the verdict. Deferred 2026-08-01; the case for and against is written out in
`ideas/token-conservation.md`, including where it is clearly right (locating things) and clearly wrong
(deciding things). Nothing blocks on it.
Nothing else blocks Arc 1 or the first deploy.
Every feature has been audited against the layer test (ADR-0046). One was wrong and was deleted (widow Every feature has been audited against the layer test (ADR-0046). One was wrong and was deleted (widow
prevention, ADR-0045); one quietly decided presentation and now offers both shapes (tag listing grouping). The prevention, ADR-0045); one quietly decided presentation and now offers both shapes (tag listing grouping). The
+261
View File
@@ -0,0 +1,261 @@
# Surface
Generated by `scripts/surface.sh` — do not edit. Regenerate with `make surface`.
Every top-level declaration in the engine, with its line. Read this before opening a source
file: it answers "where does X live" and "what is in this package" without the bodies. What each
file is *for* lives in `state.md`; why it is that way lives in `decisions.md`.
## cmd/khosra — 256 lines
check.go 45 · main.go 150 · new.go 42 · wire.go 19
- check.go:16 func runCheck(args []string)
- main.go:23 func main()
- main.go:44 func runServe()
- main.go:108 func rebuilder(fsys fs.FS, cache string, reveal bool, live *atomic.Pointer[content.Site]) func() int
- main.go:133 func defaultCache() string
- main.go:143 func fatal(msg string, err error)
- new.go:12 func runNew(args []string)
- wire.go:15 func extenders(partial render.Partial) []goldmark.Extender
## internal/content — 1042 lines + 537 test
clock.go 12 · content.go 450 · doc.go 5 · extras.go 92 · settings.go 59 · site.go 424
- clock.go:9 var now = time.Now
- clock.go:12 func Now() time.Time { return now() }
- content.go:20 const DefaultLang = "en"
- content.go:27 type Bundle struct
- content.go:70 func OpenSite(dir string) (fs.FS, error)
- content.go:82 type Problem struct
- content.go:94 func Scan(fsys fs.FS) ([]Bundle, error)
- content.go:103 func ScanReport(fsys fs.FS) ([]Bundle, []Problem, error)
- content.go:142 func Parse(name string, data []byte) (Bundle, error)
- content.go:182 func (b Bundle) Published(at time.Time) bool
- content.go:191 func (b Bundle) Assets() (string, bool)
- content.go:201 func stringList(v any) []string
- content.go:223 func asTime(v any) time.Time
- content.go:239 func asInt(v any) int
- content.go:252 func terms(v any) []string
- content.go:276 func TagSlug(tag string) string
- content.go:285 func Normalise(s string) string { return norm.NFC.String(s) }
- content.go:291 func splitName(name string) (key, lang string, ok bool)
- content.go:313 func isLangTag(s string) bool
- content.go:331 func isPartial(base string) bool
- content.go:343 func skipDir(base string) bool
- content.go:351 func splitFrontmatter(data []byte) (front, body []byte)
- content.go:366 func trimLeadingFence(data []byte, fence string) ([]byte, bool)
- content.go:383 func dropCollisions(all []Bundle) ([]Bundle, []Problem)
- content.go:405 const PerPage = 10
- content.go:411 func URL(key, lang string) string
- content.go:424 func TagURL(section, slug, lang string, page int) string
- content.go:434 const DerivedPrefix = "/derived/"
- content.go:437 func DerivedURL(name string) string { return DerivedPrefix + name }
- content.go:440 const TagsSegment = "tags"
- content.go:444 func PageURL(key, lang string, page int) string
- extras.go:14 const ExtrasDir = "extras"
- extras.go:17 type Entry struct
- extras.go:33 func Extras(fsys fs.FS, b Bundle) []Entry
- extras.go:67 func KindOf(name string) string
- extras.go:86 func ExtrasURL(route, lang, entry string) string
- settings.go:13 const SettingsFile = "site.yaml"
- settings.go:20 type Settings struct
- settings.go:33 func LoadSettings(fsys fs.FS) (Settings, error)
- settings.go:54 func Absolute(base, path string) string
- site.go:11 type Site struct
- site.go:28 func NewSite(bundles []Bundle) *Site
- site.go:45 func (s *Site) Reveal() { s.reveal = true }
- site.go:52 func (s *Site) visible(b Bundle) bool
- site.go:58 func (s *Site) Problems() []Problem { return s.problems }
- site.go:61 func (s *Site) note(path, detail string)
- site.go:72 func (s *Site) indexRoutes(bundles []Bundle)
- site.go:111 func (s *Site) KeyFor(route string) (string, bool)
- site.go:122 func (s *Site) RouteOf(key string) string
- site.go:130 func sorted(set map[string]bool) []string
- site.go:143 func (s *Site) indexAliases(bundles []Bundle)
- site.go:175 func (s *Site) Alias(alias string) (string, bool)
- site.go:184 func (s *Site) Lookup(key, lang string) (b Bundle, served string, ok bool)
- site.go:202 func (s *Site) Variants(key string) []string
- site.go:216 func (s *Site) HasLang(lang string) bool
- site.go:226 func (s *Site) Len() int { return len(s.byKeyLang) }
- site.go:232 type Query struct
- site.go:243 func (s *Site) Run(q Query) []Bundle
- site.go:268 func (s *Site) keys() []string
- site.go:284 type Sequence struct
- site.go:304 func (s *Site) Sequence(key, lang string) (Sequence, bool)
- site.go:325 func (s *Site) parent(key string) (string, bool)
- site.go:335 func (s *Site) has(key string) bool
- site.go:344 func (s *Site) members(series, lang string) []Bundle
- site.go:369 func (b Bundle) hasTag(slug string) bool
- site.go:382 func (b Bundle) Section() string
- site.go:394 func (s *Site) Everything() []Bundle
- site.go:407 func (s *Site) Sections() []string
## internal/ext/check — 223 lines + 147 test
check.go 215 · doc.go 8
- check.go:15 type Finding struct
- check.go:29 func Run(fsys fs.FS, bundles []content.Bundle, site *content.Site, problems []content.Problem) []Finding
- check.go:52 func Fatal(found []Finding) bool
- check.go:62 func inspect(fsys fs.FS, b content.Bundle, site *content.Site) []Finding
- check.go:78 var figureCall = regexp.MustCompile(`(?m)^\s*\{\{<\s*figure\s+([^>]*)>\}\}\s*$`)
- check.go:81 var altArg = regexp.MustCompile(`alt="[^"]+"`)
- check.go:87 func checkFigures(b content.Bundle) []Finding
- check.go:98 var internalLink = regexp.MustCompile(`\]\((/[^)\s"]*)`)
- check.go:104 func checkLinks(fsys fs.FS, b content.Bundle, site *content.Site) []Finding
- check.go:121 func engineOwned(target string) bool
- check.go:138 func resolves(fsys fs.FS, target string, from content.Bundle, site *content.Site) bool
- check.go:164 func asset(fsys fs.FS, trimmed string, site *content.Site) bool
- check.go:192 func mixedOrdering(bundles []content.Bundle, site *content.Site) []Finding
## internal/ext/scaffold — 102 lines + 89 test
doc.go 8 · scaffold.go 94
- scaffold.go:23 func New(siteDir, key, lang, title string) (string, error)
- scaffold.go:69 func frontmatter(title string) string
- scaffold.go:76 func titleFrom(key string) string
- scaffold.go:85 func mkdirAll(root *os.Root, dir string) error
## internal/ext/shortcodes — 569 lines + 417 test
doc.go 7 · images.go 250 · shortcodes.go 312
- images.go:31 var widths = []int{480, 960, 1440}
- images.go:38 func Derive(siteFS fs.FS, cacheDir string) (int, error)
- images.go:76 func derive(data []byte, name, cacheDir string) (int, error)
- images.go:103 func scale(src image.Image, width int) image.Image
- images.go:115 func write(file string, img image.Image, source string) error
- images.go:133 func encode(w *os.File, img image.Image, source string) error
- images.go:149 var
- images.go:159 func picture(origin render.Origin, file string) (render.Picture, bool)
- images.go:214 func derivedName(data []byte, source string, width int) string
- images.go:224 func lossless(source string) bool
- images.go:234 func showable(name string) bool
- images.go:244 func derivable(name string) bool
- shortcodes.go:24 const
- shortcodes.go:33 func New(partial render.Partial) goldmark.Extender
- shortcodes.go:37 type extension struct
- shortcodes.go:46 func (e extension) Extend(md goldmark.Markdown)
- shortcodes.go:56 var nested = parser.NewContextKey()
- shortcodes.go:64 type includes struct
- shortcodes.go:68 func (in includes) Transform(doc *ast.Document, reader text.Reader, pc parser.Context)
- shortcodes.go:88 func (in includes) convert(name string, pc parser.Context) ([]byte, error)
- shortcodes.go:118 func pending(doc *ast.Document) []*node
- shortcodes.go:136 var kind = ast.NewNodeKind("Shortcode")
- shortcodes.go:139 type node struct
- shortcodes.go:153 func (n *node) Kind() ast.NodeKind { return kind }
- shortcodes.go:155 func (n *node) Dump(source []byte, level int) { ast.DumpHelper(n, source, level, nil, nil) }
- shortcodes.go:158 type blocks struct{}
- shortcodes.go:160 func (blocks) Trigger() []byte { return []byte{'{'} }
- shortcodes.go:162 func (blocks) Open(parent ast.Node, reader text.Reader, pc parser.Context) (ast.Node, parser.State)
- shortcodes.go:193 func gallery(pc parser.Context) []render.Picture
- shortcodes.go:220 func (blocks) Continue(n ast.Node, reader text.Reader, pc parser.Context) parser.State
- shortcodes.go:224 func (blocks) Close(n ast.Node, reader text.Reader, pc parser.Context) {}
- shortcodes.go:226 func (blocks) CanInterruptParagraph() bool { return true }
- shortcodes.go:228 func (blocks) CanAcceptIndentedLine() bool { return false }
- shortcodes.go:231 type fragments struct
- shortcodes.go:235 func (f fragments) RegisterFuncs(reg renderer.NodeRendererFuncRegisterer)
- shortcodes.go:243 func (f fragments) render(w util.BufWriter, source []byte, n ast.Node, entering bool) (ast.WalkStatus, error)
- shortcodes.go:270 func parse(line string) (name string, args map[string]string, ok bool)
- shortcodes.go:297 func argument(s string) (key, value, rest string, ok bool)
## internal/ext/watch — 137 lines + 103 test
doc.go 8 · watch.go 129
- watch.go:23 var
- watch.go:29 type Changed func()
- watch.go:35 func Watch(fsys fs.FS, stop <-chan struct{}, onChange Changed)
- watch.go:75 func Fingerprint(fsys fs.FS) string
- watch.go:94 func record(sum hash.Hash, p string, d fs.DirEntry, err error) error
- watch.go:117 func dropping(name string) bool
## internal/render — 694 lines + 359 test
chrome.go 110 · render.go 454 · view.go 130
- chrome.go:19 var chrome = map[string]map[string]string{
- chrome.go:33 var months = map[string][]string{
- chrome.go:41 var digits = map[string][]rune{
- chrome.go:47 var funcs = template.FuncMap{"t" text, "num" numerals, "day" day}
- chrome.go:53 func text(lang, key string, args ...string) string
- chrome.go:73 func numerals(lang string, n int) string
- chrome.go:82 func day(lang string, t time.Time) string
- chrome.go:98 func localiseDigits(lang, s string) string
- render.go:24 var themeFS embed.FS
- render.go:28 type Renderer struct
- render.go:56 type Partial func(name string, data Fragment) ([]byte, error)
- render.go:59 type Fragment struct
- render.go:69 type Picture struct
- render.go:86 type Origin struct
- render.go:95 var originKey = parser.NewContextKey()
- render.go:98 func OriginFrom(pc parser.Context) (Origin, bool)
- render.go:105 func WithOrigin(pc parser.Context, origin Origin)
- render.go:118 func New(siteFS fs.FS, settings content.Settings, extend func(Partial) []goldmark.Extender) (*Renderer, error)
- render.go:160 func (r *Renderer) head(title, lang, canonical string) head
- render.go:177 func (r *Renderer) absolute(path string) string
- render.go:185 func (r *Renderer) Navigation(sections func() []string) { r.sections = sections }
- render.go:189 func (r *Renderer) Reload() { r.reload = true }
- render.go:193 func (r *Renderer) fresh() error
- render.go:208 func (r *Renderer) Partial(name string, data Fragment) ([]byte, error)
- render.go:226 func parseSet(siteFS fs.FS, names ...string) (*template.Template, error)
- render.go:248 func readStyle(siteFS fs.FS) (template.CSS, error)
- render.go:266 func (r *Renderer) Extras(b content.Bundle, served string, entries []content.Entry, selected *Selected) ([]byte, error)
- render.go:290 func (r *Renderer) RenderText(kind string, data []byte) (template.HTML, error)
- render.go:308 func (r *Renderer) Bundle(b content.Bundle, served string, variants []string, seq *content.Sequence) ([]byte, error)
- render.go:345 func (r *Renderer) Listing(section, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:367 func (r *Renderer) Tag(section, slug, lang string, all []content.Bundle, page int) ([]byte, error)
- render.go:394 func (r *Renderer) sequence(seq *content.Sequence, lang string) *Sequence
- render.go:421 func (r *Renderer) item(b content.Bundle, lang string) Item
- render.go:426 func (r *Renderer) paginate(title, lang, canonical string, all []content.Bundle, page int, url func(int) string) (List, []content.Bundle)
- render.go:448 func (r *Renderer) execute(set *template.Template, data any, what string) ([]byte, error)
- view.go:16 type head struct
- view.go:36 type Page struct
- view.go:55 type Sequence struct
- view.go:71 type Extras struct
- view.go:82 type Selected struct
- view.go:91 type List struct
- view.go:105 type Group struct
- view.go:111 type Item struct
- view.go:122 type Alternate struct
## internal/web — 734 lines + 1586 test
asset.go 58 · discover.go 71 · extras.go 93 · feed.go 125 · resolve.go 170 · web.go 217
- asset.go:22 func serveAsset(w http.ResponseWriter, req *http.Request, site *content.Site, siteFS fs.FS, res resolution) bool
- discover.go:14 const
- discover.go:24 func serveRobots(w http.ResponseWriter, req *http.Request, siteFS fs.FS, base string)
- discover.go:45 func serveSitemap(w http.ResponseWriter, req *http.Request, site *content.Site, base string)
- discover.go:67 func xmlEscape(s string) string
- extras.go:18 func serveExtras(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer,
- extras.go:69 func renderExtras(w http.ResponseWriter, r *render.Renderer, b content.Bundle, served string,
- extras.go:85 func find(entries []content.Entry, want string) (content.Entry, bool)
- feed.go:13 const feedFile = "feed.xml"
- feed.go:17 const feedMax = 20
- feed.go:24 type atom struct
- feed.go:34 type atomLink struct
- feed.go:40 type atomEntry struct
- feed.go:52 func serveFeed(w http.ResponseWriter, req *http.Request, site *content.Site, res resolution, settings content.Settings) bool
- feed.go:102 func dated(all []content.Bundle) []content.Bundle
- feed.go:113 func feedTitle(settings content.Settings, res resolution) string
- resolve.go:12 type resolution struct
- resolve.go:40 func resolve(path string, site *content.Site) (resolution, bool)
- resolve.go:102 func cutLang(key string, site *content.Site) (lang, rest, redirect string)
- resolve.go:117 func cutExtras(key string) (bundle, entry string, ok bool)
- resolve.go:131 func cutFeed(key string) (rest string, ok bool)
- resolve.go:142 func cutTag(key string) (tag, section string, ok bool)
- resolve.go:154 func cutPage(key string) (rest string, page int, ok bool)
- web.go:19 type Current func() *content.Site
- web.go:22 func Fixed(site *content.Site) Current { return func() *content.Site { return site } }
- web.go:27 func Handler(current Current, r *render.Renderer, siteFS, derivedFS fs.FS, settings content.Settings) http.Handler
- web.go:60 func serveStatic(sub fs.FS) http.Handler
- web.go:76 func serveListing(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
- web.go:100 func serveTags(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
- web.go:125 func write(w http.ResponseWriter, out []byte, what string)
- web.go:134 func writeAs(w http.ResponseWriter, contentType string, out []byte, what string)
- web.go:142 func serve(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, siteFS fs.FS, settings content.Settings)
- web.go:173 func serveBundle(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer,
+1 -1
View File
@@ -42,4 +42,4 @@ you, not the agent; `verify.sh` keeps it honest without reading it into context.
- [deferred-decisions.md](deferred-decisions.md) — five pre-code ADRs demoted to intent; each returns when something implements it. **parked** - [deferred-decisions.md](deferred-decisions.md) — five pre-code ADRs demoted to intent; each returns when something implements it. **parked**
- [specs-as-secondary-artifacts.md](specs-as-secondary-artifacts.md) — optional per-feature specs, derived by default, plus the named-test convention. **parked** - [specs-as-secondary-artifacts.md](specs-as-secondary-artifacts.md) — optional per-feature specs, derived by default, plus the named-test convention. **parked**
- [engine-design-review.md](engine-design-review.md) — open design decisions for a multi-type site; items graduate to ADRs one at a time. **parked** - [engine-design-review.md](engine-design-review.md) — open design decisions for a multi-type site; items graduate to ADRs one at a time. **parked**
- [token-conservation.md](token-conservation.md) — cut agent token cost without losing output quality. **parked** - [token-conservation.md](token-conservation.md) — cut agent token cost without losing output quality. **adopted → `docs/context-economy.md`**, except the subagent question
+24 -2
View File
@@ -1,6 +1,8 @@
# Token conservation # Token conservation
Status: parked Status: adopted → `docs/context-economy.md` + ADR-0053 (2026-08-01). All six mechanisms landed, plus
the compressed-form, machine-discovery and quiet-output disciplines the human added. The open question
below is the only part still undecided; it is also recorded in `state.md` "Open questions".
Raised: 2026-07-28 Raised: 2026-07-28
## Why it came up ## Why it came up
@@ -40,7 +42,27 @@ Do not re-derive these.
**Subagents for fan-out search.** Dispatching "find every place X appears" to a subagent keeps its **Subagents for fan-out search.** Dispatching "find every place X appears" to a subagent keeps its
file dumps out of the main context: you pay for the conclusion, not the search. A real multiplier file dumps out of the main context: you pay for the conclusion, not the search. A real multiplier
on exploration-heavy work. Currently disallowed unless explicitly requested, so it is a policy on exploration-heavy work. Currently disallowed unless explicitly requested, so it is a policy
call, not a judgment call. call, not a judgment call. Deferred again on 2026-08-01, with the case laid out:
*For.* The criterion is read volume ÷ conclusion length. `/audit`, `/refresh-docs` and `/invariants`
all read thousands of lines to produce a verdict of twenty, so their ratio is ~100:1 and the reading
is pure waste in the main window. Several passes could run at once. A subagent that goes down a wrong
path costs nothing here, which makes speculative exploration cheap for the first time.
*Against.* Four real costs. (1) The subagent reads under its own judgment, not the read-order
discipline in `CLAUDE.md` §1 — the thing that keeps this project's read set small is exactly what a
delegated agent is worst at honouring. (2) A verdict without its evidence cannot be audited: "no
abstraction has earned extraction" is only trustworthy if the counters were read correctly, and the
report is all that comes back. (3) Findings arrive as prose to be re-verified, and re-verification of
a wrong report costs more than the original read would have. (4) Total tokens go *up*, not down — the
saving is entirely in *this* window, which matters for session length and not at all for usage limits.
*Where it is clearly right.* Read-only sweeps whose output is a list of locations, where being wrong is
visible immediately: "every doc mentioning the old form after a rename", "which files import X".
*Where it is clearly wrong.* Anything that decides something — a verdict, an earn-it call, an
invariant, whether a warning is a defect. Those need the evidence in the window where the decision
gets made.
## Cheapest next step ## Cheapest next step
+6
View File
@@ -10,6 +10,7 @@ EXT_LOC_MAX=2000 # internal/ext/ — composition, grows after the core fre
FILE_LOC_WARN=500 # any single .go file — advisory FILE_LOC_WARN=500 # any single .go file — advisory
FUNC_LOC_WARN=60 # any single function — advisory FUNC_LOC_WARN=60 # any single function — advisory
DEPS_MAX=6 # total modules in go.mod, direct plus indirect DEPS_MAX=6 # total modules in go.mod, direct plus indirect
CLAUDE_LOC_MAX=150 # CLAUDE.md only — the one file billed on every turn (ADR-0053)
# Two ceilings because "the core stops growing after Arc 2" (architecture.md invariant 9) is only an # Two ceilings because "the core stops growing after Arc 2" (architecture.md invariant 9) is only an
# invariant if something measures it: post-freeze CORE holds and only EXT rises. # invariant if something measures it: post-freeze CORE holds and only EXT rises.
@@ -21,3 +22,8 @@ DEPS_MAX=6 # total modules in go.mod, direct plus indirect
# Feeds, sitemap, OpenGraph, shortcodes, image sizing, galleries, paging, indieweb ~900-1200 → EXT # Feeds, sitemap, OpenGraph, shortcodes, image sizing, galleries, paging, indieweb ~900-1200 → EXT
# 2000, deliberately loose since that is where growth belongs. Reaching CORE asks what to delete; # 2000, deliberately loose since that is where growth belongs. Reaching CORE asks what to delete;
# reaching EXT asks whether a template would have done it — only .go lines count here. # reaching EXT asks whether a template would have done it — only .go lines count here.
#
# CLAUDE_LOC_MAX is a different kind of budget: every other ceiling here bills once, when someone reads
# the code, while CLAUDE.md is re-sent on every turn of every session. 150 leaves ~20 lines of headroom
# over the current file. Reaching it means moving detail to the doc that owns the topic, never deleting
# a rule to fit — `docs/README.md`'s compression contract governs which of the two you are doing.
+24 -1
View File
@@ -2,4 +2,27 @@
# Makes the gate non-optional. Enable once per clone: # Makes the gate non-optional. Enable once per clone:
# git config core.hooksPath scripts/hooks # git config core.hooksPath scripts/hooks
# Bypass deliberately with `git commit --no-verify` — and say why in the commit body. # Bypass deliberately with `git commit --no-verify` — and say why in the commit body.
exec ./scripts/verify.sh set -uo pipefail
cd "$(git rev-parse --show-toplevel)" || exit 1
# Anything mechanically derived from the code is regenerated and staged here, not remembered. A
# generated file that only fails a gate teaches nothing — the commit should simply carry the current
# one. `verify.sh` still compares, because a clone without core.hooksPath set has no hook at all.
#
# The generated file describes the working tree, so a partly-staged .go file would put a surface in the
# commit for code the commit does not contain — the same failure the staged-tree gate exists for.
# Checked before generating, because a staged deletion also breaks the generator.
unstaged=$(git diff --name-only -- '*.go')
if [ -n "$unstaged" ]; then
printf 'FAIL unstaged .go changes — docs/surface.md would describe code this commit lacks:\n'
printf ' %s\n' $unstaged
printf ' stage them, stash them, or commit with --no-verify and say why in the body.\n'
exit 1
fi
# One generated artifact, so one call. A second one makes this a two-line list; it does not make it a
# registry (CLAUDE.md rule 1).
./scripts/surface.sh --write >/dev/null || exit 1
git add docs/surface.md
exec ./scripts/verify.sh --quiet
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
# Generates the compressed form of the code: every package, its size, and every top-level declaration
# with the line to jump to.
#
# Usage: scripts/surface.sh print it
# scripts/surface.sh --write regenerate docs/surface.md (`make surface`)
#
# Why this exists: reading a 400-line file to learn what is in it costs forty times what reading its
# surface costs, and the answer to "where does X live" is almost never in the bodies. `go doc` already
# does this for *exported* identifiers; this also covers unexported ones, which is where most of khosra
# lives — nothing outside cmd/ has many exported names by design.
#
# Generated, never hand-edited, and gated: verify.sh regenerates and compares, so a stale surface fails
# the build rather than misleading a reader. Purposes and intent stay in docs/state.md — this file is
# mechanical and says only what is there.
set -uo pipefail
cd "$(dirname "$0")/.." || exit 1
emit() {
printf '# Surface\n\n'
printf 'Generated by `scripts/surface.sh` — do not edit. Regenerate with `make surface`.\n\n'
printf 'Every top-level declaration in the engine, with its line. Read this before opening a source\n'
printf 'file: it answers "where does X live" and "what is in this package" without the bodies. What each\n'
printf 'file is *for* lives in `state.md`; why it is that way lives in `decisions.md`.\n'
git ls-files '*.go' | grep -v '_test\.go$' | xargs -n1 dirname | sort -u | while read -r dir; do
files=$(git ls-files "$dir/*.go" | grep -v '_test\.go$' | sort)
[ -z "$files" ] && continue
tests=$(git ls-files "$dir/*_test.go" | sort)
code=$(echo "$files" | xargs wc -l | awk '$2!="total"{t+=$1} END{print t+0}')
testloc=0
[ -n "$tests" ] && testloc=$(echo "$tests" | xargs wc -l | awk '$2!="total"{t+=$1} END{print t+0}')
printf '\n## %s — %s lines' "$dir" "$code"
[ "$testloc" -gt 0 ] && printf ' + %s test' "$testloc"
printf '\n\n'
# One manifest line per package, so the cost of opening any file is visible before you open it.
echo "$files" | while read -r f; do printf '%s %s · ' "$(basename "$f")" "$(wc -l <"$f" | tr -d ' ')"; done |
sed 's/ · $//'
printf '\n\n'
for f in $files; do
# Top-level declarations only: in gofmt'd Go those are exactly the lines starting in column one.
# The trailing brace and any body opener go, so a signature reads as one line.
grep -nE '^(func|type|const|var) ' "$f" |
sed -e 's/ {$//' -e 's/ ($//' -e 's/[[:space:]]*$//' |
awk -F: -v n="$(basename "$f")" '{ line = $1; $1 = ""; sub(/^:/, ""); sub(/^ +/, ""); printf "- %s:%s %s\n", n, line, $0 }'
done
done
}
if [ "${1:-}" = "--write" ]; then
emit >docs/surface.md
printf 'wrote docs/surface.md (%s lines)\n' "$(wc -l <docs/surface.md | tr -d ' ')"
else
emit
fi
+31 -5
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# The objective gate. No opinions, only exit codes. # The objective gate. No opinions, only exit codes.
# Usage: ./scripts/verify.sh # Usage: ./scripts/verify.sh [--quiet|--list]
set -uo pipefail set -uo pipefail
# --list answers "does this gate actually exist?" without reading the script. Docs that claim a gate are # --list answers "does this gate actually exist?" without reading the script. Docs that claim a gate are
@@ -19,13 +19,20 @@ EXT_LOC_MAX=${EXT_LOC_MAX:-2000}
FILE_LOC_WARN=${FILE_LOC_WARN:-500} FILE_LOC_WARN=${FILE_LOC_WARN:-500}
FUNC_LOC_WARN=${FUNC_LOC_WARN:-60} FUNC_LOC_WARN=${FUNC_LOC_WARN:-60}
DEPS_MAX=${DEPS_MAX:-6} DEPS_MAX=${DEPS_MAX:-6}
CLAUDE_LOC_MAX=${CLAUDE_LOC_MAX:-150}
fail=0 fail=0
warn=0 warn=0
pass() { printf ' ok %s\n' "$1"; } # --quiet prints only what needs acting on: failures, warnings, and the verdict. A green run is two
# lines instead of forty-three, which matters because this is the most-run command in the repo and its
# output is read far more often by an agent's context window than by a person. Passing gate names are
# never information — `--list` answers "does this gate exist?" and does it without running anything.
quiet=0
[ "${1:-}" = "--quiet" ] && quiet=1
pass() { [ "$quiet" -eq 1 ] || printf ' ok %s\n' "$1"; }
bad() { printf ' FAIL %s\n' "$1"; fail=1; } bad() { printf ' FAIL %s\n' "$1"; fail=1; }
note() { printf ' warn %s\n' "$1"; warn=$((warn + 1)); } note() { printf ' warn %s\n' "$1"; warn=$((warn + 1)); }
head_() { printf '\n%s\n' "$1"; } head_() { [ "$quiet" -eq 1 ] || printf '\n%s\n' "$1"; }
if ! command -v go >/dev/null 2>&1 || ! command -v gofmt >/dev/null 2>&1; then if ! command -v go >/dev/null 2>&1 || ! command -v gofmt >/dev/null 2>&1; then
echo "FAIL go toolchain not found on PATH — cannot verify anything" echo "FAIL go toolchain not found on PATH — cannot verify anything"
@@ -178,6 +185,24 @@ if [ -d .git ] && command -v git >/dev/null 2>&1; then
note "docs/state.md 'verified against' does not name a commit this repo knows ($recorded)" note "docs/state.md 'verified against' does not name a commit this repo knows ($recorded)"
fi fi
fi fi
# The generated surface is only worth reading if it cannot be wrong. Regenerate and compare rather
# than trusting that whoever added a function also ran the script.
if [ -x scripts/surface.sh ]; then
if ! scripts/surface.sh | cmp -s - docs/surface.md; then
bad "docs/surface.md is stale — run 'make surface'"
else
pass "docs/surface.md matches the code"
fi
fi
# CLAUDE.md is re-sent on every single turn, which makes it the most expensive file per byte in the
# repo. A ceiling here is not tidiness; it is the only budget that bills per turn.
claudeloc=$(wc -l <CLAUDE.md | tr -d ' ')
if [ "$claudeloc" -gt "$CLAUDE_LOC_MAX" ]; then
bad "CLAUDE.md is $claudeloc lines, over CLAUDE_LOC_MAX=$CLAUDE_LOC_MAX (move detail to the owning doc)"
else
pass "CLAUDE.md $claudeloc / $CLAUDE_LOC_MAX lines"
fi
else else
note "not a git repo — doc coupling unenforceable" note "not a git repo — doc coupling unenforceable"
fi fi
@@ -185,10 +210,11 @@ fi
result_and_exit() { result_and_exit() {
head_ "result" head_ "result"
if [ "$fail" -eq 0 ]; then if [ "$fail" -eq 0 ]; then
printf ' PASS %s warning(s)\n\n' "$warn" printf ' PASS %s warning(s)\n' "$warn"
else else
printf ' FAIL fix the above before reporting success\n\n' printf ' FAIL fix the above before reporting success\n'
fi fi
[ "$quiet" -eq 1 ] || printf '\n'
exit "$fail" exit "$fail"
} }