move robots and sitemap out of core, and raise the ceiling on purpose

Item 0 of the roadmap's order of work, and it blocked everything after it: core
sat at 2965 of 3000 while the review scheduled four core-bound items, the first
of which — logging — wanted the whole remainder.

/robots.txt and /sitemap.xml are exact paths somebody else's software asks for by
name. They own no core concept and pass every test the architecture applies to a
feature; they lived in internal/web only because a feature could not own a route
until ADR-0081. internal/ext/discover/ now holds them. Core 2965 → 2913.

The seam gained one parameter to make it possible: a func() *content.Site, since a
sitemap must list what is served now and the index is swapped whole on every
rebuild (ADR-0077). A captured pointer would have frozen the site at startup —
which is the kind of bug that only shows up after a rebuild, in production.

The ceiling rises to 3400 as well as the move, because the move alone could not buy
the room. feed.go and web/extras.go cannot follow discover out: a feed lives at
/{section}/feed.xml and extras under a bundle's own URL, so both are resolver cases
while the seam mounts exact paths only. Raising by the minimum that unblocks one
item produces a ceiling nobody believes, so 3400 fits the View cluster with
headroom. HARNESS.md asks that a raise be read as evidence something belongs in
ext before evidence the number was small; both readings were true, so both actions
were taken.

web no longer reserves those two paths, so a clash between features is wire.go's:
it merges route maps in declaration order, keeps the earlier claim, logs the loser.
Verified — a site shipping root/robots.txt starts, serves the engine's robots.txt,
and logs the passthrough claim, where an unguarded mux.Handle would have panicked.

Evidence: robots.txt and sitemap.xml are byte-identical before and after the move
against the demo site (67 and 2701 bytes, cmp clean), and the sitemap keeps its
application/xml type.

One real cost, recorded in both places rather than hidden. internal/web's
visibility test asserted that a listing, a feed *and* a sitemap all hide
unpublished bundles — one property, one test, because all three share a Query. The
sitemap half moved to the feature instead of a web test importing ext, which would
invert the one-way layering the architecture gate enforces. That property is now
asserted twice, once per package owning a surface.

Three gates caught real mistakes on the way: the staged-tree check found a partial
stage where git rm had staged a deletion while the caller edits were unstaged, the
coupling gates demanded state.md and HARNESS.md, and the nesting advisory rejected
a closure that put the merge loop one level too deep — fixed by making it a plain
function rather than tolerated.

Extensions 6 → 7. Routing cases unmoved: exact paths are mux entries, never
resolver cases, which is what that counter's exclusion column already said.

13 files. Core 2913/3400, ext 2495/3500.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

harness/decisions.md

@@ -1500,3 +1500,34 @@ URL, which is outbound work that must not happen on the request path. That is th
 trigger, not a fourth one.
 Revisit if: an Effect is neither artifact-producing nor outbound, which would mean a third kind and a real
 gap in this split.
+
+## ADR-0085 — `discover` becomes a feature, and the core ceiling rises to 3400
+Date: 2026-08-03 · Status: accepted
+Decision: `/robots.txt` and `/sitemap.xml` move from `internal/web/discover.go` into
+`internal/ext/discover/`, mounted through ADR-0081's Routes seam. The seam gains one parameter — a
+`func() *content.Site` — because a sitemap must list what is served *now* and the index is swapped whole on
+every rebuild (ADR-0077); a captured pointer would freeze the site at startup. `CORE_LOC_MAX` rises from
+3000 to 3400.
+Why the move: these are exact paths somebody else's software asks for by name, own no core concept, and pass
+every test the architecture applies to a feature. They sat in core only because a feature could not own a
+route until ADR-0081. Core went 2965 → 2908.
+Why the raise as well as the move: the review on 2026-08-02 scheduled four core-bound items — logging, the
+View layer, declared content types and a minimal settings cascade — and 35 free lines would not have fitted
+the first of them. `feed.go` and `web/extras.go` cannot follow `discover` out, because a feed lives at
+`/{section}/feed.xml` and extras under a bundle's own URL: both are **resolver** cases, and the seam mounts
+exact paths only. So the move alone could not buy the room, and raising by the minimum that unblocks one item
+produces a ceiling nobody believes. 3400 fits the View cluster with headroom.
+`HARNESS.md` asks that a core raise be read as evidence something belongs in `internal/ext/` before evidence
+the number was small. Both readings are true here, which is why both actions were taken rather than either.
+Consequence: `web` no longer reserves those two paths, so a clash between *features* is `wire.go`'s to
+settle — it merges route maps in declaration order, keeps the earlier claim, and logs the loser. Verified: a
+site shipping `root/robots.txt` starts, serves the engine's robots.txt, and logs the passthrough claim,
+where an unguarded `mux.Handle` would have panicked. Output is byte-identical before and after for both
+paths on the demo site.
+One real cost: `internal/web`'s visibility test asserted that a listing, a feed **and** a sitemap all hide
+unpublished bundles — one property, one test, because all three share a Query. The sitemap half moved to the
+feature rather than a `web` test importing `ext`, which would invert the one-way layering the architecture
+gate enforces. The property is now asserted twice, once per package that owns a surface. That is a genuine
+loss, recorded in both tests.
+Revisit if: the resolver gains feature participation, at which point `feed.go` and `web/extras.go` can leave
+too and the ceiling should be reconsidered downward rather than left as headroom.

harness/state.md

@@ -33,6 +33,7 @@ table owns.
 | `internal/ext/notation/` | the inline marks CommonMark lacks: `~sub~`, `^sup^`, `==mark==`, and `~~strike~~`, which it owns so a single tilde can mean subscript (ADR-0061). `abbr.go` adds `*[TERM]:` definitions and the pass that expands them (ADR-0062) |
 | `internal/ext/scaffold/` | writes one draft directory bundle into a site root through `os.Root`: never an overwrite |
 | `internal/ext/watch/` | polls `content/` and `templates/` on an interval it is given, ignores editor droppings, and reports a settled change (ADR-0022, ADR-0048, ADR-0056). `site.yaml` is deliberately not fingerprinted (ADR-0055) |
+| `internal/ext/discover/` | seventh feature: `/robots.txt` and `/sitemap.xml`, absolute and only with a declared base (ADR-0039). Left core in ADR-0085 — exact paths somebody else's software asks for by name, owning no core concept |
 | `internal/ext/passthrough/` | fifth feature, and the first to own a **route** (ADR-0081): files in `root/` served at the exact path they occupy, `.tmpl` rendered as text with the site's own settings, headers declared per path in `root/_headers.yaml`, underscore-prefixed names not addressable |
 | `internal/ext/check/` | third feature: validates a site root — what the engine worked around, broken internal links, missing titles and alt text, mixed series ordering, and calls left in the retired shortcode form (ADR-0059) |
 | `cmd/khosra/wire.go` | the only list of enabled features (`extensions.md`) — `extenders()` for the ones goldmark composes and `routes()` for the ones owning a URL path (ADR-0081), the `theme` function that builds the renderer this build ships (ADR-0072), and the Markdown dialect with it — tables, footnotes, definition lists, strikethrough, task lists (ADR-0058, ADR-0078) |
@@ -40,8 +41,7 @@ table owns.
 | `internal/web/extras.go` | the extras route: listing, one entry selected, or `?raw` bytes, all behind the bundle lookup |
 | `internal/web/asset.go` | files inside a bundle's own directory, looked up through the owning bundle so visibility can only ever inherit (ADR-0024) |
 | `internal/web/feed.go` | Atom for the site, a section or a tag, from dated bundles via one Query (ADR-0043) |
-| `internal/web/discover.go` | `/robots.txt` and `/sitemap.xml`, absolute and only with a declared base (ADR-0039) |
-| `internal/web/web.go` | handler: `Snapshot` pairs the index with the theme that was current with it (ADR-0077); `serve` dispatches by kind, `serveBundle` answers the commonest one; listings, `/static/`, `/derived/`, degrade on failure. Mounts the exact paths features own, skipping any the engine already answers — a duplicate pattern would panic (ADR-0081) |
+| `internal/web/web.go` | handler: `Snapshot` pairs the index with the theme that was current with it (ADR-0077); `serve` dispatches by kind, `serveBundle` answers the commonest one; listings, `/static/`, `/derived/`, degrade on failure. Mounts the exact paths features own, skipping any the engine already answers — a duplicate pattern would panic (ADR-0081). Since ADR-0085 it reserves only `/`: `/robots.txt` and `/sitemap.xml` are a feature's, so a clash *between* features is `wire.go`'s to settle |
 | `cmd/khosra/main.go` | flags (including `-poll`, zero to stop watching), wiring, startup, the derivative pass, and the one atomic swap a change goes through, theme and index together in `rebuilder` (ADR-0077). `main` dispatches subcommands, `runServe` assembles the server, `rebuilder` is used at startup and on every change alike |
 | `cmd/khosra/check.go` | the `check` subcommand: parse, print, exit code. What counts as a finding lives in the feature |
 | `cmd/khosra/new.go` | the `new` subcommand: arguments in either order, then the feature does the writing |
@@ -105,7 +105,7 @@ a row that leaves it empty (ADR-0070).
 | Collection pages | 4 | **1** — done | Query primitive: `content.Query{Section, Tag, Lang}` + `Site.Run` | A series archive. Membership is structural and the sort ascends, so it resolves through `Site.Sequence` — sharing the index but not the Query |
 | Views — **per-bundle selection only** | 0 | **2** | The View layer `architecture.md` describes: `view:` in frontmatter choosing a presentation, resolved through the cascade. Nothing selects a view yet | Output formats. HTML, sitemap XML and Atom are three functions with nothing to share — an interface over them would have one member and no leverage |
 | Effects | 1 | **2** | Effect runner + trigger wiring (change / schedule / demand). The only one is the derivative pass (ADR-0042), called from `cmd` inside `rebuilder`, so it already answers both triggers it will ever need — startup and a settled change (ADR-0048) | An in-memory swap. Replacing the index or the theme re-reads the site root into memory, writing no artifact and calling nothing outbound (ADR-0055) |
-| Extensions | 6 | **3** — passed, and the registry is now partly built | Extension registry (`extensions.md`). It reached 3 once before and went back to 2 when the widows feature was deleted (ADR-0045) — a threshold reached by a feature that should not exist was never a threshold. The note below says which field was built and why the rest were not | An upstream extension enabled in the list. `Table`, `Footnote` and `DefinitionList` are goldmark's, so they are dialect rather than features of this engine (ADR-0058) — only a package under `internal/ext/` counts |
+| Extensions | 7 | **3** — passed, and the registry is now partly built | Extension registry (`extensions.md`). It reached 3 once before and went back to 2 when the widows feature was deleted (ADR-0045) — a threshold reached by a feature that should not exist was never a threshold. The note below says which field was built and why the rest were not | An upstream extension enabled in the list. `Table`, `Footnote` and `DefinitionList` are goldmark's, so they are dialect rather than features of this engine (ADR-0058) — only a package under `internal/ext/` counts |
 | Interface implementations | — | **2** | The interface itself | An interface this repo did not declare. Satisfying `fs.FS`, `http.Handler` or `goldmark.Extender` is using somebody else's abstraction, which is the opposite of inventing one |
 | Non-stdlib dependencies | 4 direct | budget in `scripts/budgets.env` | — | The standard library, and a dependency's own test-only modules — `go list -m all` shows those, and the gate counts `require` entries instead (`scripts/budgets.env`) |
 
@@ -120,10 +120,10 @@ landed (ADR-0061, ADR-0062) and the count was never incremented, though the pros
 five. Six now, with `passthrough`. This is the latent item about the counters having no mechanical check,
 demonstrating itself; the count is authoritative only because someone just ran `ls internal/ext/`.
 
-**A registry over the *other* attachment points would still buy nothing.** The six features
+**A registry over the *other* attachment points would still buy nothing.** The seven features
 attach in four unrelated ways: `shortcodes` and `notation` are goldmark extenders listed in `extenders()`,
 `check` and `scaffold` are functions `cmd` calls for a subcommand, `watch` is a goroutine, and
-`passthrough` hands back a map of URL paths. A registry would
+`passthrough` and `discover` hand back maps of URL paths. A registry would
 have to abstract over "extends Markdown", "validates content", "writes a file" and "polls a directory", which
 share nothing but the word *feature* — one member and no leverage. Adding `notation` made this clearer rather
 than more urgent: two goldmark extenders compose in goldmark's own extender list, which is already the registry

harness/surface.md

@@ -6,22 +6,23 @@ Every top-level declaration in the engine, with its line. Read this before openi
 file: it answers "where does X live" and "what is in this package" without the bodies. What each
 file is *for* lives in `state.md`; why it is that way lives in `decisions.md`.
 
-## cmd/khosra — 320 lines + 308 test
+## cmd/khosra — 346 lines + 309 test
 
-check.go 45 · main.go 177 · new.go 42 · wire.go 56
+check.go 45 · main.go 178 · new.go 42 · wire.go 81
 
 - check.go:16 func runCheck(args []string)
 - main.go:23 func main()
 - main.go:44 func runServe()
-- main.go:100 func pollInterval(dev bool, chosen time.Duration) time.Duration
-- main.go:116 func watching(fsys fs.FS, every time.Duration, rebuild func() int)
-- main.go:128 func rebuilder(fsys fs.FS, settings content.Settings, cache string, reveal bool,
-- main.go:160 func defaultCache() string
-- main.go:170 func fatal(msg string, err error)
+- main.go:101 func pollInterval(dev bool, chosen time.Duration) time.Duration
+- main.go:117 func watching(fsys fs.FS, every time.Duration, rebuild func() int)
+- main.go:129 func rebuilder(fsys fs.FS, settings content.Settings, cache string, reveal bool,
+- main.go:161 func defaultCache() string
+- main.go:171 func fatal(msg string, err error)
 - new.go:12 func runNew(args []string)
-- wire.go:20 func theme(siteFS fs.FS, settings content.Settings) (*render.Renderer, error)
-- wire.go:33 func extenders(partial render.Partial) []goldmark.Extender
-- wire.go:54 func routes(siteFS fs.FS, settings content.Settings) map[string]http.Handler
+- wire.go:22 func theme(siteFS fs.FS, settings content.Settings) (*render.Renderer, error)
+- wire.go:35 func extenders(partial render.Partial) []goldmark.Extender
+- wire.go:61 func routes(siteFS fs.FS, settings content.Settings, site func() *content.Site) map[string]http.Handler
+- wire.go:72 func claim(out map[string]http.Handler, feature string, from map[string]http.Handler)
 
 ## internal/content — 1074 lines + 598 test
 
@@ -115,6 +116,17 @@ check.go 223 · doc.go 8
 - check.go:172 func asset(fsys fs.FS, trimmed string, site *content.Site) bool
 - check.go:200 func mixedOrdering(bundles []content.Bundle, site *content.Site) []Finding
 
+## internal/ext/discover — 107 lines + 139 test
+
+discover.go 98 · doc.go 9
+
+- discover.go:14 const
+- discover.go:24 func Routes(siteFS fs.FS, settings content.Settings, site func() *content.Site) map[string]http.Handler
+- discover.go:40 func robots(w http.ResponseWriter, siteFS fs.FS, base string)
+- discover.go:61 func sitemap(w http.ResponseWriter, req *http.Request, site *content.Site, base string)
+- discover.go:83 func xmlEscape(s string) string
+- discover.go:93 func writeAs(w http.ResponseWriter, contentType string, out []byte, what string)
+
 ## internal/ext/notation — 411 lines + 149 test
 
 abbr.go 246 · doc.go 8 · notation.go 157
@@ -355,15 +367,11 @@ chrome.go 115 · render.go 499 · view.go 192
 - view.go:167 type Picture struct
 - view.go:184 type Origin struct
 
-## internal/web — 765 lines + 1423 test
+## internal/web — 687 lines + 1324 test
 
-asset.go 58 · discover.go 71 · extras.go 93 · feed.go 125 · resolve.go 170 · web.go 248
+asset.go 58 · extras.go 93 · feed.go 125 · resolve.go 170 · web.go 241
 
 - asset.go:22 func serveAsset(w http.ResponseWriter, req *http.Request, site *content.Site, siteFS fs.FS, res resolution) bool
-- discover.go:14 const
-- discover.go:24 func serveRobots(w http.ResponseWriter, req *http.Request, siteFS fs.FS, base string)
-- discover.go:45 func serveSitemap(w http.ResponseWriter, req *http.Request, site *content.Site, base string)
-- discover.go:67 func xmlEscape(s string) string
 - extras.go:18 func serveExtras(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer,
 - extras.go:69 func renderExtras(w http.ResponseWriter, r *render.Renderer, b content.Bundle, served string,
 - extras.go:85 func find(entries []content.Entry, want string) (content.Entry, bool)
@@ -386,10 +394,10 @@ asset.go 58 · discover.go 71 · extras.go 93 · feed.go 125 · resolve.go 170
 - web.go:28 type Current func() *Snapshot
 - web.go:31 func Fixed(site *content.Site, theme *render.Renderer) Current
 - web.go:43 func Handler(current Current, siteFS, derivedFS fs.FS, settings content.Settings, routes map[string]http.Handler) http.Handler
-- web.go:91 func serveStatic(sub fs.FS) http.Handler
-- web.go:107 func serveListing(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
-- web.go:131 func serveTags(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
-- web.go:156 func write(w http.ResponseWriter, out []byte, what string)
-- web.go:165 func writeAs(w http.ResponseWriter, contentType string, out []byte, what string)
-- web.go:173 func serve(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, siteFS fs.FS, settings content.Settings)
-- web.go:204 func serveBundle(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer,
+- web.go:84 func serveStatic(sub fs.FS) http.Handler
+- web.go:100 func serveListing(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
+- web.go:124 func serveTags(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, res resolution) bool
+- web.go:149 func write(w http.ResponseWriter, out []byte, what string)
+- web.go:158 func writeAs(w http.ResponseWriter, contentType string, out []byte, what string)
+- web.go:166 func serve(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer, siteFS fs.FS, settings content.Settings)
+- web.go:197 func serveBundle(w http.ResponseWriter, req *http.Request, site *content.Site, r *render.Renderer,