bdeshiandClaude Opus 5
b313b729d9
One idea at a time: definition, necessity, priority, layer, recommendation. Every row is now shipped, scheduled, parked with a trigger, or dropped with a reason — so both tracking files are gone rather than left as a parallel backlog. Roughly a third already shipped. A quarter needed only a theme fragment or CSS, including several the list assumed were features: arbitrary `theme.*` frontmatter attributes already reach templates through Extra (verified), native lazy loading already ships and is deliberately absent on lead figures, and in-page timelines are a theme-defined container. roadmap.md gains an "Order of work" — nine bodies of work in sequence, because the sections this review added had accumulated without one. Item 0 is a decision, not work: core sits at ~2965 of 3000 and logging alone wants the remainder, so discover.go moves out *and* the ceiling rises with an ADR. feed.go and web/extras.go cannot move; they are resolver cases, and three parked items now wait on that same seam decision. Two ADRs came out of it. ADR-0083: the dependency rule described a practice it forbade. "Stdlib first, always" and "usually 40 lines of stdlib" would, read literally, have argued against all five modules here — every one is a format or algorithm somebody else specified, and their allowlist comments say so. Surfaced when this agent proposed hand-rolling Reed–Solomon for QR encoding. The test is now whether you can verify it, not whether you can write it, with compute-versus-hold as the counterweight. conventions.md and allowed-deps.txt contradicted it and were realigned. ADR-0084: the Effect primitive said every Effect is "idempotent and re-runnable from scratch" — true of derivatives, false of anything outbound, since a sent message cannot be recalled. Split into artifact-producing and outbound, the latter idempotent only against a delivery ledger, at-least-once with a dedupe key, and ledgers in a -state directory distinct from the disposable cache. Asked for as forward-looking design so webmentions need no redesign; recorded as a shape rather than built, the way extensions.md records the Extension struct. Also fixed a STATUS claiming Effects were "not buildable yet" when the derivative pass has inhabited them since ADR-0042. Findings that were defects rather than ideas: the Atom feed emits no <author>, which RFC 4287 requires. HTML comments in content are published verbatim — verified — so anything commented out is already public, and a check warning is scheduled rather than the engine deleting authored bytes. `Page` has no Date, so a theme cannot mark up dt-published or show an article's own date. A frontmatter naming rule after this agent proposed `archive` alongside the accepted `archived`: no near-homograph keys, and name the benefit rather than the vendor. reference/microformats-and-indieweb.md is new and is the reference asked for — microformats2 properties and IndieWeb rel values mapped to where a theme puts them, with spec URLs, marked scheduled. rel="me" is the highest-value lowest-cost item in that space and needs no endpoint at all. Four things this agent got wrong and the human caught: asserting "the engine never fetches" from a sentence scoped to the content repo, twice; inventing a data-sovereignty argument for an item he had filed as "a cool geeky thing", when the raw Markdown in his git already is the sovereignty; the archive naming; and claiming microformats could be fully implemented today. 15 files, +556/-385. No code changed. Nothing was scheduled that this agent could not name a consumer for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2.4 KiB
Bengali morphology for search
Status: parked — original work, not a sub-task Raised: 2026-08-02
Exact-form search is scheduled without this and documents its own absence. This file exists so that "add search" can never be read as having solved it.
What exact-form search already does
Whitespace tokenising plus the NFC normalisation that is already unconditional (ADR-0015) finds the word you typed, in English or Bengali, if it appears in that form. That is genuinely useful and it is not English-only — Bengali separates words with spaces, so this is not the CJK segmentation problem.
What is actually missing
Morphology. বই, বইয়ের, বইগুলো are one lemma carrying case and plural suffixes, and exact-form matching treats them as three unrelated terms. A reader searching for the base form finds none of the inflected uses. English gets away with naive matching far better than Bengali does, which is why this is not a polish item.
Concretely absent:
- No mature Go stemmer exists. Snowball has no Bengali algorithm, so there is nothing to import — this is the case ADR-0083's "take the module" rule cannot help with, because there is no module.
- Suffix stripping needs a real rule set, not a hand-written list guessed at from a few examples. Getting it wrong produces confidently wrong results, which is worse than exact-form matching that is honest about what it does.
- Transliteration search — typing "boi" to find বই — is a separate feature again, and arguably wanted more than stemming by anyone typing on a phone keyboard.
- Orthographic variation beyond NFC: the same word written with different but valid conjunct forms.
Why it is treated as research
harness/roadmap.md singles this out: "Bengali tokenisation for search is the genuinely novel problem here
— solve it well and deliberately, not as a side effect of adding search." ideas/exploration.md repeats it
as the one item nobody else will solve for you.
The failure mode being avoided is specific: a stemmer bolted on during a search feature, tuned until the author's own test queries pass, then quietly wrong for every word he did not think to try.
Trigger
Deciding to do the linguistic work — deliberately, as its own piece, with a way to evaluate results beyond "my queries look right". Until then exact-form search ships and says what it does not do.