-
September 19, 2026
The site was ready to publish.
The AI agent doing the publishing used one of the methods, Did It Land, to answer a single question before calling the job done: are the files going out the same files that were checked?
-
The check
Half of them weren't.
The site kept a list of its files with a fingerprint for each one. Compared against the copies actually saved for publication, 14 of the 28 files didn't match. Nothing looked different on screen.
-
The cause
Two invisible characters.
The copy on the Windows machine ended each line of text with two hidden characters. The repository was set to store text with one. Same words, different bytes. The fingerprint list had been made from the Windows copy.
-
The fix
Rebuild the list from what Git stores.
The corrected list was built from the files the repository actually stored. Differences went from 14 to zero.
-
The proof
Then check what actually shipped.
After the site went live, the agent downloaded every file from the public site and compared each one with the list. All 28 matched.
Anyone can replay both checks from the public history of this site's repository:
python scripts/replay_manifest_case.py
Work Epistemic Skills
Epistemic Skills · AI agentsTeaching AI agents to check their own work.
AI agents say "done" too early. They guess at causes and report fixes they never confirmed. Epistemic Skills is a public set of seventeen written methods that tell an agent how to find the real cause and how to prove a change actually worked.
Public source · thirteen releases since July 2026, latest 7.2.0 on September 26 · see the record
The day this site went live, the method caught a mismatch the screen didn't show.
AI is a lever. The authority stays outside it.
A model can propose a cause or a verdict. What settles it is something observed, a check that gives the same answer every time, or a person who owns the outcome.
Find the real cause
Compare the competing explanations and run the test that tells them apart before fixing anything.
Check where it runs
Follow a change to where it actually runs and look at the result there. A green test on the source doesn't settle it.
Keep it in proportion
Fixing a typo doesn't need a review panel. The heavier methods are for questions where a wrong answer costs something.
Leave a record
Big decisions and long tasks get written records that outlast the chat, so the next session can pick them up and check them.
It's the same rule I'd apply to legal work. Interleaf, the contract workstation on this site, is built around it: language models help at the edges, such as sorting documents or putting findings into plain English, and never decide what changes in a document or whether a result goes out. Interleaf's own build used these methods. In one review, AI reviewers tried to break a claim and caught an earlier write-up overstating what one of its tools verifies.
Pick the method the question needs.
Each method says when to use it, what it should hand back and when to stop. Each method tells an agent that uses it to say so, in a line.
Triage
Which observation would distinguish this cause from the alternatives?
Compare competing explanations, look at the relevant evidence, and use a targeted test to narrow the cause. Then carry the supported repair through to verification.
Hands back: A supported cause and a checked repair.
An example of choosing a method, not a recording of an agent at work.
What went into this work.
What I did
I decide what each project is for and check what comes back. For version 7, I set out to fix three problems I kept seeing: agents skipping a method that applied, piling process onto routine work, and stopping before the work was finished. I also required that an agent say when it used a method, even in one line.
What the AI did
AI tools write the code. A Codex agent built version 7 and reviewed it at my direction, so that review wasn't independent.
What's proven, and what isn't yet.
- Recorded
- The publication check above, on September 19, 2026. Its record and the script that replays it are public.
- Source
- All seventeen methods, their shared checks, worked examples and release receipts are public on GitHub under GPL-3.0-or-later.
- Not proven yet
- Whether the methods make agents perform better. The August comparison found no difference between groups, and the version 6 to 7 comparison produced no valid paired trials. Both results are published.
The full record: all seventeen methods and the test notes
The package holds seventeen methods: one usage guide and sixteen disciplines. The guide, epistemic, sets the routine the rest assume: find a method that fits, load its instructions, apply it, say briefly what it contributed, and carry on with the task. The tables follow the four groups of the public catalog, and each row names the method by its folder in the repository.
Begin and examine the reasoning
| Method | The question it helps answer | What it hands back |
|---|---|---|
epistemic | How should these methods be used here? | The routine above, applied only as far as the task needs. |
metacognate | Is our reasoning aimed at the right outcome? | A corrected assumption, check or confidence statement. Routine work that can be checked directly skips it. |
perspective | What does another focused angle reveal? | An insight, an improvement or a bounded finding, handed back to the work. |
gauntlet | Which judgment holds up when several separate reviews look at it? | A reasoned verdict that keeps the material disagreements and says what the reviews covered. |
Understand and settle uncertainty
| Method | The question it helps answer | What it hands back |
|---|---|---|
recon | What are we missing about the task and what it depends on? | A corrected brief or a map of the decisions ahead, before any building starts. |
resolve | Which evidence can settle this question? | An answer from reasoning, published research or a small throwaway test. |
open-questions | Which consequential choices need the user? | Answered questions, and any action held until the user decides. |
context-audit | Are the loaded instructions conflicting or stale? | A diagnosis and a proposed change someone can review before it is made. |
Observe, diagnose and verify
| Method | The question it helps answer | What it hands back |
|---|---|---|
health | What state is the system in now? | A status for each part, including what could not be checked. A summary that reads twelve OK while three parts were unreachable is the failure this exists to prevent. |
triage | What caused this particular failure? | A cause backed by an observation that rules out the alternatives, or an open verdict: narrowed, unknown, or not broken after all. |
did-it-land | Did the change reach the thing that runs? | A result observed where the change is used. Until then, the verdict is unverified. |
watch | Who will notice a problem that appears between sessions? | A separate monitor, tested all the way to its alert. The method itself does not keep watch. |
evidence-locked-uat | Does a user-facing result meet its acceptance criteria? UAT is user acceptance testing. | An acceptance verdict tied to what was observed, and no stronger than its weakest required evidence. |
Keep work recoverable
| Method | The question it helps answer | What it hands back |
|---|---|---|
decision-ledger | Can later work recover and recheck this decision? | One lasting record with its assumptions and the condition that would reopen it. |
manifest | Does this multi-session task need a formal record that survives interruptions? | The task's recorded state, with acceptance by someone other than whoever did the work. |
write-goal | What counts as done, and how can the AI tool track it? | A written definition of done, which starts only when someone authorizes it. |
outsource | Can someone outside this chat do the work from what we hand over? | A handoff saved at one fixed version of the repository, and a check of what comes back. |
Test notes
The release records note where an AI tool or operating system limited what could be tested. Eight methods (epistemic, metacognate, perspective, did-it-land, health, triage, watch and manifest) say in their own instructions that the conditions for using them rest on design reasoning, and that no shipped set of tests yet shows they are picked up when they should be. That note was added on September 20, 2026, the day after version 7.0.0.
The publication check describes this site's first publication. The site has grown since, so its counts are historical.
