blog · June 2026 · 3 min read

We measured ourselves at 38.9%

We built an eval harness and graded our own AI's memory. The honest number was bad. We published it anyway — and made it the plan.

LAPLAS does one strange thing. The AI you already use writes its own memory of your work — the projects, the decisions, the reasons underneath them — into plain files on your disk. You think; it remembers.

Which raised a question we put off for too long: is the memory any good?

Not "does it feel good in a demo." Demos always feel good. The real question: given an actual working session, does the system capture what mattered — the right entities, the right decisions, the right causal links — and nothing it made up?

So we built an eval harness. We took real sessions, wrote out by hand what a perfect memory of each one would contain, and graded our own AI against that gold standard. The metric was capture precision: of everything the AI wrote into your memory, what fraction was real, correctly named, and worth keeping?

The honest number was 38.9%.

38.9%
capture precision · first harness run

Sit with that for a second. More than six out of ten things our AI confidently wrote into the memory were wrong, invented, mangled, or noise. A memory engine that misremembers most of what it hears.

Most products would hide that number. Bury it in an internal doc, redefine the metric until it looked respectable, ship anyway. The landing page would still say "AI-powered memory." Nobody audits a landing page.

We made it the plan instead. We set a hard gate: 90% precision before we scale anything. No growth push, no aggressive auto-capture, no "trust us" — until the harness says we've earned it. The gate is the roadmap now.

What the gate changed

A number you refuse to hide starts making decisions for you. Concretely:

  • AI-invented names get rejected. The model loves coining entities that sound right — a project name you never used, a person who doesn't exist. If a name can't be traced to something you actually said, it doesn't get written.
  • Every node needs one plain sentence. Not a tag cloud, not a fragment. If the AI can't state what it learned in a sentence a person would write, it didn't learn it. Empty nodes are precision rot.
  • Receipts after every write. When the AI commits to your memory, you see exactly what it wrote, right there in the session. Memory that changes silently isn't memory — it's drift.
  • The override layer. Everything the AI writes sits in a layer your hand can overrule. Edit it, delete it, pin your own version on top. The human hand always wins, by construction, not by policy.

None of this came from a brainstorm about trust. It came from staring at the failure cases in the harness, one by one, and asking what would have stopped each lie from landing on disk.

We're not at 90% yet. We re-run the harness on every meaningful change, and the number moves — sometimes up, sometimes embarrassingly down. When we cross the gate, you'll see it here, with the methodology, because a number without its harness is just marketing.

A memory you can't trust is just a database. And trust is measured, not promised.