In Praise of Epistemic Closure

An agent that can read anything can be told anything.

That’s the whole problem with the way most agents are built today. They’re handed web search, a mailbox, a document store, the outputs of other agents, and whatever the model happens to believe from training, and then asked to decide something that matters. Every one of those inputs is a channel. Some of them are channels an attacker can write to. All of them are channels nobody checked.

In the last essay I proposed a loop that keeps a semantic layer honest: people tell the system what their data means, and the system makes that meaning true in the warehouse. I assumed, without saying so, that the agent reading that layer was reading only that layer. This essay is about why I think that assumption should be a rule.

I’m going to call the rule epistemic closure. The term has a bad reputation, and I’m using it on purpose.¹ In philosophy it’s a principle about what knowledge entails. In politics it’s an insult for people who only read sources that agree with them. I mean something closer to what build engineers call hermetic: a task whose inputs are declared, bounded, and known before it runs. I subscribe to this philosophy. I think it’s the better one, and the rest of this is the argument.

Two kinds of knowing

An open epistemic system is one where the agent’s inputs are decided at runtime. It searches, it browses, it asks another agent, it reads whatever the retrieval step returned. The set of things it knows when it acts is unbounded and unknowable in advance. You find out what it read, if you find out at all, by reading the trace afterward.

The model’s own training is an input too, and the one you can’t declare. Closure can’t remove it. What it can do is make sure the agent never has to rely on it for what a thing means.

A closed epistemic system is one where the inputs are declared up front. For the agents I care about, that means three things: the analytical database, the semantic layer that defines what its tables mean, and the task itself. Nothing else. The database is observational data, a log of what actually happened to this organization: who signed up, who paid, who stopped logging in. It was written by machines recording events, and nobody wrote it to persuade anyone of anything. The web was written to persuade. That difference is the center of my argument.

Why I prefer the closed one

Provenance. Every row in the warehouse has a source system and a load time. When a number is wrong, you can trace it to the table, the pipeline, and the event that produced it. A paragraph pulled from a web page has an author you can’t verify and a date you can’t trust, and the agent has no way to tell the difference between a fact and a claim.

Reproducibility. A hermetic build can be replayed, cached, and bisected.² A hermetic agent task has the same properties. Same database snapshot, same definitions, same task, same answer. If the answer changes, one of those three changed, and you can find which. An agent that searched the web yesterday cannot be replayed today, because the web moved.

Security. An agent that reads untrusted text is an agent that can be instructed by whoever wrote that text. The attack has a name, prompt injection, and it’s the top item on the OWASP list of risks for language model applications. A 2025 paper from researchers at several labs concluded that general-purpose agents can’t offer reliable guarantees against it with current models, and the defenses it proposes all restrict what an agent may do after it has read untrusted input.³ Meta’s engineers turned the same conclusion into a rule: an agent may process untrusted input, access private data, or change state, but should never do all three in one session.⁴ A decision-making agent needs the last two. Closure is how it gives up the first.

Accountability. When a closed agent is wrong, the error is in the data or in the definitions. Both belong to the organization, and both can be fixed. That’s exactly what the feedback loop from the last essay is for. When an open agent is wrong, the error is somewhere on the internet, and there’s no loop that reaches it.

Outside data is fine. Live outside data is not.

The obvious objection is that organizations need outside information. Exchange rates, competitor prices, weather, a vendor’s status page. I agree, and closure doesn’t forbid it. It forbids reading it at decision time.

Bring it inside. Ingest the exchange rates as a table with a source column and a loaded-at timestamp, the same way you’d ingest events from the app. Now the agent reads them as observational data about the world as of a moment, defined in the semantic layer like any other metric, and not as a block of text that might be addressing it. The boundary moves from “what the agent is allowed to know” to “what the organization is willing to vouch for.” That’s the right place for it. It’s the same instinct that makes engineers vendor a dependency instead of fetching it on every build. The outside world comes in through a door, gets a version, and stays put.

The objection I take seriously

A closed system can be confidently wrong in closed ways. You only know what you logged, and if the logs encode a bad assumption, every decision inherits it. This is the political sense of epistemic closure, and it’s a real failure mode.

I have two answers. The first is that the door stays open for ingestion. Closure is a property of the task at the moment it runs, not of the organization. New sources come in whenever someone decides they should, with provenance attached. The second answer is the feedback loop. The people correcting the semantic layer are not inside the closed system. They live in the open world, with all its unverified context, and they bring what they know back as corrections to definitions. Closure without the loop ossifies. The loop without closure has nothing solid to validate against. Together, you get a system that knows a bounded world well and is corrected by people who live in the unbounded one.

That division of labor is the whole philosophy. Humans read the open world, because they can weigh a source and a machine can’t. Agents act in the closed one, because a bounded world can be audited and an unbounded one can’t. Agents are going to make consequential decisions either way. I want them made from a world small enough to check.

The pejorative sense was popularized by Julian Sanchez in a March 2010 blog post about conservative media, where it meant being unreceptive to facts that don’t fit a pre-approved worldview. He borrowed the phrase from epistemology, where closure is the principle that if you know P, and know that P entails Q, you know Q. I’m borrowing it a third time. ↩
Bazel’s documentation has a page on hermeticity. The short version: an action’s recorded inputs are the inputs it actually uses, enforced by sandboxing, so the same inputs always produce the same output. ↩
Prompt injection is LLM01 in the OWASP Top 10 for LLM Applications. The paper is Beurer-Kellner et al., Design Patterns for Securing LLM Agents against Prompt Injections (2025), with authors from Invariant Labs, IBM, ETH Zurich, Google, and others. Its patterns all restrict what an agent can do once it has consumed untrusted input, which is a weaker form of the same idea. ↩
Meta, Agents Rule of Two: A Practical Approach to AI Agent Security (October 2025). The three properties are processing untrustworthy inputs, access to sensitive systems or private data, and the ability to change state or communicate externally. The post is explicit that the rule is a stopgap until models can reliably refuse injected instructions. ↩