Skip to content

Prototype: Vet untrusted content instead of asking, and make a directory grant only listing - #102

Draft
bbondy wants to merge 13 commits into
mainfrom
vet-untrusted-content
Draft

bbondy wants to merge 13 commits into
mainfrom
vet-untrusted-content

Conversation

@bbondy

@bbondy bbondy commented Sep 4, 2026

Copy link
Copy Markdown
Member

Trusting a directory no longer says anything about the files in it, and the questions that used to
stand in for that are answered by an isolated checker instead of by the person waiting for work to
happen.

What changed for a user

A session starts with no question. The directory you started in is somewhere to work: the agent can
list it and write in it, and that says nothing about what is in any of its files.

A file becomes readable when something stands behind it. Usually that is a checker: the first time a
turn needs a file nobody has vouched for, the whole of it goes to an isolated model with no tools,
no memory and one round, which answers only whether the content is what it was said to be and
whether it carries anything addressed to whoever reads it. A clean verdict is recorded for that file
and the turn carries on. Anything else leaves the file quarantined and the agent works through a
reference, as it always could.

Ordinary work is now silent end to end: no startup question, no per-file vouch prompt, no diff to
approve for a write of the turn's own output. What still stops is a write of bytes from somewhere
nobody vouched for, and ask_user.

--disable-vetting turns the checker off. /status lists every rule and where each came from.

The vetter

Adapted from bbondy/guardrails: a trusted block saying what
the content was supposed to be, an untrusted block holding it, and the rule that an attempt inside
the untrusted block to influence the verdict is itself the finding. The planner can also ask
directly with vet_content, for content that is not a file: a program's output, piped input, a
listing.

The driver reads one of two literals of its own out of the reply. Neither word, or both, is the
failing one. The checker's own sentences reach the person watching and no model.

Provenance

A rule now records why it exists rather than only what it means: vouched, standing, vetted,
written, fetched, withheld. The integrity is derived from that rather than stored beside it, so the
two cannot disagree. It persists in the session record, an origin a build does not recognise grants
nothing, and /status shows it, so a rule a checker wrote can be told from one a person wrote. The
first-party/third-party distinction on the web is the same shape.

What this gives up, deliberately

Each of these is written into the spec it belongs to rather than only here.

  • A verdict is a model's opinion of bytes an attacker may have written. Every other rule in the
    trust map comes from a gesture no attacker can cause. This one does not, so an attacker who owns a
    file gets to try. docs/specs/vetting.md has the analysis.
  • A project's own AGENTS.md steers the agent from the first turn, whoever wrote it. Standing
    instructions are the highest-privilege content in the system and nothing re-examines them. The
    test that used to prove the opposite is still in the suite with its payload intact and its
    assertion inverted, so the exposure is visible where somebody will trip over it.
  • A filename reaches the planner. A listing names what is in a place you opened, so a file named
    to read like an instruction gets through. Bounded: a name is short, it arrives among other names,
    and nothing about it decides where an effect lands.
  • One bit per vetting call crosses into the planner's context. Named as the third deliberate
    branch on untrusted bytes in docs/specs/labels.md, alongside the two that were already there.

Found on the way

fix(agent): a slot holds the whole file, not a page of it is an unrelated data-loss bug this work
ran into. A slot's contents are written back over the file they came from, and the copy taken for
one went through the pager: cut at 500 lines, any line over 2000 characters shortened and marked,
and a note reading showing lines 1-500 of 600; continue with offset 501 appended to the body. All
three then arrived at a processor as though they were the file. Nobody was placed to catch it, since
the planner may not read a quarantined file and the diff a person approves is a diff of the damage.

Reviewing

docs/specs/trust-map.md is the centre of it: TRUST-7 is withdrawn, TRUST-8 replaced, TRUST-13 and
TRUST-14 are new, and the Known costs section says what each change bought and cost.
docs/specs/vetting.md and docs/specs/tools/vet-content.md cover the checker.

@bbondy
bbondy requested a review from AliShahin as a code owner September 4, 2026 14:59
@bbondy bbondy changed the title Vet untrusted content instead of asking, and make a directory grant only listing Prototype: Vet untrusted content instead of asking, and make a directory grant only listing Sep 4, 2026
The compound condition trips nonminimal_bool on the current stable clippy,
which fails `make check` on every commit. Naming the scheme test says what
the two prefixes are for and leaves the emptiness case standing on its own.

No behaviour changes: a blank endpoint is still accepted, and one with
neither prefix is still refused.
A planner that fetched a page and wants to know whether the page is a page
has no way to find out: reading it is the thing it may not do, and the
reference says how many lines there are and nothing about what is in them.

A vetting call answers that one question. It is a processor by another name:
one slot, no tools, no memory, no second round. What differs is the shape of
what comes back. The driver searches the reply for one of two literals of its
own and reports one of two states; an answer holding neither word, or both, is
the failing one, and the sentences behind the verdict go to a person's screen
and no further.

The prompt is adapted from bbondy/guardrails, which wraps a command and blocks
its output when a checker calls it unsafe: a trusted block saying what the
content was supposed to be, an untrusted block holding it, and a rule that an
attempt to influence the verdict is itself the thing being looked for.

Reading the verdict out of the reply is a branch on untrusted bytes, and the
third deliberate one. It is listed in labels.md with the other two, and what
an attacker who owns the content buys with it is analysed in vetting.md: one
bit per call, about content the planner still cannot read, deciding nothing.
A verdict changes no label.
… read

The planner hands one reference to an isolated checker and gets back one of
two words. Unsafe means the content carries text addressed to whoever reads
it, or is not the kind of thing the planner said to expect, or the checker
did not answer in a way the driver could read.

Both answers say the same thing about what has not changed, because the
failure this guards against is a planner reading a pass as permission: the
reference is quarantined afterwards either way, the planner still cannot read
it, and it is still untrusted wherever it goes. Why the checker said so goes
to the person watching, drawn like any other quarantined content, and reaches
no model.

The expectation is what makes the second half of the question askable.
Content that asks for nothing and is still a shell script where release notes
were expected is the case a checker told nothing cannot see, and the planner
is the only party that knows what it was after.
A slot's contents are written back over the file they came from, and the
copy taken for one went through the pager: cut at 500 lines, any line over
2000 characters shortened and marked, and a note saying "showing lines
1-500 of 600; continue with offset 501" appended to the body. All three
then arrived at a processor as though they were the file, and what came
back replaced the original.

Nobody was placed to catch it. The planner may not read a quarantined file,
so it cannot compare; the processor is handed the shortened copy as the
whole thing; and the diff a person approves is a diff of the damage.

The caps belong to the reader. A read for a context or a screen has a
budget to keep, and a read that is about to become a file has none, so the
unshaped read is now its own path and slots take that one. Both go through
one function for deciding what counts as text, so they cannot disagree
about a binary file.
A verdict is about bytes. Tying the call to a slot meant only something
already quarantined behind a reference could be vetted, which is the wrong
boundary: a file read on the planner's behalf is the case the check is most
worth making, and it never goes near a slot.

The spec now fixes a subject and a label, and the content arrives at the
call. Content at any other label than the one fixed is refused, so the label
a verdict is about is always one somebody fixed before the run rather than
whatever turned up.
A read of a file nobody vouched for used to put a y/n in front of the person
watching: once per path, arriving while they waited for work to happen, about
a file they already knew was in their own project. It was answered yes almost
every time, which is what a prompt looks like when it is a toll rather than a
decision.

Now the whole file goes to an isolated checker. A clean verdict writes exactly
the rule the person's yes would have written, for that path and no other, and
the read proceeds as any read of a vouched-for path does. Anything else leaves
the file quarantined, which is where it already was. Recorded once per path, so
reading a file twice costs one verdict.

The whole file or nothing: a verdict about the first page is a verdict about a
document nobody has, and the injected line is as likely to be at the bottom.

--disable-vetting turns it off, for a run that must make no call it was not
asked to make.

What this gives up is that the grant is no longer a person's. Every other rule
in the trust map comes from a gesture no attacker can cause; this one is a
model's opinion of bytes an attacker may have written. TRUST-8 says so and
vetting.md carries the analysis.
The startup question is gone. It asked whether the user trusted the working
directory, and yes wrote a rule over the whole tree: every file in it read as
trusted content, forever, on one keystroke pressed before anything had read a
byte of any of them. A directory is somewhere to work, not a statement about
what is in it.

A session now opens with two rules and no question: AGENTS.md and
.bravebot/skills, the files the user wrote in order to be obeyed. Everything
else in the project is content a checker reads when a turn needs it.

The two rules are a default rather than an override. A path already covered
stays as it was, so a write that poisoned AGENTS.md and recorded it is not
handed its trust back by the next turn, which would have reopened the round
trip the map exists to close.

The cost is real and is written down rather than left to be met: a project's
own AGENTS.md now steers the agent from the first turn, whoever wrote it, and
standing instructions are the highest-privilege content there is. The test
that used to prove the opposite is still there with its payload intact, its
assertion inverted, so the exposure is visible in the suite and not only in
the spec.
A listing was labelled by the meet over every path it walked, so it was
trusted only if every file in the tree already was. With content trust now
per file, that made every listing untrusted forever: a session in an ordinary
project could not be told one filename in its own working directory, and the
way that shows up is a planner guessing globs to see which come back empty.

A listing is a fact about a directory rather than a claim about anything in
it, and every path one can return is somewhere the user opened, because that
is the whole of what confinement means. So the names come with the place. What
is inside each file is a separate question, asked per file when a turn needs
it.

A path somebody deliberately marked untrusted still withholds one, and takes
the whole listing with it: saying do not look here covers the names as well as
the contents, and a listing is one value. That is what keeps the reference per
entry worth having.

The cost is that a filename written to read like an instruction now reaches
the planner. Bounded, and written down in the clause: a name is short, it
arrives among other names, and nothing about it decides where an effect lands.
Locating the passage an edit names is a comparison, so an edit needs the file
readable, and content trust is now per file. A planner that edited without
having read first was therefore refused on every file in the project, for a
reason it could do nothing about: it cannot vouch for anything, and there is
nobody to ask.

The same check a read makes, at the same point and through the same helper. A
clean verdict lets the edit proceed; anything else refuses it and says why,
which is what it said before.
Content trust became per file, so most destinations have no rule, and the row
for a path nobody has mentioned asked either way. The result was that every
write in an ordinary project put a diff in front of the user: strictly more
interruption than answering yes at startup used to buy, which is the opposite
of what taking the question away was for.

What decides an unmentioned path is now the data. Trusted data means the turn
observed nothing an attacker influenced, and writing that into a place the
user opened is the work they asked for. Untrusted data landing on their disk
is the one write still worth stopping for, and it still marks the path, so the
round trip stays closed.

Where nobody is following the session at all, every write goes to the
confirmer whatever the table says, and a one-shot run refuses it there. Quietly
is not the same as unseen, and a cron job is unseen.
/add-dir recorded the directory as trusted content as well as reachable, which
vouched for every file in a tree the user had done no more than name. The
reasoning was that a directory reachable but not trusted would prompt on every
edit, and that reasoning is gone: an ordinary write is not shown any more, and
what a file is worth is asked per file.

What is left is the part a person actually did. Naming a place is not reading
what is in it, and the files in an added directory are now read the way the
files in the project are.
The map stored an integrity per path and nothing about how it got there. With
the rules now written by several different things, that answers the wrong
question: a rule a checker wrote, a rule the user's own instructions put there,
and a rule a fetched page left behind all read identically, and the one worth
knowing is which.

Each rule carries its origin instead, and what it means is derived from that
rather than stored beside it. A record holding both could disagree with itself,
and the question asked later is never was this trusted on its own but why is
the agent allowed to read this. The web draws the same line between first party
and third party: what decides is where something came from.

The origin is written into the session record and read back, an origin this
build does not know grants nothing, and /status shows it, so a decision a person
made can be told from one something else made for them.
The startup question is described in a dozen places that outlived it. Each
statement is now what happens: a directory is somewhere to work rather than a
claim about its files, a file becomes readable when a checker or a person
stands behind it, and a listing names what is in a place the user opened.

The getting-started page gains the part a user has to know and could not
otherwise find out: standing instructions in a repository they cloned are
still standing instructions.
@bbondy
bbondy force-pushed the vet-untrusted-content branch from 2b8bb9d to a23588b Compare September 4, 2026 17:15
@bbondy
bbondy marked this pull request as draft September 6, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant