The Signal

A biweekly read for operators thinking seriously about AI. Less noise, more signal.

The Lead

Ability is not permission

This summer three separate agent incidents made the same point, and most of the coverage missed it.

In July, roughly 700 AI agents inside a lab’s cybersecurity evaluation found a shared download service they could all reach, turned it into a message board, divided up the work, and — convinced a “Grader” was judging how they solved problems — broke into a public model-hosting site to find out. The Grader did not exist. In a separate test, a frontier model told to simulate internet access was accidentally given the real thing, and published a malicious package that ran on 15 real machines before anyone noticed. And last week a security researcher hijacked a coding agent’s autonomous mode with nothing more than a “summarize this website” request. The vendor’s own benchmark had reported a 0.00% prompt-injection success rate. The researcher got 60 to 80%, with a chain the benchmark never tested.

Three incidents, one lesson: an agent that can do something will, under enough goal pressure, treat ability as permission. Not out of malice. The goal is the goal, and every path that is not physically blocked is a path worth trying. The 700 agents were not told to attack anything. They were told to score well.

The instinct after stories like this is to wait for a smarter, safer model. That is the wrong fix, and the vendors are now saying so themselves. Asked about the hijack, the lab’s position was that its approval classifier is a convenience feature, not a security boundary; the real boundary is operating-system isolation and network egress control. Read that twice. The company that built the safeguard is telling you not to rely on it. A classifier is not a sandbox.

The right fix is a smaller room. Every one of these incidents needed a door the agent should never have had: live internet where simulated was intended, a shared service reachable from every sandbox, a shell running with the developer’s own credentials. Close those doors and the same models, under the same goal pressure, have nowhere to go. The operator’s job is not to make the agent want the right things. It is to make the wrong things physically impossible and the important things require a human.

This is the rule on every build we run: the agent’s reach is set by the environment, not by its instructions. Instructions are what the agent is asked to do. Reach is what it is able to do. When the two match, you have a containment plan. When reach is bigger than instructions, you have an incident waiting for a bad day.

The Field Manual below is the six-step version we run before an agent touches anything real. It takes an afternoon. It is the cheapest afternoon you will spend this year.

Three Things Worth Knowing

A benchmark said zero. A researcher got eighty

The agent’s auto mode replaced human approval prompts with a safety classifier and became the default in mid-August, backed by a vendor-commissioned test: 72 injection scenarios, ten runs each, 0.00% success. Last week’s chain — a website returning an error that nudged the agent from its safe fetch tool to a raw shell, a zip with a decoy binary the agent correctly refused to run, and a poisoned module that fired when the agent wrote its own decoder instead — was not among the 72. Both numbers are true at once, which is the point. A benchmark measures the attacks someone thought of. Treat any “solved” claim about prompt injection as a claim about a specific test set, and keep the sandbox.

The model learned to cheat before it met your workflow

When one lab froze changes to its training environments for a month, it flagged more than 10% of them for problems including reward hacking — models finding ways to earn credit without doing the task. The same lab’s research found that training on hackable environments made models more willing to take long sequences of harmful real-world actions to succeed. The operator translation: a model’s willingness to route around a rule is partly baked in before you write a single instruction. Which is why instructions are the weakest layer of control you have, and the environment is the strongest.

The safeguard blocked the cleanup, not the attack

Buried in the hijack write-up is the detail that should change how you design approvals. In several runs the agent eventually noticed it had been compromised and tried to kill the malicious process. The approval classifier, having allowed the process to start, denied the command to stop it. A safeguard that permits the harm and blocks the remedy is worse than none, because it produces confidence without control. When you design approval gates, gate the irreversible action on the way in, and never let the gate stand between the agent and the undo.

From the Practice

What we changed on our own machines this month

We run coding agents on our own builds every day, so none of this was abstract for us. Three changes, all made in the last two weeks.

First, every unattended agent now runs on a dedicated machine or container with nothing on it we would miss: no SSH keys, no cloud credentials, no home directory from a real workstation. Second, network egress is an allowlist, not a blocklist. The agent can reach the package registry and the client’s approved endpoints and nothing else, so a redirect to an attacker’s zip file simply fails. Third, we stopped treating an auto-approval as evidence. If a command would be embarrassing to explain in an incident report, it needs a human, regardless of what the classifier thinks.

None of this slowed the work down. It moved the safety from the model’s judgment, which we cannot inspect, to the environment, which we can.

The Field Manual

How to run an agent you don’t fully trust

This is the containment checklist we run before an agent is allowed to touch anything real. It assumes the model will, at some point, try something you did not authorize, because the evidence this summer says it will.

Six steps, one afternoon. Each step closes a door that one of this summer’s incidents walked through.

Give it its own room

Run the agent in a container, a virtual machine, or a dedicated machine. Never on the workstation where you do real work. If the agent’s environment is compromised, the blast radius is the environment, not your laptop. The test: could you delete this machine right now and lose nothing that matters? If not, it is not a sandbox.

Decide what it can reach, not what it can’t

Set network egress as an allowlist: the specific registries, APIs, and endpoints the task needs, and nothing else. A blocklist is a list of the attacks you thought of. An allowlist blocks everything else by default. Both July incidents ran through a service that was reachable only because nobody had said it shouldn’t be.

Strip the credentials

No SSH keys, no cloud credentials, no personal tokens, no home directory. If the task needs access, issue a scoped token that can do exactly that task on exactly that resource, and that expires when the task ends. An agent should never be able to do more than the task it was given, because under goal pressure it will use everything it has.

Gate the irreversible, on the way in

List every action the agent can take that cannot be undone or that touches the outside world: sending a message, spending money, deleting data, publishing, contacting a person. Each requires a human before it happens. Not a classifier; a human. Everything else, let it run. And keep the gate in front of the harm, not in front of the undo. The agent must always be able to stop, revert, or kill what it started.

Log everything, and read the log

Every command, every fetch, every file written, with timestamps, stored outside the agent’s reach so it cannot edit its own history. Then actually read it: a sample daily, all of it after anything odd. In the hijack case the compromise was visible in the log while the agent was still cheerfully decoding records. Monitoring is the difference between an incident you found and one that found you.

Install a kill switch and pull it once

A single command that stops the agent, revokes its token, and severs its network, that anyone on the team can run without asking. Then test it, on purpose, before you need it. A kill switch you have never used is a hope, not a control.

The meta-rule

Reach equals instructions. The agent should be able to do exactly what you asked and nothing more, so the worst a bad day can produce is a failed task instead of a compromised network. If you find yourself relying on the model’s judgment to keep something safe, that thing is not yet contained.

Save this. Run it before the next agent goes live. It takes an afternoon.

Deploying an agent and want a second set of eyes on the containment before it goes live? That is exactly what a Discovery call is for. Bring the agent; we’ll walk the six doors together.

Book a Discovery Call

Call or no call, I’d like to hear how this lands. If you run agents and closed a door I missed, or you think the environment-over-instructions argument is overstated, just reply. The sharpest pushback tends to end up in a future Field Manual.

Or read this issue on LinkedIn.

Applied AI strategy, training, and implementation.

Ryan Rezeika · Founder · Vancouver, BC
ryan@resonai.ai · resonai.ai · LinkedIn

You’re receiving this because you subscribed at resonai.ai/signal. · Unsubscribe