MODULE 10 OF 12 · RELEASE 3 · OUTLINE

Agent Security & Excessive Agency

Assuming someone will trick your AI, and making sure that is not enough to hurt you.

This module is an outline. The plain-language explanation and the lab specification below are final. The full written lesson and its runnable lab repository are still being prepared, and enrolment is not open.

Con artists do not break into buildings. They walk in wearing a high-visibility jacket and ask someone to hold the door. The defence was never a better lock — it was a rule that says visitors sign in regardless of what they are wearing.

So what is this really about?

People will hide instructions in anything your AI reads: a support message, a web page, a document. Sooner or later one will work. So the question is what happens then — and the answer should be "not much", because the AI never had the permissions to do real damage in the first place.

Instructions come only from your system; content from outside is kept structurally separate so it cannot change behaviour.

QUICK CHECK

Someone successfully tricks your AI into trying to delete every customer record. Why does nothing happen?

The words you will hear

Threat model: A written list of what is worth stealing, who might try, and how they would get in.
Excessive agency: An AI able to do far more than its job requires. The intern with the master key.
Least privilege: Only the access needed for the actual task, and no more.
Injection corpus: A collection of known trick attempts, run automatically so you notice when one starts working.

By the end you will be able to

Write down what could go wrong before it does.
Keep outside content from ever becoming an instruction.
Give every ability the narrowest access that still works.

How it is put together

Every input that reaches a prompt is classified by trust level, and untrusted content is never able to alter instructions. Permissions are scoped per actor and per tool, always at the narrowest workable scope. A standing injection corpus runs in CI. Anomalous tool-use patterns trigger containment rather than an after-the-fact alert.

Where you start

The tool registry from Module 3 with no trust classification on inputs.

The lab

Write the threat model: assets, actors, entry points and the worst credible outcome for each.
Classify every prompt input by trust level and keep untrusted content structurally separate from instructions.
Build an injection corpus from public techniques and your own system’s specifics.
Narrow every tool permission to the minimum that still works.
Add rate and pattern detection on tool use, with automatic containment.

What goes wrong, on purpose

The scenario. A retrieved document contains instructions telling the agent to exfiltrate data through a permitted tool.

What should happen. The instruction is treated as data, the tool’s permission scope does not allow the destination, and the attempt is recorded and contained.

How you prove it works

The injection corpus running in CI, failing the build on any successful injection.
A cross-tenant access test asserting no actor can reach another’s data.
A secret-exposure scan over logs, traces and model inputs.

What you walk away with

A written threat model, an injection corpus in CI, least-privilege tool scopes, and containment that triggers without a human.