How Leash Works v5
This is the stored snapshot for the approved document version. The diff below shows what changed from the previous version.
Document snapshot
How Leash Works
Leash is a per-user enforcement layer for local AI agents on Linux. You attach it to one host user — the leashed UID the agent runs as — and it makes a hard allow/block decision for everything that user does across four surfaces. The core is one shared policy engine; the surfaces are thin enforcers that intercept at the OS layer, ask the engine, and apply the verdict.
Identity model
leash init --user fidorecords the leashed username in config. The installer creates the account if it does not exist. The agent runs as that user.- Leash itself runs as a privileged service (root, with the capabilities to
install nftables rules and a fanotify mark —
CAP_SYS_ADMIN+CAP_NET_ADMIN). This is the asymmetry that makes enforcement real: the agent is unprivileged and runs as the leashed UID, so it cannot remove Leash's rules or unmark the daemon. - The username is resolved to a UID at load (
pwd.getpwnam). Every decision is keyed to that UID.
More than one subject on a host
A host can carry many leashed UIDs, and the service discovers them itself at startup rather than being told:
- Discovery reads the kernel's own database. nftables matches
meta skuid <uid>and the fanotify daemon filters events by UID, so the subject is a UID — and the one database that says which UIDs exist is the one the kernel resolves them through. So the service enumerates passwd for accounts in the agent namespace (leash-*, the prefix Sentinel mints an agent's Linux account with). A list passed in from elsewhere could disagree with passwd by one character and produce a confinement that silently applies to nobody. Group membership is not read: Sentinel'ssentinel-team-*groups give a team's agents their shared workspace, which is file access, not Leash policy. - Each subject's policy is resolved separately — the global floor, then its own agent overlay; two tiers, global < agent — and a subject whose policy cannot be resolved (a dangling profile, an unacknowledged loosening) fails that subject only: it is announced as a critical warning, it is given no nftables chain at all, and every filesystem and exec event it raises is denied. Every other subject's rules are untouched.
- A chain and a port are per account, for the account's life — not per run. A release never returns a port to the pool, so re-planning an unchanged host moves no listener, and starting a run needs no privileged call.
- The single configured
userstays a subject when agent accounts appear. It owns the state directory, the approvals queue and the run store, and it resolves to the floor — with no tier of its own, so noagent:binding can re-point the control plane's own confinement. On a host with no agent accounts the service takes the original single-subject path unchanged and does not even create/var/lib/leash.
Configuring one agent's policy
The config file is the floor. On top of it, a profile is a named, reusable
policy fragment (/var/lib/leash/profiles/<name>.yaml) and a binding attaches one
profile to one scope — global or agent:<account> (/var/lib/leash/bindings.yaml).
Team was a third scope until 2026-10-14; it was removed as a policy tier, and a team
that should share a policy binds one profile to each of its agents. The operator
surface is
leash profile save|list|show|delete, leash bind, leash unbind and
leash bindings.
sudo leash profile save web-research --from web-research.yaml
sudo leash bind agent:leash-iris web-research
Four things that shape it.
A profile is a fragment, not a config. Only the fields it means to set. Every key a
tier writes is a deliberate value from that tier, so a profile carrying a whole
config's worth of defaults would restate all four surfaces as policy — binding it to
narrow one agent's network would also hand it filesystem: whitelist and default-deny
the filesystem of an agent whose operator chose blacklist on purpose.
A scope is a Linux account name, never a display name. Discovery reads passwd,
so the account name is the only identity Leash has: the scope is agent:leash-iris. agent:Iris is refused rather than
written, because it is a binding that would resolve for nobody and nothing anywhere
would say so — and the account name cannot be derived back from a display name, since
the slug is folded and collision-suffixed. Where the real account exists, the refusal
names it.
Widening the floor needs explicit, per-subject consent. A profile may tighten the
floor freely; a change that widens what a subject may do is refused unless that
binding acknowledges it by field and value. The refusal lists every loosening the
profile performs — not the first — each in the engine's own words with the exact
--ack '<field>:<value>' token, plus the whole command to re-run. The acknowledgement
lives on the binding and never in the profile: one profile is reused by many
subjects, so a profile that could consent on its own behalf would consent for every
subject later bound to it. For the same reason, editing a profile revalidates every
binding that names it and refuses an edit that would loosen a bound subject beyond its
own acks — reuse means one keystroke reaches eight agents.
An edit can carry each bound agent's consent (since 2026-10-14):
leash profile save web-research --ack 'agent:leash-iris=network.allow:openai.com' --ack 'agent:leash-vader=network.allow:openai.com'. Each --ack names the agent it
is for, and its token is appended to that agent's binding only — the same token
for another agent of the same profile needs its own. The profile and the bindings are
written together (BindingStore.save_profile): under the one bindings writer lock
(the profile library has none of its own), the bindings are re-read and the edit
judged against them as they are then; the bindings that gained consent are written
first and the profile second; a failed profile write puts the bindings back before the
lock is released. Bindings first because a crash between the two renames then leaves
consent to a loosening nothing performs yet — inert, and reported stale by leash validate — never a loosened profile beside agents that have not consented, which run
start would refuse. One SIGHUP follows the whole write. leash bind re-checks under
the same lock that the profile it judged is still the one on disk. Consent for the
global binding is never taken this way: it goes through leash bind global.
scope: global is the dangerous one, and it is gated the same way. A global binding
is folded into the floor rather than overlaid, and the floor is the baseline every
comparison is made against, so it classifies nothing: without a gate, folding
{fail_mode: open, network.mode: blacklist} over a strict floor takes the whole host to
default-allow with zero loosenings reported. So the fragment is judged as an overlay
against the unfolded floor first, by both the resolver and the CLI, and leash bind global says out loud that it changes the floor for every subject on the host.
Removing a binding is judged too. A profile usually tightens — a stricter
whitelist, an extra deny, an inspection raised to block — so leash unbind can
widen what a subject may do, and unbinding a scope: global profile that narrowed the
floor widens it for every subject on the host. So an unbind resolves the subject
twice, with the binding and without it, exactly as run start would, and asks the same
loosening rules of every field that differs; a deny the removal drops is judged as
the engine's own unset.deny. A removal that only tightens proceeds; one that loosens
is refused until every finding is acknowledged with --ack, in the same words and
with the same re-run line bind gives. Because an unbind deletes the binding the
acknowledgements would have lived on, they are kept in an append-only removal
ledger, /var/lib/leash/unbinds.jsonl, written before the binding is removed — and an
unbind whose record cannot be written does not happen. A dangling binding (its
profile deleted) holds its agent fail-closed, denied everything; removing it returns the
agent to the floor, so it needs one acknowledgement of its own,
fail_closed:<profile>. And an unbind Leash cannot judge — the floor unreadable, or the
subject still unresolvable without the binding — is refused rather than waved through.
The consent check is made against the floor file alone, which is an approximation in
one direction: a field a scope: global binding already loosened is reported here and
not at run start (a refusal where none was needed — closed), while a field it already
tightened may loosen against that stricter folded base and not against the floor
file. That direction is caught by the resolver at run start, against the real base,
with the same token — so the residual error is a bind refused late rather than a subject
loosened silently.
Picking up a change without disturbing anybody
SIGHUP reconciles the subjects in place. The service re-reads passwd,
re-resolves every subject's policy, and changes only what differs:
- a new subject gets its own nftables chain and its own proxy listener, opened inside the already-running proxy loop;
- a changed policy re-applies that one subject's chain;
- an unchanged subject is not re-applied — it is not touched at all, which is the point. A re-apply takes a chain down and puts it back, so "byte-identical result" and "incremental" are not the same thing.
Restarting the service would have been the wrong instrument, and that is why this exists: a stop releases every subject's chain and stops the filesystem watcher, so restarting to pick up one agent's policy briefly un-confines all the others — the "every UID on the host at once" window the per-UID design exists to remove.
A reconcile fails the way everything else here fails. One subject whose policy will not
resolve, whose chain will not apply or whose listener will not bind is denied alone and
said out loud; every other subject is untouched. A reconcile that fails wholesale —
passwd unreadable, the port map corrupt — changes nothing at all and leaves what is
already applied in place: "I could not look" must never become "nothing to confine". A
SIGHUP that arrives during a reconcile is coalesced into one further pass afterwards
rather than re-entering a half-applied one.
A reconcile cannot freeze the host. It reads passwd, YAML and the port map — files on the root mount the filesystem watcher marks — and on 2026-10-14 that froze every process on the machine, because the watcher was a thread of the same process and the passwd read held the GIL it needed. The watcher is now a separate process (see the filesystem enforcer below). Each reconcile is also armed with a stack dump: if one ever runs past 30 seconds, every thread's Python stack goes to the journal.
A subject that leaves discovery loses its policy and keeps its mechanism: dropped
from the proxy's map, so every connection on its listener is refused and logged under
its own UID, and denied on every filesystem and exec event. Its chain is deliberately
kept. An account removed from Sentinel is never userdel'd, so the UID still exists and
may still have live processes — and "renamed out of the prefix", "really deleted" and "the
name service answered with a short list" are indistinguishable from here. Releasing on
the strength of the third would un-confine a live agent. Releasing for real is the
service's stop path, which drops every UID the port map holds.
Who sends the signal. Two senders, for two kinds of change.
Leash's own CLI sends it after every successful policy write — leash profile save, leash profile delete, leash bind, leash unbind — because those are the
changes nothing else would ever notice. Before that, a binding written today took
effect only when some unrelated lifecycle event happened to signal, or when the service
was next restarted, which reads to an operator as the binding not working at all.
--no-reconcile skips it, and a signal that fails is reported with the command to run
by hand and a non-zero exit: the policy is on disk and is not in force, and that is the
one state that must not exit 0.
Sentinel sends it at the lifecycle points that change what discovery would
return: a new agent account (sentinel/agent_users.py::reconcile_leash). A
team-membership change does not signal any more — team is not a policy tier, so it
changes nothing Leash resolves. Releasing an agent account does not
signal, and that is
correct rather than an omission — discovery is keyed on the leash- passwd prefix and
explicitly not on sentinel-agents, so a released account stays a subject with the same
policy.
Both senders run exactly the same thing: systemctl kill --kill-whom=main --signal=HUP leash.service. systemctl reload is not it — the unit ships no ExecReload=, so a
reload is refused outright on every host that already has Leash installed. And
--kill-whom=main is load-bearing: Leash shells nft out to short-lived children, and
SIGHUP terminates a process that has no handler for it, so a cgroup-wide signal can kill
an nft in flight and fail the very apply it was sent to cause.
The first agent on a host is the one case SIGHUP cannot serve. A service that
started with no agent accounts is running the single-subject proxy — one redirect,
one listener, no per-subject listener set for a new subject to join — so it cannot grow
the host's first agent into itself; it says so on the first SIGHUP instead of
ignoring it. Sentinel therefore systemctl restarts the service for that one agent,
which is harmless at exactly that moment because no agent has enforcement to lose yet —
the same argument nft.drop_legacy_shape makes for migrating at startup. It is never the
general mechanism: a restart runs ExecStopPost → leash teardown, dropping every other
subject's chains mid-run, which is why the reconcile exists at all.
One engine, four enforcers
The reusable core is a policy engine that every surface shares:
identify UID → resolve the subject → engine.decide(surface, subject) → allow | block(reason) → log
The engine is pure logic with no OS calls, so it is fully unit-tested. Enforcers are thin adapters that intercept at the OS layer, resolve the subject, call the engine, and apply the result.
- Surfaces & subjects: network
{domain, ip, port}, filesystem{path}, command{binary}, tool{name, inputs}. - Per-surface mode:
follow-whitelist(default-deny — only listed entries are allowed) orfollow-blacklist(default-allow — only listed entries are blocked). - Matching: domains match exact or by suffix (so
github.commatchesapi.github.com), with*.wildcard support; IPs match by CIDR network; paths match by exact, directory-prefix, or glob; commands match by basename or absolute path; tools match by exact name. - Entries that can never match: each matcher's grammar sits beside it in
engine/match.py(endpoint_entry_problem,path_entry_problem,command_entry_problem), andconfig.inert_entrieswalks a floor, a profile or a resolvedConfigwith them. A network entry is a host, IP or CIDR, optionally:port, and never a URL. Sohttp://host,host/path,user@host, whitespace, a non-leading*, a leading., a non-numeric or out-of-range port and non-ASCII names are inert. So are a relative or~path infilesystemand a relative path incommands, plus adirectentry nft cannot render. The rule is asymmetric. On write such an entry is refused (leash <surface> allow|deny,ProfileStore.save,BindingStore.save_profileandbind;--jsongiveskind: entryandinert_entries). On load it is accepted and reported: the floor at startup, every profile inleash validate, and each subject's resolved policy on every plan and reconcile (subjects._warn_inert). Refusing at load would stop resolving every subject bound to the profile. That subject would be denied everything over an entry that, in an allow list, already fails closed on its own. An inert deny is the case that weakens a policy, since it blocks nothing. It is reported as critical, soleash validateexits 1 on it. - Decision: a
Decision(action, reason)ofalloworblock. Every decision is written to the JSONL decision log.
Enforcers
Network (transparent proxy) — kernel-enforced for the UID
nftablesrules match the leashed UID's outbound TCP (meta skuid) over both IPv4 and IPv6 and redirect it to a local Leash proxy (the service installs these at runtime; the installer does not, to keep installs offline and testable).- One chain per UID. The hooked chain is a dispatcher holding nothing but
meta skuid <uid> jump agent_<uid>; a UID's actual rules live in its own hookless chain, named after the UID. Sonft list table ip leashshows oneagent_<uid>chain per leashed user, and onboarding or releasing one user adds or deletes exactly that chain and its one dispatch rule. Nothing flushes a chain or deletes a table, because both tables and the dispatcher are shared: with more than one leashed UID on a host, either would silently disarm the others while their config still said they were leashed. - One listener per UID. The proxy opens a listener per leashed UID, on that
UID's own proxy port, and each listener carries only that subject's resolved
policy. So the port a connection arrives on is what identifies the subject — no pid
lookup — and every decision is judged against that subject's policy and written
to the decision log under that subject's UID. The per-UID ports come from
/var/lib/leash/ports.json, which is how the kernel's redirect and the proxy's listeners still agree about which port belongs to whom after a restart. A single-agent install is the same code with one subject in the map, on127.0.0.1:18443— a port inside the reserved range, which the port map never hands to a subject. - For each redirected connection, the proxy peeks the first bytes to read the
destination host from the HTTP
Host:header or the TLSSNI— with no decryption — and reads the original destination IP and port viaSO_ORIGINAL_DST. It callsdecide_network({domain, ip, port}). - A connection that was not redirected is refused, not judged.
SO_ORIGINAL_DSTreports the destination the agent asked for on a redirected connection, but on a connection made straight to a listener it reports the address that listener is bound to — so the proxy compares the two and closes the second kind, logging it as a critical decision with no policy consulted. Otherwise a leashed UID could connect to127.0.0.1:<another agent's port>, land on that agent's listener, and have its traffic judged under that agent's policy and recorded as that agent's UID — one agent borrowing another's egress policy, by guessing a port. Because the comparison is against the listener's own address it needs no port range and no notion of loopback: it covers the single-agent listener on127.0.0.1:18443as well as the per-UID ones, and a peer that has no leash rules at all (root, another local service) connecting to a listener directly — which no nftables rule can prevent, and which would otherwise be recorded as that subject's own egress. - A destination that cannot be read is refused too.
SO_ORIGINAL_DSTis readable only on a connection the kernel redirected, so a failure to read it means the proxy cannot show the connection was redirected — the same condition as above, and it gets the same answer. It used to substitute an empty destination and judge the connection anyway, which in blacklist mode matched nothing and returned allow: the one state in which the proxy knew nothing about where a connection was going was the state in which it permitted it. - The loopback exemption excludes the proxy port range (
18000-18999) for the same reason: the attempt is not exempted, so it falls through to the UID's own redirect, lands on its own listener, and is refused there — it never crosses to another subject at all. Loopback is otherwise still exempt, because a leashed agent legitimately talks to local services (the Sentinel broker socket, a local model backend) and proxying those would break them and recurse into the proxy. Consequence: those 1000 ports on loopback are reserved for Leash, so a leashed UID cannot reach a local service of its own on one of them. The single-agent listener is one of those ports (18443), so an attempt at it is refused at the attempting UID's own listener and logged under the attempting UID — which is the whole reason it was moved there from8443, outside the range, where the refusal landed on the subject that had been aimed at instead. - A UID with a listener but no resolved policy refuses every connection, rather than falling back to a default or borrowing another subject's. Its listener is still opened, because its nft chain may still redirect there from a previous run and a connection refused by a listener that does not exist is refused with nothing written down.
- Allow → it opens a connection to the real destination and splices both directions through untouched. Block → it closes the connection (a reset) and logs the reason.
- Because the redirect is keyed to the UID in the kernel, the agent cannot dodge it by ignoring proxy environment variables.
- UDP is dropped, not proxied. A TCP proxy cannot broker UDP, and forwarding what
you cannot inspect is not enforcement — so a filter chain drops it for the leashed
UID. That is also what closes QUIC / HTTP3 on UDP/443, which a modern client will
otherwise negotiate straight past the proxy. The one exception is DNS to the host's
own resolvers (from
/etc/resolv.conf, ornetwork.dns_resolvers), because nothing resolves without it; UDP/53 to arbitrary addresses stays closed, since that is a tunnel.network.udp: allowrestores unfiltered UDP as an explicit choice. - Scope: TCP over IPv4 and IPv6 (HTTP/HTTPS and any TCP by IP/port), with UDP
dropped. Raw sockets are not intercepted — a process with
CAP_NET_RAWis outside this boundary. - Upgrading a host that ran Leash before the per-uid chains: the service drops
any leash table still in the old shape at startup, before it applies the first
subject, and says so on stderr. That is handled for you. The reason it has to be:
the old shape put a UID's rules straight into the hooked chain, where a per-subject
release cannot see them, and a stale
redirectis a terminal verdict that wins over the new dispatch rule — so the UID would stay frozen on its pre-upgrade policy whileleash statusreported the new one. If you would rather not rely on the startup migration, stop the service and deletetable ip leash,table ip6 leashandtable inet leash_filter(or just reboot) before starting the new version. - If
nftis unavailable the iptables fallback covers IPv4 TCP only, and says so on stderr rather than letting you assume parity. The fallback is single-UID only: it appends into the built-innat OUTPUTchain, which cannot express a per-UID dispatch, so enforcing several UIDs refuses to run withoutnftrather than quietly applying one user's policy to all of them.
Filesystem + Commands (one fanotify daemon) — kernel-enforced for the UID
- A single privileged fanotify daemon marks the mounts with both
FAN_OPEN_PERM(filesystem opens —cat/ls/writes) andFAN_OPEN_EXEC_PERM(process execution). Both surfaces ride one daemon because fanotify exposes both event types under one mark. - One daemon for every UID, with a UID → policy map. fanotify marks are per mount, so a second daemon on the same mount would see the same events and both would have to answer them — and re-marking as agents come and go would be wasteful and racy. So where the network side gets one listener per UID, the filesystem side keeps one watcher and holds a map from UID to that subject's resolved policy. The marks are the union of every subject's configured paths.
- For each permission event it resolves the acting PID's UID (via
/proc/<pid>/status) and looks that UID up in the map. Only for a UID in the map does it resolve the target path (via/proc/self/fd) and ask the engine —decide_command({binary})for exec events,decide_path({path})for opens — against that UID's own policy, and write the decision to the log under that UID. - A UID that is not in the map is allowed straight through, and that is not the
same posture as the proxy's. A proxy listener exists only because a subject
exists, so a listener with no policy refuses every connection. fanotify reports
events from every process on the host — root, systemd, sshd, the package manager,
your editor — so an unmapped UID is not a misconfiguration, it is the overwhelming
majority of events, and treating it as "no policy, therefore block" would deny the
whole system including whatever you would need to recover it. A UID that is in
the map with no resolved policy is the misconfiguration case: it is logged
criticalonce at startup and every one of its events is denied (or allowed underfail_mode: open), one UID at a time. - The watcher is its own process, not a thread of the service.
leash.servicespawns it (python -m leash.enforcers.fswatch) with the starting map, and it is the only process that holds the fanotify group. Until 2026-10-14 it was a thread, and everySIGHUPfroze the whole host: the reconcile's passwd read (pwd.getpwall(), which in CPython 3.12 holds the GIL while glibc opens/etc/passwd) opened a file on the marked root mount, the kernel held that open for the watcher's verdict, and the watcher thread was waiting for the GIL the opener held. Every other process on the host that opened a file queued behind it. A separate process shares no GIL and no lock with anything the service does, so the reconcile — or the proxy, or a future change — can no longer starve the thing that answers. - The watcher can still wedge itself (it is single-threaded, so an open it makes on
a marked mount has nobody else to answer it), so that is closed off and bounded:
everything its event loop needs is imported before the marks go on and any later
import is refused; source-line reads for tracebacks are disabled; the decision log is
written through a descriptor opened before the marks. And a C-level watchdog
(
faulthandler, no GIL needed) is re-armed on every pass of the loop: if the loop stops for 10 seconds it dumps every Python stack to the journal and exits. Closing a fanotify group answers every pending event allow, so the host is released; the service respawns the watcher from its current map (chains and listeners untouched), and exits for systemd to restart it only if the watcher dies more than 3 times in 10 minutes. The kernel itself offers no time-out: a pending permission event waits forever (killable only), and thefs.fanotify.watchdog_timeoutsysctl on recent kernels only logs a slow listener. - Subjects can be added and removed while the daemon runs, by changing the map:
the service holds it and forwards every change to the watcher over a pipe, waiting
for the watcher's acknowledgement, so a new subject is denied in the watcher before
its chain goes in. A subject is never removed from this map — an absent UID is an
allowed UID — only set to "no policy", which denies. Two consequences worth
knowing: a subject added between mark re-scans is unwatched for up to 5 seconds,
and the UID on an event comes from
/proc/<pid>/statusafter the event, so a recycled PID can still be attributed to the wrong subject (a pre-existing race; closing it needsFAN_REPORT_PIDFD). - It then responds
FAN_ALLOWorFAN_DENY. A denial surfaces to the agent asEPERM. - Requires: a privileged daemon and a Linux kernel ≥ 5.0 (for
FAN_OPEN_EXEC_PERM).
Tools (cooperative library) — harness-enforced
- A tools module: a registry (per-tool allow/deny + mode) plus input
inspection that reuses Muzzle's detector patterns — secrets (API keys, tokens,
private keys), PII (emails), and privilege-escalation patterns
(
sudo,chmod +s,setuid,pkexec,doas,rm -rf /). check_tool(name, inputs)first checks the registry; if the tool name is allowed, it inspects the inputs and blocks if any flagged category is configured toblock. Sensitive matches are masked when logged.- The enforcement point is the harness (Sentinel), which calls Leash before
running a tool. v1 exposes it via the
leashCLI/library so it is testable now and drop-in for the harness later. Unlike the other three surfaces, this one is cooperative — it is only as strong as the harness that calls it.
Fidelity matrix
| Surface | Mechanism | Enforcement |
|---|---|---|
| Network (TCP) | nftables meta skuid → one transparent-proxy listener per UID | Kernel-enforced per leashed UID |
| Filesystem | fanotify FAN_OPEN_PERM → one watcher, a UID → policy map | Kernel-enforced per leashed UID, each against its own policy (privileged daemon) |
| Commands | fanotify FAN_OPEN_EXEC_PERM → the same watcher and map | Kernel-enforced per leashed UID, each against its own policy (privileged daemon) |
| Tools | registry + input inspection library | Cooperative — enforced by the harness (Sentinel) |
| Network (IPv6 TCP) | nftables ip6 meta skuid → that UID's same listener | Kernel-enforced per leashed UID |
| Network (UDP) | nftables inet filter chain → drop (DNS to host resolvers excepted) | Kernel-enforced for the leashed UID |
| Network (raw sockets) | not intercepted | Outside the boundary — needs CAP_NET_RAW |
"Kernel-enforced" means the interception point is unbypassable — the UID's traffic, file opens, and execs are forced through Leash. It does not mean a determined agent can't defeat a name-based rule. v1 is a preview. Two gaps are closed and one is not:
- Closed: when a decision is made on a domain, the proxy connects to an address
it resolves rather than the one the agent supplied, so
allow: github.comcannot be used to reach an arbitrary IP behind a forged SNI. Ports are matched —host:portentries ([2001:db8::1]:443for IPv6) pin one. - Open: command rules match by name and path, so a copied or renamed binary evades
a blacklist.
leash validatereports this as a warning rather than leaving you to discover it. Use whitelist (default-deny) mode for honest blocking, especially for commands.
And the thing worth checking before you trust any of it: a surface in blacklist
mode is default-allow, and its allow: list is never consulted. leash validate
exits non-zero on that, because a policy which enforces nothing should not look like
one that does.
Configuration (leash.yaml)
The YAML config has these sections:
user— the leashed username (UID resolved at load).fail_mode—closed(default) oropen; governs every enforcer when it cannot reach a decision (see below).network/filesystem/commands— each a{ mode, allow, deny }surface policy.modeiswhitelistorblacklist.tools— the same{ mode, allow, deny }plusinspect, a map of category (secrets,pii,priv_esc) → action (block,log,allow).logging—{ level, decisions }.decisionsis a file path orstdout.enforcers—{ network, filesystem, tools }booleans to toggle each surface on or off.
Example:
user: fido
fail_mode: closed
network: { mode: whitelist, allow: ["github.com", "api.openai.com", "10.0.0.0/8"], deny: [] }
filesystem: { mode: whitelist, allow: ["/home/fido/work", "/tmp/fido"], deny: [] }
commands: { mode: blacklist, allow: [], deny: ["ssh", "rm", "curl"] }
tools: { mode: whitelist, allow: ["read_file", "search"], deny: [],
inspect: { secrets: block, pii: log, priv_esc: block } }
logging: { level: info, decisions: /var/log/leash/decisions.jsonl }
enforcers: { network: true, filesystem: true, tools: true }
Who may read and write the policy
/etc/leash/leash.yaml is 0640, owned root:<leashed user>. Leash itself runs
as root and writes it; the leashed user gets group read so a console can show
the policy — read through the group, never through ownership, because owning the
file would let the confined account rewrite its own leash.
It was previously created by a shell redirect with no mode set at all, which
means it took whatever the installer's umask gave it — usually 0644. A
world-readable egress allowlist and command denylist is not a catastrophe, but it
hands any local account the exact shape of the policy to plan around, and a mode
nobody declared is one that is right by luck. An install now states it, and
re-states it on upgrade so an existing box is repaired rather than left as it was.
CLI (leash)
Same ergonomics as muzzle. The config path comes from --config/-c or
LEASH_CONFIG (default /etc/leash/leash.yaml). With no subcommand, leash runs the
service.
leash init --user fido— record the leashed user (creating the config if needed).leash status— print the user, each surface's mode and list sizes, enforcer toggles, and the decision-log sink.leash validate— load and validate the config.leash net|fs|cmd|tool allow|deny <entry>— add an entry to a surface's allow or deny list. (net/fs/cmd/toolare aliases fornetwork/filesystem/commands/tools.)leash <surface> mode whitelist|blacklist— set the surface's mode.leash check <surface> <subject>— dry-run a decision against the live config (for network, passhost:port).leash logs --follow [--tail N]— tail the decision log.leash restart— restart the service. List/mode edits save and restart automatically.
Decision logging
Every decision is written as one JSON line to the configured logging.decisions sink
(a file path or stdout): {ts, ref, uid, surface, action, reason, subject}. The
short ref ties a block back to its log line. Sensitive tool-input matches (secrets,
PII) are masked before they are written, reusing Muzzle's masking approach.
Error handling / fail mode
fail_mode governs every enforcer. If the engine cannot decide or an enforcer hits an
internal error, closed blocks and open allows. It is per subject: with several
leashed UIDs on one host, a failure judging one subject's event is answered under that
subject's fail_mode, never another's. A fail_mode value that is neither closed
nor open — a typo — is treated as closed, so a misspelling cannot quietly disable
an enforcer. The shapes of a block differ by
surface: network block = connection reset; filesystem/exec block = EPERM; tool block
= an error to the caller. If a daemon is not running, that surface is simply
unenforced, and leash status reports each enforcer's toggle plainly.
Linux only
Leash relies on Linux kernel primitives (nftables UID matching, fanotify permission events) and is not portable to other platforms. The Python package imports cleanly on non-Linux hosts for development and unit testing — every Linux-only API is referenced lazily inside the runtime functions — but enforcement only runs on Linux.
Roadmap (post-v1)
- Sentinel integration: the fail-to-approval loop. A blocked action emits an approval request to Sentinel; a human approves; Leash allows it, either once or by adding a persistent whitelist entry. Tools enforcement becomes a first-class harness gate rather than a cooperative library.
- Broader network: per-API (path/method) rules. IPv6 TCP and UDP are no longer future work — IPv6 goes through the same proxy and UDP is dropped, with DNS to the host's own resolvers excepted.
- File-backed lists and an admin UI, mirroring Muzzle's later additions.
More
- Getting started and v1 scope:
products/leash/v1/README.md - CLI reference:
CLI.md
Diff from previous
--- HOWITWORKS@4
+++ HOWITWORKS@5
@@ -17,6 +17,205 @@
- The username is resolved to a UID at load (`pwd.getpwnam`). Every decision is keyed
to that UID.
+### More than one subject on a host
+
+A host can carry **many** leashed UIDs, and the service discovers them itself at
+startup rather than being told:
+
+- **Discovery reads the kernel's own database.** nftables matches `meta skuid <uid>`
+ and the fanotify daemon filters events by UID, so the subject *is* a UID — and the
+ one database that says which UIDs exist is the one the kernel resolves them
+ through. So the service enumerates passwd for accounts in the agent namespace
+ (`leash-*`, the prefix Sentinel mints an agent's Linux account with). A list passed
+ in from elsewhere could disagree with passwd by one character and produce a
+ confinement that silently applies to nobody. Group membership is **not** read:
+ Sentinel's `sentinel-team-*` groups give a team's agents their shared workspace,
+ which is file access, not Leash policy.
+- **Each subject's policy is resolved separately** — the global floor, then its own
+ agent overlay; two tiers, **global < agent** — and a subject whose policy cannot be
+ resolved (a dangling profile, an unacknowledged loosening) fails **that subject
+ only**: it is announced as a
+ critical warning, it is given no nftables chain at all, and every filesystem and
+ exec event it raises is denied. Every other subject's rules are untouched.
+- **A chain and a port are per account, for the account's life** — not per run. A
+ release never returns a port to the pool, so re-planning an unchanged host moves no
+ listener, and starting a run needs no privileged call.
+- **The single configured `user` stays a subject** when agent accounts appear. It owns
+ the state directory, the approvals queue and the run store, and it resolves to the
+ floor — with no tier of its own, so no `agent:` binding can re-point the control
+ plane's own confinement. On a host with **no** agent accounts the service takes the
+ original single-subject path unchanged and does not even create
+ `/var/lib/leash`.
+
+### Configuring one agent's policy
+
+The config file is the **floor**. On top of it, a **profile** is a named, reusable
+policy *fragment* (`/var/lib/leash/profiles/<name>.yaml`) and a **binding** attaches one
+profile to one scope — `global` or `agent:<account>` (`/var/lib/leash/bindings.yaml`).
+Team was a third scope until 2026-10-14; it was removed as a policy tier, and a team
+that should share a policy binds one profile to each of its agents. The operator
+surface is
+`leash profile save|list|show|delete`, `leash bind`, `leash unbind` and
+`leash bindings`.
+
+```bash
+sudo leash profile save web-research --from web-research.yaml
+sudo leash bind agent:leash-iris web-research
+```
+
+Four things that shape it.
+
+**A profile is a fragment, not a config.** Only the fields it means to set. Every key a
+tier writes is a deliberate value *from that tier*, so a profile carrying a whole
+config's worth of defaults would restate all four surfaces as policy — binding it to
+narrow one agent's network would also hand it `filesystem: whitelist` and default-deny
+the filesystem of an agent whose operator chose `blacklist` on purpose.
+
+**A scope is a Linux account name, never a display name.** Discovery reads passwd,
+so the account name is the only identity Leash has: the scope is `agent:leash-iris`. `agent:Iris` is refused rather than
+written, because it is a binding that would resolve for nobody and nothing anywhere
+would say so — and the account name cannot be derived back from a display name, since
+the slug is folded and collision-suffixed. Where the real account exists, the refusal
+names it.
+
+**Widening the floor needs explicit, per-subject consent.** A profile may tighten the
+floor freely; a change that *widens* what a subject may do is refused unless that
+binding acknowledges it by field **and value**. The refusal lists every loosening the
+profile performs — not the first — each in the engine's own words with the exact
+`--ack '<field>:<value>'` token, plus the whole command to re-run. The acknowledgement
+lives on the **binding** and never in the profile: one profile is reused by many
+subjects, so a profile that could consent on its own behalf would consent for every
+subject later bound to it. For the same reason, *editing* a profile revalidates every
+binding that names it and refuses an edit that would loosen a bound subject beyond its
+own acks — reuse means one keystroke reaches eight agents.
+
+**An edit can carry each bound agent's consent** (since 2026-10-14):
+`leash profile save web-research --ack 'agent:leash-iris=network.allow:openai.com'
+--ack 'agent:leash-vader=network.allow:openai.com'`. Each `--ack` names the agent it
+is for, and its token is appended to **that** agent's binding only — the same token
+for another agent of the same profile needs its own. The profile and the bindings are
+written **together** (`BindingStore.save_profile`): under the one bindings writer lock
+(the profile library has none of its own), the bindings are re-read and the edit
+judged against them as they are *then*; the bindings that gained consent are written
+first and the profile second; a failed profile write puts the bindings back before the
+lock is released. Bindings first because a crash between the two renames then leaves
+consent to a loosening nothing performs yet — inert, and reported stale by `leash
+validate` — never a loosened profile beside agents that have not consented, which run
+start would refuse. One `SIGHUP` follows the whole write. `leash bind` re-checks under
+the same lock that the profile it judged is still the one on disk. Consent for the
+`global` binding is never taken this way: it goes through `leash bind global`.
+
+**`scope: global` is the dangerous one, and it is gated the same way.** A global binding
+is *folded into the floor* rather than overlaid, and the floor is the baseline every
+comparison is made against, so it classifies nothing: without a gate, folding
+`{fail_mode: open, network.mode: blacklist}` over a strict floor takes the whole host to
+default-allow with **zero** loosenings reported. So the fragment is judged as an overlay
+against the unfolded floor first, by both the resolver and the CLI, and `leash bind
+global` says out loud that it changes the floor for every subject on the host.
+
+**Removing a binding is judged too.** A profile usually *tightens* — a stricter
+whitelist, an extra `deny`, an inspection raised to `block` — so `leash unbind` can
+widen what a subject may do, and unbinding a `scope: global` profile that narrowed the
+floor widens it for **every** subject on the host. So an unbind resolves the subject
+twice, with the binding and without it, exactly as run start would, and asks the same
+loosening rules of every field that differs; a `deny` the removal drops is judged as
+the engine's own `unset.deny`. A removal that only tightens proceeds; one that loosens
+is refused until every finding is acknowledged with `--ack`, in the same words and
+with the same re-run line `bind` gives. Because an unbind deletes the binding the
+acknowledgements would have lived on, they are kept in an append-only **removal
+ledger**, `/var/lib/leash/unbinds.jsonl`, written before the binding is removed — and an
+unbind whose record cannot be written does not happen. A **dangling** binding (its
+profile deleted) holds its agent fail-closed, denied everything; removing it returns the
+agent to the floor, so it needs one acknowledgement of its own,
+`fail_closed:<profile>`. And an unbind Leash cannot judge — the floor unreadable, or the
+subject still unresolvable without the binding — is refused rather than waved through.
+
+The consent check is made against the floor file alone, which is an approximation in
+one direction: a field a `scope: global` binding already loosened is reported here and
+not at run start (a refusal where none was needed — closed), while a field it already
+*tightened* may loosen against that stricter folded base and not against the floor
+file. That direction is caught by the resolver at run start, against the real base,
+with the same token — so the residual error is a bind refused late rather than a subject
+loosened silently.
+
+### Picking up a change without disturbing anybody
+
+**`SIGHUP` reconciles the subjects in place.** The service re-reads passwd,
+re-resolves every subject's policy, and changes **only what differs**:
+
+- a **new** subject gets its own nftables chain and its own proxy listener, opened
+ inside the already-running proxy loop;
+- a **changed** policy re-applies that one subject's chain;
+- an **unchanged** subject is not re-applied — it is not touched at all, which is the
+ point. A re-apply takes a chain down and puts it back, so "byte-identical result" and
+ "incremental" are not the same thing.
+
+Restarting the service would have been the wrong instrument, and that is why this
+exists: a stop releases **every** subject's chain and stops the filesystem watcher, so
+restarting to pick up one agent's policy briefly un-confines all the others — the "every
+UID on the host at once" window the per-UID design exists to remove.
+
+A reconcile fails the way everything else here fails. One subject whose policy will not
+resolve, whose chain will not apply or whose listener will not bind is denied alone and
+said out loud; every other subject is untouched. A reconcile that fails *wholesale* —
+passwd unreadable, the port map corrupt — changes **nothing at all** and leaves what is
+already applied in place: "I could not look" must never become "nothing to confine". A
+`SIGHUP` that arrives *during* a reconcile is coalesced into one further pass afterwards
+rather than re-entering a half-applied one.
+
+A reconcile cannot freeze the host. It reads passwd, YAML and the port map — files on
+the root mount the filesystem watcher marks — and on 2026-10-14 that froze every
+process on the machine, because the watcher was a thread of the same process and the
+passwd read held the GIL it needed. The watcher is now a separate process (see the
+filesystem enforcer below). Each reconcile is also armed with a stack dump: if one ever
+runs past 30 seconds, every thread's Python stack goes to the journal.
+
+A subject that **leaves** discovery loses its policy and keeps its mechanism: dropped
+from the proxy's map, so every connection on its listener is refused and logged under
+its own UID, and denied on every filesystem and exec event. Its chain is deliberately
+kept. An account removed from Sentinel is never `userdel`'d, so the UID still exists and
+may still have live processes — and "renamed out of the prefix", "really deleted" and "the
+name service answered with a short list" are indistinguishable from here. Releasing on
+the strength of the third would un-confine a live agent. Releasing for real is the
+service's stop path, which drops every UID the port map holds.
+
+**Who sends the signal.** Two senders, for two kinds of change.
+
+Leash's own CLI sends it after every successful **policy** write — `leash profile
+save`, `leash profile delete`, `leash bind`, `leash unbind` — because those are the
+changes nothing else would ever notice. Before that, a binding written today took
+effect only when some unrelated lifecycle event happened to signal, or when the service
+was next restarted, which reads to an operator as the binding not working at all.
+`--no-reconcile` skips it, and a signal that *fails* is reported with the command to run
+by hand and a non-zero exit: the policy is on disk and is not in force, and that is the
+one state that must not exit 0.
+
+Sentinel sends it at the lifecycle points that change what **discovery** would
+return: a new agent account (`sentinel/agent_users.py::reconcile_leash`). A
+team-membership change does **not** signal any more — team is not a policy tier, so it
+changes nothing Leash resolves. Releasing an agent account does **not**
+signal, and that is
+correct rather than an omission — discovery is keyed on the `leash-` passwd prefix and
+explicitly not on `sentinel-agents`, so a released account stays a subject with the same
+policy.
+
+Both senders run exactly the same thing: `systemctl kill --kill-whom=main --signal=HUP
+leash.service`. `systemctl reload` is not it — the unit ships no `ExecReload=`, so a
+reload is refused outright on every host that already has Leash installed. And
+`--kill-whom=main` is load-bearing: Leash shells `nft` out to short-lived children, and
+SIGHUP terminates a process that has no handler for it, so a cgroup-wide signal can kill
+an `nft` in flight and fail the very apply it was sent to cause.
+
+**The first agent on a host is the one case `SIGHUP` cannot serve.** A service that
+started with **no** agent accounts is running the single-subject proxy — one redirect,
+one listener, no per-subject listener set for a new subject to join — so it cannot grow
+the host's *first* agent into itself; it says so on the first `SIGHUP` instead of
+ignoring it. Sentinel therefore `systemctl restart`s the service for that one agent,
+which is harmless at exactly that moment because no agent has enforcement to lose yet —
+the same argument `nft.drop_legacy_shape` makes for migrating at startup. It is never the
+general mechanism: a restart runs `ExecStopPost` → `leash teardown`, dropping every other
+subject's chains mid-run, which is why the reconcile exists at all.
+
## One engine, four enforcers
The reusable core is a **policy engine** that every surface shares:
@@ -37,6 +236,23 @@
`api.github.com`), with `*.` wildcard support; IPs match by CIDR network; paths
match by exact, directory-prefix, or glob; commands match by basename or absolute
path; tools match by exact name.
+- **Entries that can never match:** each matcher's grammar sits beside it in
+ `engine/match.py` (`endpoint_entry_problem`, `path_entry_problem`,
+ `command_entry_problem`), and `config.inert_entries` walks a floor, a profile or a
+ resolved `Config` with them. A network entry is a host, IP or CIDR, optionally
+ `:port`, and never a URL. So `http://host`, `host/path`, `user@host`, whitespace,
+ a non-leading `*`, a leading `.`, a non-numeric or out-of-range port and non-ASCII
+ names are inert. So are a relative or `~` path in `filesystem` and a relative path
+ in `commands`, plus a `direct` entry nft cannot render. The rule is **asymmetric**.
+ On **write** such an entry is refused (`leash <surface> allow|deny`,
+ `ProfileStore.save`, `BindingStore.save_profile` and `bind`; `--json` gives
+ `kind: entry` and `inert_entries`). On **load** it is accepted and reported: the
+ floor at startup, every profile in `leash validate`, and each subject's resolved
+ policy on every plan and reconcile (`subjects._warn_inert`). Refusing at load
+ would stop resolving every subject bound to the profile. That subject would be
+ denied everything over an entry that, in an allow list, already fails closed on
+ its own. An inert **deny** is the case that weakens a policy, since it blocks
+ nothing. It is reported as **critical**, so `leash validate` exits 1 on it.
- **Decision:** a `Decision(action, reason)` of `allow` or `block`. Every decision is
written to the JSONL decision log.
@@ -44,13 +260,69 @@
### Network (transparent proxy) — kernel-enforced for the UID
-- `nftables` rules match the leashed UID's outbound TCP (`--uid-owner`) over **both
+- `nftables` rules match the leashed UID's outbound TCP (`meta skuid`) over **both
IPv4 and IPv6** and **redirect it to a local Leash proxy** (the service installs
these at runtime; the installer does not, to keep installs offline and testable).
+- **One chain per UID.** The hooked chain is a *dispatcher* holding nothing but
+ `meta skuid <uid> jump agent_<uid>`; a UID's actual rules live in its own hookless
+ chain, named after the UID. So `nft list table ip leash` shows one `agent_<uid>`
+ chain per leashed user, and onboarding or releasing one user adds or deletes
+ exactly that chain and its one dispatch rule. Nothing flushes a chain or deletes a
+ table, because both tables and the dispatcher are shared: with more than one
+ leashed UID on a host, either would silently disarm the others while their config
+ still said they were leashed.
+- **One listener per UID.** The proxy opens a listener per leashed UID, on that
+ UID's own proxy port, and each listener carries only that subject's resolved
+ policy. So the port a connection arrives on is what identifies the subject — no pid
+ lookup — and every decision is judged against *that* subject's policy and written
+ to the decision log under *that* subject's UID. The per-UID ports come from
+ `/var/lib/leash/ports.json`, which is how the kernel's redirect and the proxy's
+ listeners still agree about which port belongs to whom after a restart. A
+ single-agent install is the same code with one subject in the map, on
+ `127.0.0.1:18443` — a port inside the reserved range, which the port map never
+ hands to a subject.
- For each redirected connection, the proxy peeks the first bytes to read the
destination host from the **HTTP `Host:` header** or the **TLS `SNI`** — with **no
decryption** — and reads the original destination IP and port via
`SO_ORIGINAL_DST`. It calls `decide_network({domain, ip, port})`.
+- **A connection that was not redirected is refused, not judged.**
+ `SO_ORIGINAL_DST` reports the destination the agent asked for on a redirected
+ connection, but on a connection made straight to a listener it reports the address
+ that listener is *bound to* — so the proxy compares the two and closes the second
+ kind, logging it as a critical decision with no policy consulted. Otherwise a
+ leashed UID could connect to `127.0.0.1:<another agent's port>`, land on that
+ agent's listener, and have its traffic judged under that agent's policy and
+ recorded as that agent's UID — one agent borrowing another's egress policy, by
+ guessing a port. Because the comparison is against the listener's own address it
+ needs no port range and no notion of loopback: it covers the single-agent listener
+ on `127.0.0.1:18443` as well as the per-UID ones, and a peer that has **no** leash
+ rules at all (root, another local service) connecting to a listener directly —
+ which no nftables rule can prevent, and which would otherwise be recorded as that
+ subject's own egress.
+- **A destination that cannot be read is refused too.** `SO_ORIGINAL_DST` is
+ readable only on a connection the kernel redirected, so a failure to read it means
+ the proxy cannot show the connection *was* redirected — the same condition as
+ above, and it gets the same answer. It used to substitute an empty destination and
+ judge the connection anyway, which in blacklist mode matched nothing and returned
+ **allow**: the one state in which the proxy knew nothing about where a connection
+ was going was the state in which it permitted it.
+- **The loopback exemption excludes the proxy port range** (`18000-18999`) for the
+ same reason: the attempt is not exempted, so it falls through to the UID's own
+ redirect, lands on its own listener, and is refused there — it never crosses to
+ another subject at all. Loopback is otherwise still exempt, because a leashed agent
+ legitimately talks to local services (the Sentinel broker socket, a local model
+ backend) and proxying those would break them and recurse into the proxy.
+ **Consequence:** those 1000 ports on loopback are reserved for Leash, so a leashed
+ UID cannot reach a local service of its own on one of them. The single-agent
+ listener is one of those ports (`18443`), so an attempt at *it* is refused at the
+ attempting UID's own listener and logged under the attempting UID — which is the
+ whole reason it was moved there from `8443`, outside the range, where the refusal
+ landed on the subject that had been aimed at instead.
+- **A UID with a listener but no resolved policy refuses every connection**, rather
+ than falling back to a default or borrowing another subject's. Its listener is
+ still opened, because its nft chain may still redirect there from a previous run
+ and a connection refused by a listener that does not exist is refused with nothing
+ written down.
- **Allow** → it opens a connection to the real destination and splices both
directions through untouched. **Block** → it closes the connection (a reset) and
logs the reason.
@@ -66,20 +338,81 @@
- **Scope:** TCP over IPv4 and IPv6 (HTTP/HTTPS and any TCP by IP/port), with UDP
dropped. Raw sockets are not intercepted — a process with `CAP_NET_RAW` is outside
this boundary.
+- **Upgrading a host that ran Leash before the per-uid chains:** the service drops
+ any leash table still in the old shape at startup, before it applies the first
+ subject, and says so on stderr. That is handled for you. The reason it has to be:
+ the old shape put a UID's rules straight into the hooked chain, where a per-subject
+ release cannot see them, and a stale `redirect` is a terminal verdict that wins
+ over the new dispatch rule — so the UID would stay frozen on its pre-upgrade
+ policy while `leash status` reported the new one. If you would rather not rely on
+ the startup migration, stop the service and delete `table ip leash`,
+ `table ip6 leash` and `table inet leash_filter` (or just reboot) before starting
+ the new version.
- If `nft` is unavailable the **iptables fallback covers IPv4 TCP only**, and says so
- on stderr rather than letting you assume parity.
+ on stderr rather than letting you assume parity. The fallback is **single-UID
+ only**: it appends into the built-in `nat OUTPUT` chain, which cannot express a
+ per-UID dispatch, so enforcing *several* UIDs refuses to run without `nft` rather
+ than quietly applying one user's policy to all of them.
### Filesystem + Commands (one fanotify daemon) — kernel-enforced for the UID
-- A single privileged **fanotify** daemon marks the mount with both
+- A single privileged **fanotify** daemon marks the mounts with both
`FAN_OPEN_PERM` (filesystem opens — `cat`/`ls`/writes) and `FAN_OPEN_EXEC_PERM`
(process execution). Both surfaces ride one daemon because fanotify exposes both
event types under one mark.
+- **One daemon for every UID, with a UID → policy map.** fanotify marks are per
+ *mount*, so a second daemon on the same mount would see the same events and both
+ would have to answer them — and re-marking as agents come and go would be wasteful
+ and racy. So where the network side gets one listener per UID, the filesystem side
+ keeps one watcher and holds a map from UID to that subject's resolved policy. The
+ marks are the union of every subject's configured paths.
- For each permission event it resolves the acting PID's UID (via `/proc/<pid>/status`)
- and the target path (via `/proc/self/fd`). **Only if it is the leashed UID** does it
- ask the engine: `decide_command({binary})` for exec events,
- `decide_path({path})` for opens. Anything from another UID is allowed straight
- through.
+ and looks that UID up in the map. **Only for a UID in the map** does it resolve the
+ target path (via `/proc/self/fd`) and ask the engine — `decide_command({binary})`
+ for exec events, `decide_path({path})` for opens — **against that UID's own
+ policy**, and write the decision to the log under **that UID**.
+- **A UID that is not in the map is allowed straight through, and that is not the
+ same posture as the proxy's.** A proxy listener exists only because a subject
+ exists, so a listener with no policy refuses every connection. fanotify reports
+ events from *every* process on the host — root, systemd, sshd, the package manager,
+ your editor — so an unmapped UID is not a misconfiguration, it is the overwhelming
+ majority of events, and treating it as "no policy, therefore block" would deny the
+ whole system including whatever you would need to recover it. A UID that *is* in
+ the map with no resolved policy is the misconfiguration case: it is logged `critical`
+ once at startup and every one of its events is denied (or allowed under
+ `fail_mode: open`), one UID at a time.
+- **The watcher is its own process, not a thread of the service.** `leash.service`
+ spawns it (`python -m leash.enforcers.fswatch`) with the starting map, and it is the
+ only process that holds the fanotify group. Until 2026-10-14 it was a thread, and
+ every `SIGHUP` froze the **whole host**: the reconcile's passwd read
+ (`pwd.getpwall()`, which in CPython 3.12 holds the GIL while glibc opens
+ `/etc/passwd`) opened a file on the marked root mount, the kernel held that open for
+ the watcher's verdict, and the watcher thread was waiting for the GIL the opener held.
+ Every other process on the host that opened a file queued behind it. A separate
+ process shares no GIL and no lock with anything the service does, so the reconcile
+ — or the proxy, or a future change — can no longer starve the thing that answers.
+- **The watcher can still wedge itself** (it is single-threaded, so an open it makes on
+ a marked mount has nobody else to answer it), so that is closed off and bounded:
+ everything its event loop needs is imported before the marks go on and any later
+ import is refused; source-line reads for tracebacks are disabled; the decision log is
+ written through a descriptor opened before the marks. And a C-level watchdog
+ (`faulthandler`, no GIL needed) is re-armed on every pass of the loop: if the loop
+ stops for **10 seconds** it dumps every Python stack to the journal and exits.
+ Closing a fanotify group answers every pending event *allow*, so the host is
+ released; the service respawns the watcher from its current map (chains and listeners
+ untouched), and exits for systemd to restart it only if the watcher dies more than
+ 3 times in 10 minutes. The kernel itself offers no time-out: a pending permission
+ event waits forever (killable only), and the `fs.fanotify.watchdog_timeout` sysctl on
+ recent kernels only logs a slow listener.
+- **Subjects can be added and removed while the daemon runs**, by changing the map:
+ the service holds it and forwards every change to the watcher over a pipe, waiting
+ for the watcher's acknowledgement, so a new subject is denied in the watcher before
+ its chain goes in. A subject is never *removed* from this map — an absent UID is an
+ allowed UID — only set to "no policy", which denies. Two consequences worth
+ knowing: a subject added between mark re-scans is unwatched for up to **5 seconds**,
+ and the UID on an event comes from `/proc/<pid>/status` *after* the event, so a
+ recycled PID can still be attributed to the wrong subject (a pre-existing race;
+ closing it needs `FAN_REPORT_PIDFD`).
- It then responds `FAN_ALLOW` or `FAN_DENY`. A denial surfaces to the agent as
`EPERM`.
- **Requires:** a privileged daemon and a Linux kernel ≥ 5.0 (for
@@ -103,11 +436,11 @@
| Surface | Mechanism | Enforcement |
| --- | --- | --- |
-| Network (TCP) | nftables `--uid-owner` → transparent proxy | Kernel-enforced for the leashed UID |
-| Filesystem | fanotify `FAN_OPEN_PERM` | Kernel-enforced for the leashed UID (privileged daemon) |
-| Commands | fanotify `FAN_OPEN_EXEC_PERM` | Kernel-enforced for the leashed UID (privileged daemon) |
+| Network (TCP) | nftables `meta skuid` → one transparent-proxy listener per UID | Kernel-enforced per leashed UID |
+| Filesystem | fanotify `FAN_OPEN_PERM` → one watcher, a UID → policy map | Kernel-enforced per leashed UID, each against its own policy (privileged daemon) |
+| Commands | fanotify `FAN_OPEN_EXEC_PERM` → the same watcher and map | Kernel-enforced per leashed UID, each against its own policy (privileged daemon) |
| Tools | registry + input inspection library | Cooperative — enforced by the harness (Sentinel) |
-| Network (IPv6 TCP) | nftables `ip6` `--uid-owner` → same transparent proxy | Kernel-enforced for the leashed UID |
+| Network (IPv6 TCP) | nftables `ip6` `meta skuid` → that UID's same listener | Kernel-enforced per leashed UID |
| Network (UDP) | nftables `inet` filter chain → drop (DNS to host resolvers excepted) | Kernel-enforced for the leashed UID |
| Network (raw sockets) | not intercepted | Outside the boundary — needs `CAP_NET_RAW` |
@@ -202,7 +535,11 @@
## Error handling / fail mode
`fail_mode` governs every enforcer. If the engine cannot decide or an enforcer hits an
-internal error, `closed` blocks and `open` allows. The shapes of a block differ by
+internal error, `closed` blocks and `open` allows. It is **per subject**: with several
+leashed UIDs on one host, a failure judging one subject's event is answered under that
+subject's `fail_mode`, never another's. A `fail_mode` value that is neither `closed`
+nor `open` — a typo — is treated as `closed`, so a misspelling cannot quietly disable
+an enforcer. The shapes of a block differ by
surface: network block = connection reset; filesystem/exec block = `EPERM`; tool block
= an error to the caller. If a daemon is not running, that surface is simply
unenforced, and `leash status` reports each enforcer's toggle plainly.