How Muzzle Works

Muzzle is a transparent inspecting proxy. It takes over the host and port the client already uses, so you configure it once and then use the model provider normally — every request and response flows through Muzzle without any client changes. v1 covers Ollama, OpenAI, and Anthropic.

What "inspected" means, precisely. Muzzle parses the endpoints it has adapters for — the chat/completions, messages, image, speech and transcription routes. A request with a body to any other path has no inspector, and Muzzle refuses it with 403 rather than forwarding it. That is deliberate: it previously forwarded unknown paths verbatim, which meant /v1/completions, /v1/responses, /v1/messages and /v1/embeddings all reached the model with no detector in the path, and every new upstream API was a fresh silent bypass. Bodyless GETs (/v1/models, /api/tags) still pass — they are how clients discover the gateway and carry no prompt. policies.unknown_endpoints: forward restores the old behaviour as an explicit choice.

Request lifecycle

1. A client sends a request to a Muzzle proxy listener (the port it used to send to the provider directly).
2. The provider adapter parses the provider-native request into Muzzle's canonical model.
3. The policy engine inspects the canonical request (the input phase).
4. If allowed, Muzzle forwards to the configured upstream and reads the response. With stream_upstream on, it asks the model host to stream and REASSEMBLES the answer here; either way the whole response is in hand before step 5.
5. The adapter parses the response into the canonical model and the engine inspects it (the output phase), including any tool calls.
6. Muzzle renders the (possibly redacted or transformed) result back into the provider-native shape and returns it — re-emitting streamed responses chunk by chunk.

Endpoints that are not part of the inspected proxy surface pass through uninspected, so unrelated provider calls keep working.

Inspection

The engine is rules-first and fast. Detectors run before any optional model call:

• Prompt injection / jailbreak — detects attempts to override instructions.
• Secrets — API keys, tokens, private keys, and similar.
• PII — emails, card numbers, and other personal identifiers (linear scanning to avoid catastrophic backtracking on long input).
• Content policy — a configurable denylist of terms.

Each detector maps to a category, and each category has a configurable action per direction (input and output):

Tool calls are blocked by default.

Muzzle fails closed by default: if inspection cannot complete, the request is blocked rather than passed through. Set fail_mode: open to invert that.

The LLM judge

An optional LLM judge adds a model-based opinion on top of the rules. It is off by default and configured under llm_judge (enabled, model, base_url).

• When it runs. Only after the hard rule gates (content-policy, secrets, plus the other rule detectors) approve a request. If the rules already block, the judge never runs.
• What it checks. Both input and output, for prompt injection, PII, and banned subjects (see below).
• Direct, unfiltered call. The judge classifies by calling its own model endpoint (llm_judge.base_url) directly, which bypasses Muzzle's inspection. So the text being judged — and the judge's own reply — are never re-filtered by the input/output policies, and there is no recursion. Point base_url at a real model server, never at Muzzle's own listener.
• Actions. A judge finding maps to the configured action: prompt-injection and PII use the per-direction policies actions; each subject uses its own action. block rejects the request; log/allow let it through (judge findings are classifications, not spans, so redact/transform are recorded but not applied).
• Failure. If the judge model is unreachable, Muzzle honors fail_mode (closed → block, open → allow) and logs the outcome.

Subjects (topic policy)

content_rules.subjects is a list of banned topics, each with its own action, e.g. { name: "weapons", action: block }. When the judge is enabled it determines whether the input or output is about any configured subject and applies that subject's action.

A blocked request returns a provider-style error, e.g. for Ollama: {"error": "muzzle rejection: <reason> (log#<ref>)"}. The log#<ref> ties the rejection to a line in the decision log.

Architecture

• Canonical model + provider adapters. Each adapter (Ollama, OpenAI, Anthropic) parses provider-native requests/responses into one canonical request/response/ event/tool-call model and renders canonical results back out. The policy engine runs once against the canonical form, so all providers are inspected identically and new providers are additive work.
• Per-listener routing. Muzzle is configured via a YAML file. Each listener binds a host:port and has a kind: a proxy listener routes to one configured upstream; an admin listener serves the local admin UI. Multiple upstreams are supported by running multiple proxy listeners.
• Streaming, in two independent places. Towards the client: a streamed response is buffered, inspected as a whole, then re-emitted preserving chunk ordering, so policy applies to the complete answer while the client still sees a stream. Towards the model host (upstreams.<name>.stream_upstream, off by default): Muzzle asks the upstream to stream and reassembles the reply into the same complete body it would have returned anyway. Nothing a caller sees changes — what changes is that the hops in between stop being silent for the whole generation. That silence is why a long turn could be cut off by the shortest timeout in the path while the model was alive and working: a read timeout is a gap between bytes, not a total.
• A read budget per upstream. upstreams.<name>.read_timeout_s (default 300) is a property of the model behind the listener, not of the proxy — a local 7b and a remote reasoning cluster do not belong on one number. Measured on a live cluster at ~21 tokens/second, an unstreamed 300s budget is really a ~6,000-token ceiling, and a reasoning model spends its chain-of-thought from the same allowance. A timeout now names the upstream, how long it waited, and that the upstream was reachable, instead of reporting an internal error. The connect timeout stays at 10s, so a genuinely dead host still fails in seconds.
• Decision logging. Every decision is written as a JSON line to the configured logging.decisions sink (a file path or stdout), with the upstream, stage, action, categories, redaction count, reason, and a short ref.

Who may reach the admin listener

The admin listener serves the whole config and accepts a full rewrite, so it is the most sensitive surface Muzzle exposes. The gate is the caller, not the bind — the bind says who could reach the port, the peer address says who actually did:

• a configured admin.token must match, from any peer;
• with no token, a loopback peer is allowed — on a single-host install the OS boundary is the control, and this is the path a local console takes;
• with no token, a remote peer is refused with a 403 that says how to fix it. The request is refused, not the process, so the proxy listeners keep serving.

Installing Muzzle generates a token when the config has none. The loopback exemption assumes every local account is trusted, and that does not hold where Muzzle is deployed alongside a confined agent account: that account is a loopback peer, and without a token it could read the policy back or rewrite the detectors that inspect its own traffic. The token lives in muzzle.yaml, which is 0640 owned by the muzzle service, so an unprivileged local account cannot read it.

An operator's own admin.token or admin.token_env is left alone, and a generated one is never rotated by a later install — rotating it would break whatever was configured to use it.

Under Sentinel this means the console reaches Muzzle through Sentinel's root broker rather than calling the admin API directly. A script of your own that called it as an unprivileged user will now get 401: run it as root, or read the token from the config as root and send it as a bearer.

Configuration

The YAML config has these sections:

• listeners — list of { bind, kind, upstream }. Proxy listeners require an upstream; admin listeners ignore it.
• upstreams — map of name to { provider, base_url, read_timeout_s, stream_upstream }. The last two are per-upstream for the reasons above; read_timeout_s defaults to 300 and stream_upstream to false.
• policies.default — input and output maps of category → action.
• policies.overrides — per-upstream policy sets that replace the default for that upstream.
• fail_mode — closed (default) or open.
• llm_judge — { enabled, model, base_url }.
• secrets — { mode, file }. mode is rules (built-in regex, default), file (exact values from the encrypted file), or both.
• content_rules.denylist_terms / subjects — inline lists (apply to both directions), plus file references below.
• logging — { level, decisions }.

List & secrets files

To keep the YAML small, the bulk lists live in files (one entry per line), with a separate file per direction. The config holds the paths; the entries live in the files.

• secrets.file — encrypted file of literal secret values, one per line. It is encrypted with a generated Fernet key at <file>.key (chmod 600, owned by the service user). Never hand-edit the ciphertext — use the CLI or admin portal, which decrypt in memory and re-encrypt on save.

The installer pre-creates all of these under /etc/muzzle/ (empty list files, the secrets key, and an empty encrypted secrets file), wires their paths into the config, and chowns them to the service user — so they exist and are editable from the admin Files tab right after install. The engine also tolerates missing files, treating them as empty.

• content_rules.content_policy_files.{input,output} — plain term files for the content-policy detector, per direction.
• content_rules.subject_files.{input,output} — plain subject files, per direction; each line is name: action (action defaults to block).

Edit them with muzzle secrets|terms|subjects … or in the admin portal. Files are loaded when the config reloads, so the CLI restarts the service after an edit.

Operating a v1 install

The installed build runs as a systemd service and exposes an admin listener in the same process, so operators can manage it without leaving the VM.

• Admin UI (on the admin listener) — a tabbed page: a Configuration form (General, Upstreams, Listeners, Policies with default + per-upstream overrides, Content rules) where rows can be added and removed; an Advanced YAML tab for raw editing; a Logs tab that live-tails the decision log with filters; and a Simulation tab. Saving validates the config and reloads the running service.
• muzzle CLI (on PATH) — validate, edit, status, restart, logs (with --follow, --tail, and filters), add (denylist terms), input/output (set default policy actions), upstream list|remove|add (plus a muzzle HOST:PORT NAME shorthand to add one), and simulate. With no subcommand, muzzle runs the proxy. The config path comes from --config/-c or MUZZLE_CONFIG.
• Install/uninstall — install.sh writes /opt/muzzle/v1, /etc/muzzle/ muzzle.yaml, the systemd unit, /var/log/muzzle/decisions.jsonl, the /usr/local/bin/muzzle wrapper, and a virtualenv with all dependencies. uninstall.sh removes that full footprint.

Simulation

• The CLI (muzzle simulate) and the admin UI both dry-run policy decisions against the exact same engine the proxy uses in production.
• Simulation accepts provider-native payloads and reports the resulting decision, redactions, and tool-call handling without forwarding anything upstream — so a preview matches what live traffic would do.

Docs workflow

• The living files in products/muzzle remain the source of truth.
• An approved export step writes a fresh immutable MongoDB snapshot for each document, stamped with time, source path, commit, and content digest.
• The API serves the latest approved snapshot for each doc.
• The website can browse document history and diffs for each document version.

More

• Getting started and v1 scope: products/muzzle/v1/README.md
• CLI reference: CLI.md