Learner — For the Bots

An operator manual for AI coding agents (Claude Code, Codex, and similar). Dense and action-oriented: install, configure, edit, and operate Learner from a shell. Every path, command, and field is real (from products/learner/v1/). Humans: see README / HOWITWORKS.

Your mental model

Learner is a standalone learner CLI (no daemon, no service) that digests trusted sources on a topic — a web article URL, a YouTube transcript, a local file/PDF, or inline text — into a persistent per-topic SQLite knowledge base (sources + overlapping chunks + embeddings + LLM-distilled "knowledge items"), then serves RAG over it and, on demand, synthesizes a topic into a portable Sentinel SKILL.md playbook.

Concrete outputs: (a) the KB at <data_dir>/knowledge_base.sqlite3 (default ~/.learner/); (b) a skill file <slug>.skill.md (or .skill.zip) whose YAML frontmatter (name, description) is byte-compatible with Sentinel's skill importer; (c) a JSON migration bundle from export.

Prerequisites & install

• Python ≥ 3.12; Linux for the packaged install. Needs a live embeddings + chat backend — Ollama by default (http://127.0.0.1:11434) with nomic-embed-text (embed) and gpt-oss:20b (chat) pulled. No systemd unit, no service user.
• Base deps are slim (langchain-{ollama,openai,anthropic}, pypdf, youtube-transcript-api, beautifulsoup4, …). torch / sentence-transformers are NOT in the base install — only the hf extra adds local HuggingFace embeddings.

sudo bash install.sh                          # in products/learner/v1/
sudo LEARNER_INSTALL_EXTRAS=hf bash install.sh # add local HF embeddings (heavy: torch)

It rsyncs to /opt/learner/v1 (venv .venv), pip installs the package, chmod -R a+rX so a leashed user can run it, writes the wrapper /usr/local/bin/learner, and seeds config at /etc/learner/learner.yaml (never overwrites an existing one — new defaults go to learner.yaml.example). Overrides: LEARNER_INSTALL_{PREFIX,CONFIG,BIN_DIR,PYTHON,EXTRAS}. Uninstall: sudo bash uninstall.sh (preserves config unless LEARNER_PURGE=1).

Configuration — /etc/learner/learner.yaml or ~/.learner/learner.yaml

Load order (first existing wins): /etc/learner/learner.yaml, then ~/.learner/learner.yaml; --config <path> overrides per command. learner init writes ~/.learner/learner.yaml.

data_dir: ~/.learner            # holds knowledge_base.sqlite3

llm:
  endpoint: direct              # "direct" | "muzzle://<host:port>"
                                #   muzzle://h:p -> routes as an OpenAI-compatible endpoint
  chat_provider: ollama         # ollama | openai | anthropic
  chat_model: gpt-oss:20b
  base_url: http://127.0.0.1:11434

embeddings:
  backend: ollama               # ollama | hf | openai
  model: nomic-embed-text
  base_url: null                # ollama backend falls back to llm.base_url when null

Resolution: endpoint: muzzle://… → ChatOpenAI(base_url=<listener>) regardless of provider. Else by chat_provider: ollama→ChatOllama, openai→ChatOpenAI (needs OPENAI_API_KEY), anthropic→ChatAnthropic (needs ANTHROPIC_API_KEY). Embeddings by backend: ollama / hf (lazy; needs hf extra) / openai.

The CLI (complete surface)

learner <cmd>. All commands except init accept --config.

--manifest is JSON {"topic": "...", "resources": [{"title","kind","value", ...}]}. Ingest is idempotent (unchanged-by-checksum sources are skipped). build-skill exits 1 if the topic has no knowledge items (ingest without --no-knowledge first).

Common tasks (recipes)

learner init

# Ingest sources
learner ingest --topic espresso --kind article_url --resource "https://example.com/dialing-in" --refine
learner ingest --topic espresso --kind file_path  --resource ~/docs/barista-guide.pdf
learner ingest --topic espresso --kind video_url  --resource "https://youtu.be/VIDEOID"
learner ingest --topic espresso --resource "Puck prep: WDT then tamp level."   # default kind=text
learner ingest --manifest ./espresso.manifest.json
learner ingest --topic espresso --kind text --resource "..." --no-knowledge     # chunks/embeddings only

# Use the KB
learner query --topic espresso --question "why is my shot sour?" --limit 8
learner rag-answer --topic espresso --question "steps to dial in a new bag"

# Produce a Sentinel skill/playbook
learner build-skill --topic espresso --out ./out          # ./out/<slug>.skill.md
learner build-skill --topic espresso --out ./out --zip    # ./out/<slug>.skill.zip

learner export --topic espresso --out ./espresso.bundle.json
learner refresh --topic espresso        # re-index after a model/chunking change
learner list

# Point at OpenAI: chat_provider: openai, base_url: <openai-compatible>, embeddings.backend: openai,
#   embeddings.model: text-embedding-3-small; export OPENAI_API_KEY. (No 'openai' extra needed — see gotchas.)
# Route through Muzzle: llm.endpoint: muzzle://127.0.0.1:8090

Key paths & data store

/etc/learner/learner.yaml (or ~/.learner/learner.yaml) · /opt/learner/v1 (venv) · /usr/local/bin/learner · KB: <data_dir>/knowledge_base.sqlite3 (tables sources, chunks, knowledge_items; vector search is brute-force cosine in Python — fine for modest KBs, O(n) per query). Env: OPENAI_API_KEY, ANTHROPIC_API_KEY (fall back to a placeholder for local/muzzle). Knowledge-item kinds: rule | workflow | definition | warning | example | note. Chunking is chunk_size=1800, overlap=250 (hardcoded, not config-exposed).

Gotchas & failure modes

• Requires a live backend. With defaults, an Ollama server at 127.0.0.1:11434 with nomic-embed-text + gpt-oss:20b must be reachable, or ingest/query/build-skill fail.
• The hf backend pulls torch + sentence-transformers — heavy. Base install stays slim.
• No openai extra exists. README/HOWITWORKS mention learner[openai] / LEARNER_INSTALL_EXTRAS=openai, but pyproject.toml declares only hf and dev. The openai/langchain-openai clients are already base deps, so embeddings.backend: openai works with no extra — but pip install 'learner[openai]' / EXTRAS=openai errors.
• build-skill needs knowledge items — ingesting with --no-knowledge leaves none.
• refresh re-fetches remote sources (re-download + re-embed; text sources reuse stored raw text) — network-dependent.
• muzzle:// endpoint maps to an OpenAI-compatible client; Sentinel deliberately does NOT use it (its Muzzle listener is Ollama-flavored) — it points Ollama's base_url at the Muzzle listener via endpoint: direct instead.
• YouTube ingest needs available captions; article extraction is heuristic (BeautifulSoup, truncated ~120k chars). Single-writer SQLite — no concurrency guarantees.

Integration with Sentinel

Learner is standalone (needs neither Muzzle nor Sentinel to run), but integrates two ways:

1. Static skill import. learner build-skill emits a SKILL.md (frontmatter {name, description} + body; .zip = single top-level SKILL.md) matching Sentinel's importer. Import via POST /api/skills/upload.
2. Sentinel Learner add-on (sentinel/addons/). Sentinel's control plane downloads + sha256-verifies + installs Learner, auto-seeds the leashed user's ~/.learner/learner.yaml pointing chat+embeddings at Sentinel's Muzzle listener (endpoint: direct, chat_provider: ollama, base_url: <muzzle_endpoint>), and drives learner as the leashed user (scrubbed env, under Leash + Muzzle). It exposes a learner_query MCP tool (learner MCP server, read-only auto) so an agent granted the topic's skill can retrieve KB excerpts and answer from them (true RAG). The web console (/api/addons/learner/{ingest, query,build-skill,rag-answer,topics}) lets an operator ingest, query, RAG-answer, and one-click build+register a skill straight from the browser.

Quick reference

Config:  /etc/learner/learner.yaml or ~/.learner/learner.yaml   (learner init)
KB:      <data_dir>/knowledge_base.sqlite3    (default ~/.learner/)
Ingest:  learner ingest --topic T --kind {article_url|video_url|file_path|text} --resource R [--refine]
Ask:     learner rag-answer --topic T --question "..."     Search: learner query --topic T --question "..."
Skill:   learner build-skill --topic T --out ./out [--zip]   -> import into Sentinel via /api/skills/upload
Backend: needs Ollama (nomic-embed-text + gpt-oss:20b) or configure openai/anthropic