# The AI Assistant

FaustWave ships an in-app AI assistant pane with full read + write access to the IDE. It's not a chat sandbox — when you ask it to add a node, it adds the node. When you ask it for a chord progression, it drops the progression into your sequencer. Every action it takes lands in the Activity log so you can audit + (where applicable) undo. This section covers the user-facing side; for what the assistant sees + how external clients can drive the same surface, jump to § 3 (MCP).

## Opening the assistant

```yaml
actions:
  - action_id: chat.panel.toggle
    input: { open: true }
    label: "▶ Open the Assistant panel"
  - action_id: settings.open
    input: { section: ai-provider }
    label: "▶ Jump to provider setup (endpoint / model / key)"
```

Press **Ctrl+L** (Cmd+L on macOS), click the **Sparkles** icon in the topbar right cluster — or the first button above. The Assistant pane docks to the right of the workspace.

If you haven't configured a model + API key yet, the pane prompts you. Click **Open Settings** (or the second button above) to jump to the configuration section.

> 🟦 **NEW TO THE PATTERN?** Anthropic's Claude, OpenAI's GPT, and most other modern LLM APIs share a "tool use" protocol: you give the model a catalog of tools it can call, and it returns either a text reply or a tool call. FaustWave's Assistant passes the entire action registry as tool definitions, so the model can do anything you can do. No "limited subset", no permission tier — same surface, different operator.

## Bring your own key

FaustWave does NOT host its own LLM. You bring your own provider + API key. Two reasons: (1) latency to a self-hosted model means seconds-long pauses on every tool call, ruinous UX; (2) your key, your data, your spend.

Supported providers:

| Provider | Status |
|---|---|
| **Anthropic Claude** | First-class — recommended for tool-use richness |
| **OpenAI** | Supported — uses the chat/completions tool-use protocol |
| **Any OpenAI-compatible endpoint** | Works — point at the URL + key (Groq, Together, local Ollama with the OpenAI compat shim, etc.) |

Configure via Settings → AI Assistant:

- **Endpoint** — Anthropic's default URL, or your OpenAI-compatible URL.
- **API key** — pasted, encrypted at rest in `app.json`.
- **Model** — the model id (e.g. `claude-sonnet-4-6`, `gpt-4o`, `llama-3.1-70b-instruct`).

You can swap models mid-conversation; the next turn uses the new model.

## What the assistant sees

Every turn, FaustWave injects a system prompt + tool catalog into the model's context. The system prompt is composed of fragments contributed by each loaded extension:

```
<core extension prompt> ← project, FaustWave conventions, key idioms
<dsp-builder prompt>    ← node-kinds reference, MIDI conventions, mono/poly
<sequencer prompt>      ← track + slot + scale + chord-progression API
<keyboard prompt>       ← keyboard-source idioms
<hub prompt>            ← publishing + installation workflows
<recorder prompt>       ← recording + recordings library
```

Each fragment is short (< 1 KB typically) — they're not embedded reference content, just operational hints. For deeper context, the assistant searches the local Knowledge Base — your installed books, including this manual.

The tool catalog is the curated set of hot-path actions (~25 today, well under the 128-tool cap) plus the `palette.*` meta-actions for reaching the rest of the ~130-action registry. See § 3 for the discipline.

The dsp-builder prompt fragment also teaches the assistant about per-kind library imports — kinds from `faust-vintage` declare `imports: ["tubes.lib"]` / `["tonestacks.lib"]` / `["vaeffects.lib"]`, and `faustGen` collects the union + emits `import("…")` lines after the default `stdfaust.lib` header. The assistant doesn't have to manage imports — it just adds the kind and the generator does the right thing. See **FaustWave — Node-Pack Authoring** § 3 for the plumbing.

## The tool-use loop

A turn looks like:

1. **You type a message** ("add a 4-voice poly sine to my Builder + sequence A minor over 16 steps").
2. **Assistant receives** your message + the system prompt + the tool catalog + recent message history (up to a context-length cap).
3. **Model responds** with either:
   - A text reply ("here's what I'll do…").
   - A list of tool calls (e.g. `builder.graph.node.add { module_id, kind: "osc" }` + `builder.graph.connect { ... }` + ...).
4. **FaustWave executes** each tool call through the action registry (same path you'd hit via UI or palette), captures the result, feeds it back to the model.
5. **Model responds again** with text or more tool calls. Loop until text-only response, or the model signals done.

The loop streams as it happens — you see each tool call land in the **Activity** rail panel (right rail) in real time, and the assistant's text turns stream token-by-token.

Everything the assistant writes is copyable: every fenced code block in a reply carries its own **copy button** (top-right of the block), and each assistant message has a **copy-as-markdown** button that grabs the whole reply as raw markdown — handy for moving a generated Faust snippet into the `.dsp` editor or a JSON blob into a node-pack manifest.

## Activity — audit + undo

Open the **Activity** rail panel from the right rail (it ships seeded; if you removed it, `+ Add` → *Activity*). It shows every action that flowed through the registry — your clicks, palette runs, AI tool calls, MCP calls — with input, outcome, timing, and result / error.

Each entry shows:

- **Timestamp**.
- **Surface** — `mcp` for AI / external MCP, `palette` for your gestures, `controller` for a controller binding fired by a MIDI trigger (a pad, a button).
- **Action id** + input.
- **Outcome** (`completed` / `failed`) + duration (ms).
- **Result** (success path) or error (failure path).

For a multi-step AI session, the entries land in execution order. You can scroll back, see exactly what happened, and:

- **Re-run** an action with the same input (handy for "do that again").
- **Copy as MCP** the action id + input, so you can replay it from a script.
- **Filter** by surface (mcp / palette / controller) and outcome (completed / failed) to find issues.

> 🟦 **THE AUDIT FLOW**: this is FaustWave's answer to "how do I trust an AI editing my files?" — the AI can't do anything the user can't see + audit. Sequencer + DSP edits have first-class undo (`sequencer.undo`, undo-able dsp actions); other actions you reverse by reapplying the inverse. If a session went sideways, scroll the Activity log back, identify the bad step, reverse from there. No black box.

## Common patterns

### "Build me a DSP"

Describe the sound. The assistant adds nodes + wires them.

> User: build me a fat saw lead in poly mode, 8 voices, with an envelope and a lowpass with some resonance
>
> Assistant: I'll set up a `saw` source → `adsr` envelope → `moog_vcf` lowpass → `output`, in 8-voice poly mode. Adding nodes…
> *(executes `dsp.polyphonic.set { voices: 8 }`, `builder.graph.node.add` ×4, `builder.graph.connect` ×4)*

Five seconds, working DSP.

### "Tweak this"

You're hearing the DSP + want a change. Ask for it.

> User: lower the cutoff a bit and bring up the resonance to something dramatic
>
> Assistant: dropping cutoff to 600 Hz and resonance to 0.85.
> *(`builder.graph.node.param.set` ×2)*

The change is audible immediately (params live-write to the running engine).

### "Suggest a progression"

The KB has 8 curated progressions with theory metadata. The Suggest Progression UI surfaces them; the AI does too.

> User: drop a sad, 4-chord progression in A minor over my pattern
>
> Assistant: that fits the "i-vi-iv-v" loop in minor — A minor, F major, D minor, E minor. Inserting at steps 0/4/8/12 of the current sequencer.
> *(`sequencer.progression.insert { progression_id: "i-vi-iv-v", key: "A", distribution: "bar" }`)*

### "Walk me through X"

The assistant searches the KB before answering, so it can ground itself in the actual project + bundled theory + node reference. Ask things like *"what's the difference between a `moog_vcf` and an `lp` node?"* — it pulls the relevant node-pack entries and answers with the actual schemas.

## KB books — research the assistant keeps

Ad-hoc answers evaporate when the conversation ends. For research worth keeping — "how does FM synthesis actually work", a deep-dive on a genre's drum programming, the harmonic analysis of a song you're covering — turn it into a **KB book**: a local book the assistant (and you) can recall in every future session.

The flow:

1. **Research** — ask the assistant (or any external MCP client) to dig into the topic.
2. **Write the book** — save the findings as a project `.md` file: one `# Title` heading, then `## Section` headings per chapter. Tutorial `actions:` YAML blocks work here too, so a book can carry playable chains.
3. **Install** — `kb.book.install { file_path }`. No Hub, no auth, no network — the file indexes straight into the local KB. The pack id derives from the title as `local-<slug>`; re-installing the same book replaces its previous sections (clean update, nothing lingers).

```json
{ "action_id": "kb.book.install", "input": { "file_path": "asset:assets/fm-synthesis-notes.md" } }
```

(No button for this one — it needs YOUR file: a project ref like above, or an absolute OS path for a file outside the project. Ask the assistant to run it, or fire it via `palette.run`.)

Once installed, the book behaves like any Hub-installed book: it appears in the Knowledge rail's *Installed Packs* list, opens in the Book Reader, and its sections answer `kb.search` — so next week's *"what was that sidechain trick again?"* grounds in YOUR book, not a fresh guess. Remove with `kb.book.uninstall { book_id }`; share it later with `hub.publish kind: "book"` using the same file.

## Scoping the assistant to a section

When you open a section in the **Book Reader**, the Reader auto-pushes that section as the assistant's primary context. The next time you ask a question while reading **FaustWave — Builder** § 1, the answer is grounded in that section first, then the rest of the manual, then the global corpora.

You can pin a scope explicitly via the MCP `kb.context.scope.set` action — useful for a tutorial-walkthrough scenario where you want every question scoped to one chapter. `kb.context.scope.clear` releases the pin; the assistant returns to global grounding.

## Limits + things to know

- **Context length** — the session history the model sees is capped: the system prompt plus the last 50 messages; older turns are dropped. The message the assistant is currently working on is never dropped, however many tool calls it takes. Tool results always reach the model in full — nothing is truncated.
- **Response length** — *Settings → AI Provider → Max response tokens* (default 8192, minimum 256) is the output budget per request. Pay-per-use providers reserve it against your credit up front; reasoning models spend it on thinking too, so raise it if replies get cut off.
- **No tool retries** — if a tool fails (e.g. wrong `module_id`), the error feeds back into the model and it can correct. You see the failed entry in the Activity log.
- **No daemon mode interaction** — the Assistant pane is a renderer surface. For headless / scripted automation use MCP via Claude Desktop or a custom client; see § 3.
- **Cost** — the assistant pays for tokens at YOUR provider's rate. Long sessions with the full tool catalog can run dozens of cents per turn at Claude rates. Use a smaller model (haiku, gpt-4o-mini, an open 70B) for cheaper iteration.

## The hard parity rule

This is the rule that makes the assistant trustworthy as a working partner:

> Every UI gesture in FaustWave routes through `api.actions.run(action_id, input)`. The Assistant + the Command Palette + the MCP server all expose the same registry. There's no "AI-only" surface, no "human-only" surface — what you can do, the AI can do. What the AI can do, you can replay manually.

The registry is the single source of truth for what's possible in the IDE. The lint that enforces this (`tools/check-ai-parity.mjs`) grep-checks that every action handler is reachable from both paths and runs on every PR.

## Where to go from here

- **§ 2 (The Command Palette)** — the human-side surface to the same registry.
- **§ 3 (MCP — external clients)** — same surface, different transport (Claude Desktop, custom scripts, daemon mode).
- **FaustWave — Reference** for the full keyboard-shortcuts catalog.